[assessment] MS 150, SC 130, SC/SS 115 update

SC 130 Physical Science

This past week the lecture focused on time and place. The quiz also focused on time and place, with some interesting results. A traditional physical science quiz on time and place would tend to focus on having the students answer specific questions of fact. This quiz sought to examine how the students know something and how they think they should know that something. So the quiz started with the odd question, "What time is it?" From the student's perspective this is an odd question. Students tend to focus on getting the "right answer" but this question has no single right answer.

The second question was "Is that the right time?" Here I am asking the student to engage in metacognition, to engage in thinking about their thinking in the form of thinking about whether their answer is correct. The lecture section should have prepared the thoughtful student to answer no if they used a local time source such as their watch or the wall clock.  A couple students knew that Telecom was synchronized to a global time standard and used their cell phones to obtain the time. In this case the answer would be yes, their time is the "right" time for planet earth. Yes, students in the class are encouraged to use technologies including their cell phone, provided that usage is appropriate to the task at hand in the class.

Question three asked, "How do you know?" This question was the one that let me know if a student had checked telecom. One thing that I learned along with the class is that the time on a cell phone is not necessarily the correct time. Many cell phones have internal clocks that are set by the user. To get the "Telecom" time one texts to get their balance and that the returning message includes a time stamp.

The quiz continued in this vein of asking a fact and then asking how the student knows that fact. This proved very illuminating. A student might get a fact right but for the wrong reason. Individual quizzes showed clearly that students were able to parrot back a fact with either no understanding of how that fact is actually obtained or with a faulty understanding.

This meant that the right answers to the traditional "just the fact" questions at the end of the quiz cannot be taken as proof of learning. To the question "What causes the seasons on the planet earth," 22% of the students correctly answered that the tilt of the earth is responsible for the seasons. Yet some of those 22% are likely just parroting back a fact without true understanding.

This disjunction between memorized content and the actual model of a system being held in the brain of the student is made most clear in the recent laboratory on heat conduction where many student's explanations of the phenomenon is confused at best.

The distinction between memorized content and conceptual understanding is clearest to me in the series of the following series of questions:

Only 26% of the students knew their latitude and longitude.  Bear in mind that this was covered on two separate occasions, Monday and Wednesday, by going outside with a GPS, determining our latitude and longitude, and writing this on the white board. I was only looking for answers to the nearest degree, not the nearest minute. Yet only 26% could answer that we are at 6º North latitude and 158º East longitude. However, 59% could tell me HOW to determine the correct latitude and longitude. For that question I allowed answers that referred to the GPS unit and answers that referred to using technologies such as Google Earth. The students do better on how to get an answer than the answer itself, and that is for me part and parcel of the impact of deep learning versus surface learning. My hope is that the students will learn how to learn. Yes, content is important, but it is also ultimately what is most forgotten.

The content level is always a source of some humor on any science quiz, and SC 130 is no exception. There is some content I do want my students to know. Like the number of days in the year. Three people missed this question, citing 356 days (obvious digit reversal), 163 days, and 135 days. That any college student would not know how long the year is in days has to be indicative of the lack of contextual knowledge our students bring to any science course.

Other interesting student theories included that a solar day is "A day that changes things around and it is also a day of partying and enjoying." Actually, a solar day is 24 hours, noon to noon technically.

Many of my labs use the simplest of equipment. Yes, there is some fancy physics equipment in the laboratory. I always fear, however, that students will think that the physical law being illustrated holds only for some fancy piece of special equipment. Science as magic. For heat labs what I really want is an uhm, but the campus lacks one of these. Students really understand an uhm. As an example of technology as magic, one student noted, "Using the GPS, you face out and let the solar system out in space to give you the approximate latitude and longitude of where you are." 

One student clearly lives in a very uncertain pre-Copernican world noting that the cause of the seasons is "the sun is orbiting the earth. Sometimes the sun rise east and sometimes it set west." Only sometimes. Not only is science magic, their whole daily world runs on a sometimes, a maybe the sun will rise in the east. Of course, with the sun orbiting the earth, lots of things are possible.

The item analysis for all items on the quiz is as follows:

Laboratory five was turned in this past week, the heat conduction laboratory. Images from this laboratory can be seen at lab05photos.html

Overall performance on the laboratory reports continues to rise, with the average on the rubric rising to 80% at present. The students have, by and large, mastered the format I have specified. Students did very well on the basic analysis of what happened in the heat conduction laboratory. Their tables and charts were generally spot on. Their conclusions, however, where they tried to explain what happened, betrayed a mix of many different mental models. I do not try to correct all of these models as that would be simply my word against their deeply held understanding. I could opt to design experiments that knock down the incorrect explanations, but we would spend a significant part of the term on one single concept (heat conduction in this case). I may one day decide that is what should be done, but that would lead to very deep and very narrow learning and even I am not ready to go there quite yet.

On the writing side grammar issues that are probably connected to L1 remain problematic. Students simply have a very hard time with subject-verb agreement between the singular and the plural. For example, the constructions "Copper conducts heat" and "The metals conduct heat." The L1s which I am familiar with do not have have singular and plural verb constructions, so this remains an area of difficulty for the students. I am not an English instructor, so my approach may be wrong, but I often try to shake the students loose from their ideas by saying that "s" is not plural, it is simply required in all present tense sentences. For the singular "s" attaches to the verb, for the plural "s" attaches to the noun. This is of course not universally true, but it is a starting point.

The other advise I will be giving to the students is to simplify their sentences. Stick to noun-verb-noun sentences. Avoid commas. Avoid conjunctions. Too often the student seems to get lost in a sentence, "The reason why the convection [sic] of heat starts from the hot water to the cool water is because the hot water is bigger than the cool water but, if the hot water is smaller than the cool water, we have to put the thermometer in the hot water."

Bear in mind that the student may be referring to either the amount of water or the temperature of the water when the student says "bigger" and "smaller." As to why the thermometer would be put in the hot water I remain unclear as the lab procedure did not call for monitoring of the hot reservoir.

I have to admit to enjoying the explanation by one student, "The heat is transfer by conduction of the hot water in the foam cup through the two copper wires to the other foam cup." The copper wires were solid six gage copper wire, a thick solid cylinder of copper used in electrical systems.

While marking the laboratories remains a significant effort each and every weekend, I find I am gaining too much insight into the abilities and thinking of my students to give up this approach. A fill-in the blank recipe approach would mark quickly and easily, but would be as meaningful as the dry facts spit back on a test. As one alumni said to me, "I do not remember what we did or learned, but I remember filling in the blanks in the laboratory." The alumni noted that they did well in the course, they put the right numbers in the right blanks, but they never really understood what the numbers meant.

The laboratories are also a lot of work for the students - a paper a week. The turn-in rate for laboratory five was 83% of the students who participated in the laboratory. There was one absence on lab day, there is also now apparently a withdrawal from the course by another student. The laboratories continue to be marked for grammar, vocabulary, organization and cohesion. On lab five 50% of the 24 points were on content items in the rubric and 50% were on grammar, vocabulary, organization, and cohesion. Writing remains and will remain a crucial component of the laboratories.