Assessment preview fall 2007

19 August 2007

The following pretest assessment data is based conceptually on aggregation of course level student learning outcomes, the model proposed in the Student Learning Outcomes Assessment Plan. In this model, course level measurable skills and abilities are aggregated to report on accomplishment of program learning outcomes.

Course level internal outcome assessment data was always deemed to be insufficient and required triangulating data in order to determine whether a program was producing students who could perform the skills post-graduation. The present design of programs, courses, and articulation issues currently preclude the type of in situ assessment that schools such as Kapiolani Community College are deploying. I suspect that environmental differences will always complicate in situ assessment. Kolonia is simply not as large nor as diverse a community environment as Honolulu.

Until such time as alternate tools may be developed to perform assessment at the course level in the classroom, I will continue to employ item analysis as one way to gather data on student performance.

Students entering MS 150 Statistics include repeating students. The pretest covers basic statistics and linear relationships. In the table below, the fourth column (description) describes the concept tested by the pretest question. The seventh column (SLO) notes the outline outcome to which the question maps. The last two columns note the number of students who answered correctly and the percentage of students answering correctly. For example, 81% of the students correctly determined the minimum value in a given data set.

The above questions map to the outline and thus a report on average performance on the proposed outline can be determined. This is shown in the following table:

The last row above is the current mathematics program student learning outcome and average aggregate performance on that outcome.

SC 130 Physical science will utilize basic statistical concepts during the course. Uncertainty in a particular measurement can be reduced by repeated measurements. Precision of a measurement is often specified by the standard deviation of a set of measurements about their mean. Many simply physical systems obey linear relationships and a linear regression can be used to find the relationship. All of these concepts will play a role in the course.

While some stateside college students might arrive at college unable to calculate a standard deviation, many are now familiar with mode, median, and mean due to their inclusion in textbooks since the revision of school standards arising from the 1989 NCTM standards for mathematics documents. I cannot make this presumption, hence the pretest focused more on statistical calculation skills and not science skills. The pretest also included two problems that involved mixed time units (hours and minutes) which had to be converted to decimal hours to be solved. This was intended to provide insight into the students' abilities to work with units.  The results are in the following table.

Of some interest in the above table is the high performance since on basic minimum and maximum questions, mirroring the statistics results. Students appear to grasp the meaning of the words minimum and maximum. The remaining results are weak and indicate little contact with the statistical concepts that physical science laboratories presume.

Although the aggregation approach of the student learning outcomes assessment plan does not map directly to the introduce, practice, and demonstrate/mastery model, ultimately whether a particular outcome was introduced, practiced, or mastered will have to be measured. The education division seems to be the farthest along with the I,P,D/M model.

The education division has clearly chosen tests external to courses to determine student performance on their program student learning outcomes. How measurements internal to courses will interact with the external measurements remains something I am interested in seeing. I suspect that over time instructors in education will find their course performance implicitly measured by how well their students due on the external tests. The tests will eventually drive the courses. This is not necessarily a bad thing. If the tests are well-aligned to skills and content knowledge needed in the classroom, then this drive can be a positive one.

Physical science is primarily a general education core course. At best it can only introduce concepts. Many majors will never again take a laboratory science, hence there is no possibility of an assessment "stack" where students go on to practice and show mastery. The skills may also not be of obvious value post-graduation. Determining whether a business major graduate working at midtown video renting out DVDs is  "Performing experiments that gather scientific information and utilizing, interpreting, and explaining the results of experiments and field work in a field of science," is not be something that will be observable let alone measurable.