MS 150 Statistics spring 2009 assessment

College mandated assessment report format

Review of performance: MS 150 Spring 2009. 56 students enrolled in course. Submitted by Dana Lee Ling

nSLOProgram SLOI, D, MReflection/comment
1Identify levels of measurement and appropriate statistical measures for a given leveldefine mathematical concepts, calculate quantities, estimate solutions, solve problems, represent and interpret mathematical information graphically, and communicate mathematical thoughts and ideas. M52 of 56 students were successful on this SLO based on an item analysis of the comprehensive final examination
2Determine frequencies, relative frequencies, creating histograms and identifying their shape visuallyM42
3Calculate basic statistical measures of the middle, spread, and relative standingM48
4Perform linear regressions finding the slope, intercept, and correlation; generate predicted values based on the regressionM35
5Calculate simple probabilities for equally likely outcomesM48
6Determine the mean of a distributionM48
7Calculate probabilities using the normal distributionM48
8Calculate the standard error of the meanM48
9Find confidence intervals for the meanM37
10Perform hypothesis tests against a known population mean using both confidence intervals and formal hypothesis testingM30
11Perform t-tests for paired and independent samples using both confidence intervals and p-valuesM23

Assessment report

Performance in MS 150 Statistics is measured by quizzes and tests throughout the term. A comprehensive final examination consisting of fifty fill-in-the-blank questions was administered. The fifty questions each map back to a course level student learning outcome. Performance against the outline proposed in 2008 has been measured for the past two terms and provides a basis for comparison across the terms. The following table indicates the average percent of the number of student who answered questions in a particular area correctly.

OutnumStudents will be able to: Sp 08Fa 08Sp 09
1Identify levels of measurement and appropriate statistical measures for a given level78%82%94%
2Determine frequencies, relative frequencies, creating histograms and identifying their shape visually83%84%75%
3Calculate basic statistical measures of the middle, spread, and relative standing90%80%86%
4Perform linear regressions finding the slope, intercept, and correlation; generate predicted values based on the regression80%79%63%
5Calculate simple probabilities for equally likely outcomesN/AN/AN/A
6Determine the mean of a distributionN/AN/AN/A
7Calculate probabilities using the normal distributionN/AN/AN/A
8Calculate the standard error of the mean84%84%87%
9Find confidence intervals for the mean66%85%67%
10Perform hypothesis tests against a known population mean using both confidence intervals and formal hypothesis testing36%56%55%
11Perform t-tests for paired and independent samples using both confidence intervals and p-values55%63%43%
12Go beyond outline33%16%
PSLOdefine mathematical concepts, calculate quantities, estimate solutions, solve problems, represent and interpret mathematical information graphically, and communicate mathematical thoughts and ideas. 72%76%69%

Sp 08 refers to performance on the spring 2008 final examination, Fa 08 to performance on the fall 2008 final examination, and Sp 09 refers to performance on the current term final examination. Performance marked N/A indicates that the material was not directly performed on the final examination. Material in those sections was tested during the term and is foundation material to that tested in later chapters.

Overall performance levels remained stable in each area, fairly typical for a mature course that is in its ninth year and nineteenth running. There are indications that performance on the final this term was slightly lower than previous terms. Of note was that the 8:30 section, which took the final on Tuesday 05 May attained an average of 71% while the the 9:30 section which took the exact same test two days later attained a 67% average. The expectation would be that the 9:30 students would have a couple days to determine the material on the final and prepare with foreknowledge. There is no indication that this occurred.

The final five questions on the final examination are from material beyond the outline. The students are unaware that this material will appear, the material is not covered in the textbook, and the material is deleted from the posted finals that students use to study for the examination. This material is a complete surprise to the students. The point of this material is noted on the final examination itself, "One intention of any course is that a student should be able to learn and employ new concepts in the field even after the course is over."

The 16% success rate of students on this material is not that encouraging. Although the material builds on ideas that the students have encountered, only one student in six successfully answered the questions in this material.

Making inferences

Students show strength in the ability to make basic calculations. The success rate on basic statistics was 86%, with a 75% success rate on making a histogram. The more complex operation of calculating a confidence interval saw a 67% success rate.

Using the underlying item analysis, the area of greatest weakness was rawing conclusions from the data, making inferences and correct interpretations of the results. Questions that required an inference, an interpretation of a result, had only a 40% success rate. The low performance on inferences has existed in previous terms, but as success rates on basic calculations have improved, the lower performance on inferences becomes increasingly apparent. Underneath the lower success rate is a lack of understanding as to the "meanings" behind the numbers. Improving student ability to make inferences is an area in which I have yet to find a way to improve student performance. This remains a challenge.

Projects

For a third term in a row students were asked to engage in putting together a basic statistical research project. The number of projects completed improved with the addition of an earlier (fourth week of class) initial commitment to a project idea. Writing on the projects also showed improvement, but this was not specifically analyzed this term.

Attendance

The college attendance policy will shift next fall from a single college-wide "one size fits all" policy to an policy of letting instructor's set their own attendance policy. With this impending change, there is an interest in noting present attendance patterns. MS 150 Statistics meets for an hour a day Monday, Wednesday, and Friday.

statisticabsences
sample size n56
min0
max16.5
range16.5
midrange8.25
mode1.3
median4.65
mean5.4
stdev4.12
Correlation-0.59

The above table represents 56 students who range from perfect attendance (0 absences) to 16.5 absences with a mean of 5.40 absences.

In my courses a late is a third of an absence. Since I run attendance as a point per day, a third of an absence would be "0.6666..." Thus I actually enter "0.7" making a late technically 30% of an absence. Extremely late (arriving half way through the period) causes a reduction to 50% of a point, hence the fractional mode and median.

Increasing number of absences is negatively correlated with performance, prior to administering the final examination the correlation r was -0.62, a moderate to moderately strong negative correlation. The higher the numer of absences, the lower the student's percentage (points) in the course. The linear regression suggests a loss of 3% in overall score per absence. The final examination decreased the correlation slightly to -0.59. The correlation is statistically significant.

Linear regression trend line attendance versus performance in course
Linear regression trend line attendance versus performance in course

Final examination score and grade distributions

The final examination score distribution provides another view of student comprehension of material in statistics. The fifty question final ranges from basic statistics to making inferences based on p-values for two samples. The following chart shows the distribution of the student papers as a function of the student's score. There were 57 points possible on 50 questions, no student attained more than 53 points.

Cumulative grade distribution
Cumulative grade distribution

Although not a direct assessment of learning, grade distributions do have meaning when coupled with in-course assessment that is aligned with the student learning outcomes on the outline. Questions on in-course quizzes and tests consist of questions that can each be mapped back to a student learning outcome on the outline. The bulk of the questions are valued at one point, with certain multiple-entry tables and charts being worth up to five points. Grades are then calculated from the points attained. While this is not matrix-level assessment which permits one to say, "Johnny can do x," I would argue the result is grades that have underlying meaning in terms of student learning. The course grade distribution is shown in the following chart.

Grade distribution
Grade distribution