The mapping and aggregation of item analysis
results continues to drive MS 150 assessment. This is the first term
that I've tried tackling item analysis in MS 150 for all instruments.
The work load is considerable. Coupled with physical science, grading
and analysis typically requires working Friday evening and throughout
the day on Saturday. Data entry usually occurs on Sunday morning. I
have not been logging the time consumed, but it is something like
twelve hours of work each weekend.
That said, the results do provide insights I can use in the classroom.
One result was that I spent an extra day on linear regressions and
added them to an additional quiz, with very positive results.
Q | Description | l | Outref | SLO | Corr | Corr% |
1 | linear, non-linear, or random | 5 | 5 | Perform a linear regression and make inferences based on the results | 48 | 100% |
2 | slope | 5 | 5 | Perform a linear regression and make inferences based on the results | 45 | 94% |
3 | intercept | 5 | 5 | Perform a linear regression and make inferences based on the results | 43 | 90% |
4 | correlation r | 5 | 5 | Perform a linear regression and make inferences based on the results | 45 | 94% |
5 | positive or negative relation | 5 | 5 | Perform a linear regression and make inferences based on the results | 48 | 100% |
6 | strength of relationship | 5 | 5 | Perform a linear regression and make inferences based on the results | 45 | 94% |
7 | coefficient of determination | 5 | 5 | Perform a linear regression and make inferences based on the results | 42 | 88% |
8 | perc var x that explains var y | 5 | 5 | Perform a linear regression and make inferences based on the results | 31 | 65% |
9 | predict y given x | 5 | 5 | Perform a linear regression and make inferences based on the results | 26 | 54% |
10 | predict x given y | 5 | 5 | Perform a linear regression and make inferences based on the results | 22 | 46% |
11 | probability | 1 | 1 | Calculate basic statistics | 38 | 79% |
12 | probability | 1 | 1 | Calculate basic statistics | 9 | 19% |
14 | three ways to deter. prob. | 1 | 1 | Calculate basic statistics | 6 | 13% |
Fall 2005 fx | Fall 2006 fx | Fall 2007 q04 | sSLO |
95% | 29% | 94% | Calculate the slope of the least squares line. (Calculate) |
95% | 92% | 90% | Calculate the intercept of the least squares line. (Calculate) |
65% | 92% | 100% | Identify the sign of a least squares line: positive, negative, or zero. (Define) |
53% | 90% | 54% | Solve for a y value given an x value and the slope and intercept of a least squares line. (Solve) |
45% | 48% | 46% | Solve for a x value given an y value and the slope and intercept of a least squares line. (Solve) |
91% | 37% | 94% | Calculate the correlation coefficient r. (Calculate) |
82% | 92% | 94% | Use a correlation coefficient r to render a judgment as to whether a correlation is perfect, high, moderate, low, or none. (Interpret) |
98% | 42% | 88% | Calculate the coefficient of determination r². (Calculate) |
73% | 94% | 65% | Calculate amount of variation in dependent variable explained by variation in independent variable. (Calculate) |
76% | 63% | 100% | Based on analysis, infer whether a linear regression is appropriate? (Interpret) |
Outref | Students will be able to: | Sum | Count | Avg |
1 | Calculate basic statistics | 21.62 | 40 | 54% |
2 | Represent data sets using charts and histograms | 2.96 | 8 | 37% |
3 | Solve problems using normal curve and t-statistic distributions including confidence intervals for means and hypothesis testing | 0 | 0 | 0% |
4 | Determine and interpret p-values | 0 | 0 | 0% |
5 | Perform a linear regression and make inferences based on the results | 16.3 | 25 | 65% |
PSLO | define mathematical concepts, calculate quantities, estimate solutions, solve problems, represent and interpret mathematical information graphically, and communicate mathematical thoughts and ideas. | 31.26% |
Bear in mind that 100% accomplishment is not possible. The aggregate
performance includes all performance to date. Poor performances early
in the term will "weigh down" the average throughout the term. To
remedy this, I run a separate aggregation analysis on the final
examination alone.
Q | Description | Corr | Corr% |
1 | calculate pace | 10 | 32% |
1 | units | 9 | 29% |
1 | significant digits correct | 10 | 32% |
2 | calculate velocity | 3 | 10% |
2 | units | 3 | 10% |
2 | significant digits correct | 2 | 6% |
3 | calculate momentum | 0 | 0% |
3 | units | 0 | 0% |
3 | significant digits correct | 0 | 0% |
4 | calculate kinetic energy given formula | 3 | 10% |
4 | units | 2 | 6% |
4 | significant digits correct | 1 | 3% |
5 | temperature of boiling water? | 30 | 97% |
6 | temperature of healthy human body? | 21 | 68% |
7 | what happens at zero degrees celsius? | 23 | 74% |
8 | best heat conductor in lab? | 22 | 71% |
9 | best heat insulator in lab (terminology issue) | 10 | 32% |
10 | list three ways to "move" heat | 26 | 84% |
11 | explain the three ways heat is "moved" | 2 | 6% |
1. [d] Data tables, charts, graph, display, analysis | |
3 | Very well laid out, communicated well |
2 | Minor issues or somewhat incomplet or missing an appropriate element |
1 | incomplete or unclear or confusing or diagrams without axis labels or that otherwise do not make sense |
0 | Omitted |
2. [c] Conclusions | |
4 | Strong reasoning based on the data, logically reasoned, complete and thoughtful |
3 | Moderately well-reasoned, but tangential to the data or based on misconceptions |
2 | Weak reasoning or overly brief or compromised and impaired by grammar and syntax errors |
1 | Unclear, not well reasoned, or highly incomplete, or unusually weak |
0 | Omitted |
[G] Grammar and Syntax | |
3 | No errors of grammar or word order. |
2 | Some errors of grammar or word order but communication not impaired. |
1 | Frequent errors |
0 | Errors of grammar or word order so severe as to make comprehension virtually impossible. |
[V] Vocabulary | |
3 | Appropriate terms used consistently, clear command of vocabulary with a focus on correct usage of physical science vocabulary, no misspelled words. |
2 | Occasionally uses inappropriate terms or relies on circumlocution; expression of ideas not impaired; or a few misspelled words. |
1 | Frequent errors in vocabulary or spelling, ability to communicate limited by vocabulary |
0 | Vocabulary limitations so extreme as to make comprehension virtually impossible. |
[O] Organization | |
3 | All sections present in the right order |
2 | One section missing or out of order |
1 | Multiple sections out of order, or turned in as separate spreadsheet and word processing docs |
0 | Only one section or no sections apparent. |
[C] Cohesion | |
3 | Ideas flow logically. Connector words assist the reader. |
2 | Ideas are disconnected and the conclusion reads more like an outline or answers to a list of questions without connector words. There is a choppy and disjoint sense to the writing style. |
1 | Communication impaired by a hodgepodge of inappropriate and misused cohesive structures |
0 | Incomprehensible collection of disconnected ideas and words, or only one sentence. |