10 May 2008
Back in Fall 2001 I had the opportunity to analyze my Instructor Evaluation by Student results. I have not had access to the raw data since that time, making a re-analysis difficult. Spring 2008, having never seen my evaluations from the fall of 2007, I decided to go ahead and run an evaluation and tally the data so as to run a comparison.
The questions are included in the table below.
Never | Rarely | Sometimes | Usually | Always | ||
---|---|---|---|---|---|---|
01. | Keeps regular schedule every class day | 1 | 2 | 3 | 4 | 5 |
02. | Shows interest in the subject | 1 | 2 | 3 | 4 | 5 |
03. | Gives individual help as needed | 1 | 2 | 3 | 4 | 5 |
04. | Available for student conference | 1 | 2 | 3 | 4 | 5 |
05. | Welcomes questions, suggestions, and discussions from students | 1 | 2 | 3 | 4 | 5 |
06. | Shows interest and respect for students | 1 | 2 | 3 | 4 | 5 |
07. | Helps the students in meeting individual learning needs | 1 | 2 | 3 | 4 | 5 |
08. | Uses classroom/lab time fully | 1 | 2 | 3 | 4 | 5 |
09. | Provides clear directions for assignment and instruction | 1 | 2 | 3 | 4 | 5 |
10. | Grades fairly and frequently | 1 | 2 | 3 | 4 | 5 |
11. | Makes the purpose of the course clear | 1 | 2 | 3 | 4 | 5 |
12. | Talks clearly and at an easy to follow pace | 1 | 2 | 3 | 4 | 5 |
13. | Lessons are well paced with activity as well as lecture | 1 | 2 | 3 | 4 | 5 |
14. | Makes the course interesting | 1 | 2 | 3 | 4 | 5 |
15. | Textbook was appropriate and helpful | 1 | 2 | 3 | 4 | 5 |
Averages were calculated for each question based on the above numbers. The following table rearranges the questions into descending rank order for spring 2008.
n | Question | Fall 2001 | Spring 2008 | Diff |
---|---|---|---|---|
01 | Regular schedule | 4.5 | 4.9 | 0.40 |
13 | Lessons well paced with activity | 4.7 | 4.84 | 0.14 |
02 | Shows interest in subject | 4.4 | 4.81 | 0.41 |
10 | Grades fairly and frequently | 5 | 4.81 | -0.19 |
08 | Uses classroom lab time fully | 4.8 | 4.77 | -0.03 |
14 | Makes the course interesting | 4.7 | 4.77 | 0.07 |
05 | Welcomes questions suggestions | 5 | 4.77 | -0.23 |
11 | Makes the purpose of the course clear | 5 | 4.74 | -0.26 |
15 | Textbook appropriate | 4.2 | 4.71 | 0.51 |
09 | Provides clear directions for assignments | 4.6 | 4.65 | 0.05 |
06 | Shows interest and respect | 4.7 | 4.53 | -0.17 |
07 | Helps students meet learning needs | 4.8 | 4.52 | -0.28 |
12 | Talks clearly at easy to follow pace | 4.5 | 4.52 | 0.02 |
04 | Avails for student conference | 4.7 | 4.47 | -0.23 |
03 | Gives help | 5 | 4.42 | -0.58 |
Averages: | 4.71 | 4.68 | -0.02 |
To summarize the statistics student evaluations for spring 2008, I keep a reqular schedule, have a course well paced with activities, show interest in the subject, and I grade fairly and frequently. I need to improve on giving help, providing individual attention, meeting the student's individual learning needs, and my availability for individual student conferences. Of concern to me is that my perceived helpfulness on an individual basis has fallen since 2001.
A statistical test of the 2001 versus the 2008 data shows no statistically significant difference in my overall average between the two years. Overall my gains in some areas are offset by losses in others.
The division-wide average in 2001 was 4.51. Both my 2001 and 2008 averages of 4.71 and 4.68 respectively are above the 2001 division-wide average. I have no other data against which to compare my performance.
Student comments often provide insight into the student's answers. The comments are provided below.
In SC 130 Physical Science the results were similar to MS 150 Statistics.
n | Question | Spring 2008 |
---|---|---|
02 | Shows interest in subject | 4.96 |
05 | Welcomes questions suggestions | 4.92 |
14 | Makes the course interesting | 4.89 |
09 | Provides clear directions for assignments | 4.89 |
13 | Lessons well paced with activity | 4.89 |
11 | Makes the purpose of the course clear | 4.89 |
01 | Regular schedule | 4.89 |
10 | Grades fairly and frequently | 4.86 |
06 | Shows interest and respect | 4.86 |
12 | Talks clearly at easy to follow pace | 4.79 |
07 | Helps students meet learning needs | 4.79 |
03 | Gives help | 4.75 |
04 | Avails for student conference | 4.68 |
08 | Uses classroom lab time fully | 4.54 |
15 | Textbook appropriate | 3.96 |
Average: | 4.77 |
Two items shifted position in the rank order, use of classroom and lab time fully and the appropriateness of the textbook. The later item fell to the lowest average for all items across both courses. Against results in 2001, a 3.96 is lower than all but one textbook in use in the division at that time. Note that one of the 3.9 values in the 2001 table was for the IS 201 Introduction to Computer Informations Systems course text book. I happened to be teaching that course that particular term.
Overall, as an instructor, students rated me higher in SC 130 Physical Science than in MS 150 Statistics. A 4.77 average would have placed me in first position in the rank order back in 2001.
Student comments often provide insight into the student's answers. The comments are provided below.
Although the sample size was smaller for physical science (n = 28), there were more written comments than were received for statistics class. There is a suggestion that the writing core to the course spilled over into the student's willingess to add written comments.
The Student Self-Evaluation form was given in both MS 150 Statistics and SC 130 Physical Science. This survey was run by the college in conjunction with the instructor survey back in the 1990s.1
Never | Rarely | Sometimes | Usually | Always | ||
---|---|---|---|---|---|---|
1. | I come to class on time | 1 | 2 | 3 | 4 | 5 |
2. | I am prepared each day | 1 | 2 | 3 | 4 | 5 |
3. | I turned in my homework on time | 1 | 2 | 3 | 4 | 5 |
4. | I spoke up when called upon | 1 | 2 | 3 | 4 | 5 |
5. | I asked questions in class | 1 | 2 | 3 | 4 | 5 |
6. | I met with my instructor during office hours | 1 | 2 | 3 | 4 | 5 |
7. | I paid attention in class | 1 | 2 | 3 | 4 | 5 |
8. | I asked for clarification when needed | 1 | 2 | 3 | 4 | 5 |
9. | I took the course seriously | 1 | 2 | 3 | 4 | 5 |
10. | I tried my best | 1 | 2 | 3 | 4 | 5 |
11. | My grade should be: | A | B | C | D | F |
The results for MS 150 Statistics and SC 130 Physical Science are reported below. The order is in descending order of the average for the two courses. This value is not displayed in the table below and was used only to derive a common order for both courses.
n | Question | MS 150 | SC 130 |
---|---|---|---|
10 | I tried my best | 4.43 | 4.50 |
9 | I took the course seriously | 4.21 | 4.35 |
7 | I paid attention in class | 4.10 | 4.35 |
1 | I come to class on time | 3.81 | 4.38 |
8 | I asked for clarification when needed | 3.32 | 4.35 |
4 | I spoke up when called upon | 3.63 | 4.00 |
2 | I am prepared each day | 3.74 | 3.77 |
3 | I turned in my homework on time | 3.42 | 3.96 |
5 | I asked questions in class | 2.47 | 3.38 |
6 | I met with my instructor during office hours | 2.13 | 3.15 |
Average: | 3.53 | 4.02 | |
11 | My grade should be: | 2.74 | 3.21 |
Students in both courses self-report that they tried their best, took the course seriously, paid attention in class, and came to class on time. Physical science students may have been more likely to ask for clarification and to have spoken up when called upon. Students self-report asking questions only rarely or sometimes.
Students also report not meeting with their instructor during office hours. This suggests that the lack of individual attention and help reported in the instructor survey by student is in part connected to students not coming to the office for assistance.
The statistics student's would collectively award themselves a 2.74 grade point average. The actual average for the course is 2.24. The student's meta-cognitive knowledge of their performance in the course is not as unaligned with reality as I had surmised it would be.
In physical science the student's self-estimated their GPA would be 3.21, the actual GPA for those who were surveyed was 2.75. This also a difference of half a grade point. Although individual grade estimates may or may not have been accurate, collectively the students only over-estimated their own grade by half a grade. This suggests that the course did a good job of helping the students know how they were performing. Bear in mind the question was "My grade should be" not "My grade is," the later would be a better measure of their meta-cognitive knowledge of their performance.
Student comments from both courses.
In conjunction with the above surveys, an affective domain survey was run in physical science and is available on line.
1The lack of the widespread deployment and use of research technologies like optical mark readers has forced the college curtail many of these surveys as hand-marking is impossible on the scale of the college. Research at the college using surveys has decreased over the past decade due in part to a lack of technological support for surveys out into the academic divisions. The college has made little effort to empower faculty as researchers in their own classrooms, while at the same time extolling the same faculty to engage in assessment activities. That a single scanner is kept in the administration building is of little use to faculty who typically work weekends and after hours on their course materials.