Review
Type I Outcomes: Institutional and Program outcomes
Type II Outcome: Institutional and Program level student learning outcomes
MS/ED 110: An example of a special subprogram
MS 150: When general objectives need to be tossed or moved to mid tier
Nota bene: The links in the email version of this document will not all function properly, such as the table of contents links above. These links will only function in the web page version that will be posted at a later date.
This work is part of an ongoing monologue on my progress towards measurable learning outcome implementation for my courses, for the division, and the institution. This presumes a prior knowledge of the material at slorevolution.html and slorevolution_ii.html
In slorevolution.html I described the grade book impact of moving to measurable student learning outcomes.
One of the two strongest impacts were increases in the skill specific record keeping. The second was tendency towards a competent - not competent overall grading scheme. In this document I developed the concept of core and peripheral outcomes. This development reopens the possibility of skill differentiations that would allow grading other than the all or nothing pass/fail structure of a competent - not competent overall grading scheme. This is now reflected in the PE 101j Joggling outline.
In slorevolution_ii.html I described the impact of the new outline format and noted complications arising from many-to-many relationships between program outcomes and specific measurable student learning outcomes. I also found program level outcomes that described things I wanted the students to value for which there were no specific measurable student learning outcomes. These later outcomes, call them one-to-none program level outcomes, are generated intrinsically by the course. These are measurable, but are currently not measured. Measuring might require post-course Likert scale surveys. Thus the outcome would not be a part of the student's evaluation. A couple of these are given in the new (proposed) PE 101j Joggling outline.
The next step in this Pirsigian journey is to work on program level outcomes and institutional outcomes.
Type I outcomes are those outcomes that are also known as indicators. Section sizes, faculty-to-student ratios, and success rates are all indicators. At both the institutional level and the program level, these outcomes drive many decisions from scheduling to budgeting, from curricular decisions to hiring specifications.
Many of these outcomes are pre-existing in institutional strategic plans, performance based budget reports, or individual divisional plans.
Institutional memory note: For those newer to the College, many of these began life in mid-1990's as "Program Health Indicators". These would later be called "Objective Health Indicators" during an effort to align pre-existing division level Program Health Indicators with division goals and division objectives. This was the genesis of the phi.html file. Essentially I am now converting, by rewriting and re-aligning, the division objectives into division outcomes and the goals into institutional outcomes. This might seem like a "reach", but division goals were so broadly written that they can probably metamorphose into specific institutional outcomes.
The following excerpt from phi.html should help illustrate this process.
| Institutional Outcome | Program Outcome | Objective Health Indicator | Present Status | Recommendation | Projected result |
|---|---|---|---|---|---|
| The institution will provide a quality mathematical education for our students by hiring and maintaining qualified faculty. | Faculty who teach college level math courses will have at least a Master's degree in field or a related field with 18 college credits in mathematics. Exception: A hire done in coordination with staff development. | All math instructors have MS degree in the field or a related field with 18 credits in field. | 100% compliance among full time faculty 2000 to 2002. | Maintain the status quo. A goal of, for example, 100% PhDs would not necessarily be appropriate especially in the lower level math courses. |
The original first column entry was, "The institution will provide a quality mathematical education," and was listed as a division goal. This is now attributed as an institutional outcome.
I will now tackle some questions that come to my mind at this point.
Should an institutional outcome include division specific language such as the word mathematical in an institutional outcome? Doesn't this preclude other divisions from meeting the objective? No, just as all courses in a division will not meet each and every division level outcome, not all divisions will meet each and every institutional outcome.
Why not drop the word mathematical from the institutional outcome and make the outcome more general? I would argue that in requiring all majors to complete a general education core, the institution has stated that there are some subject areas that transcend divisions and majors. These are subjects are in areas such as mathematics, science, English, and social science. The institution is, by demanding a general education core, placing value on specific areas. Hence institution wide outcomes for these areas should exist.
Why is this being placed at the institutional level? Isn't this really a division level issue only? No, and for two reasons. The first is tied to the above paragraph: the general education core crosses all division and major boundaries. The second is unique to this outcome: only the President can hire faculty. Thus any outcome that impinges on hiring impinges on the President. The President is by definition an institutional level office.
Won't this lead to there being too many institutional indicators? Just as courses will be specifying every skill required by that course, so to the College will be specifying what could run into the hundreds of outcomes. The number of outcomes is not necessarily a problem. Managing and evaluating those outcomes might be a problem, but not necessarily. Consider the above example. The measurement is done by the division not the institution. The division reports on whether it is meeting the institutional outcome. I suspect this will be true of many institutional outcomes. Like our once-upon-a-time program health indicators, reports on this sort of institutional outcome will be reported out of the divisions. The load of evaluating the institutional outcomes will not necessarily all fall on one person.
Note that Type I outcomes (or indicators if you like) do not usually percolate down into the course outlines. This a key feature of these outcomes. They are developed, in all likelihood, by the division chair in coordination with others in administration. They are then monitored by either the division chair or a higher level administrator.
There is one other odd aspect to Type I indicators. Type I indicators generate, from the bottom up, specific institutional outcomes. In writing the above, I have effectively committed the institution to hiring and maintaining qualified mathematics faculty. This will be one of the likely results of this process: to some extent the divisions will be telling the institution what outcomes the institution needs to achieve.
The impact of these cannot be underestimated. I head into budget defense hearings on Monday. Part of my defense might be based on institutional outcomes I am presently crafting such as, "The institution will maintain reasonable section sizes based on prevailing standards at institutions similar to the College of Micronesia-FSM." "The institution will maintain reasonable faculty to student ratios based on prevailing standards at institutions similar to the College of Micronesia-FSM." Couple those with an institutional outcome that "The institution will, to the best of its ability, meet student demand for courses," and you have a budget defense for adding faculty in the face of rising enrollments.
Type II outcomes are division, program, and institutional level measurable student learning outcomes. The institution might have an outcome along the lines of "Students will be mathematically literate by graduation." This outcome might not be directly measured near graduation, but would certainly be measured by underlying program level outcomes that were in turn supported by specific measurable student learning outcomes in the course outlines. Consider the following (only a small piece of the phi.html document:
| Institutional Outcome | Program Outcome | Objective Health Indicator | Present Status | Recommendation | Projected result |
|---|---|---|---|---|---|
| Students will be mathematically literate by graduation. | Students will be able to determine the answers to arithmetic expressions. | ||||
| Students will be mathematically literate by graduation. | Students will be able to solve algebraic expressions in one and two variables. |
Note that the above table is NOT intended to include the specific learning outcomes found in the individual course outlines. This would appear in outlines such as the pre-collegiate math outline in the following format (this is HIGHLY ABBREVIATED:
Assessment by quizzes, tests, midterm, and final examination. Core outcomes will be evaluated on the final examination and success on core outcomes will not be determined until that time.
Technical note: As noted in slorevolution_ii.html, in HTML I cannot invoke legal numbering, hence my outlines are in the format seen above. HTML and CSS do not currently have a legal numbering standard module.
Note too that the outline uses the core and peripheral labeling introduced in the slorevolution.html document. The above should NOT be taken as a statement by me that graphing method or addition method are peripheral skills. Each instructor will have to decide which, if any, skills are peripheral. My guess is that any course that has no peripheral skills, where all outcomes are core, will have to go to competent - not competent or pass/fail grading.
A third and final note. As I said in slorevolution_ii.html, there is no way to tell what complications will arise when one applies the "rubber to the road." II.B.2 and 3 involves an unanticipated and new feature. I had thought, as I think Sue had thought when she presented the format, that outlines would have only program level outcomes and course specific learning outcomes. This generates a two layer structure to the outline.
The more I climb the Pirsigian mountain path I am on, the more I see the need for an layer between the program level outcomes and the course specific outcomes. A three tier outline structure. The division is not going to maintain division level outcomes at the level of granularity implied by "Students will be able to solve a system of two linear equations in two unknowns." The division level program outcome is properly "Students will be able to solve algebraic expressions in one and two variables." Hence a tier in the middle is sometimes required.
Why only sometimes? Please consider my example to be somewhat artificially contrived, but II.B.1. is a directly measurable student learning outcome and not a middle tier. II.B.2 and II.B.3, on the contrary, will not be directly measured themselves. They will be measured by completion of the core outcomes under them. II.B.2 and II.B.s are middle tier outcomes.
MS/ED 110 has the following general objectives:
The process I am undertaking demands that I either replace these general objectives with other extant program level outcomes, or add these to the program level outcomes for College Level Mathematics. Note that the institutional level outcome here is the old division goal, "To provide future educators the mathematical skills needed to teach math at the elementary and secondary level."
I chose to simply add the existing general objectives as division level outcomes, but to consider them as unique to the course. In effect I elevated MS/ED 110 to being a single course program. MS/ED 110 is unique. The course only serves a single major, a single program: Education. The content and intent are unique. Hence I now consider it a separate subprogram of College Level Mathematics. There is a strong possibility that the program level outcomes for this course belong in the Education program and not the College Level Math program. The result of the above is the following section in phi.html:
| Institutional Outcome | Program Outcome | Objective Health Indicator | Present Status | Recommendation | Projected result |
|---|---|---|---|---|---|
| MS/ED Subprogram | |||||
| To provide future educators the mathematical skills needed to teach math at the elementary and secondary level. | Better mathematics skills among the teachers and, downstream, among their students | Have a course in place specifically for educating future teachers. | MS/ED 110 Math for Teachers Course. 38 Students in two sections. | Continue teaching the course. | Continued proper preparation of teachers for the teaching of mathematics. |
| To provide future educators the mathematical skills needed to teach math at the elementary and secondary level. | Students will learn mathematics constructively through the appropriate use of manipulatives, models, and diagrams. | ||||
| To provide future educators the mathematical skills needed to teach math at the elementary and secondary level. | Students will acquire confidence in using mathematics meaningfully and be able to apply mathematical thinking and modeling to solve problems that arise in other disciplines, such as art, music, psychology, science, and business. | ||||
| To provide future educators the mathematical skills needed to teach math at the elementary and secondary level. | Students will be able to design hands-on activities in order to teach a variety of mathematical concepts ranging from counting to algebra to elementary and middle school children. | ||||
The general objectives for MS 150 statistics are too course specific to be converted into program outcomes:
These cannot easily be translated into program level outcomes. The only extant program level outcome is "To provide students with mathematical knowledge needed to further their education at a subsequent college or university" Reworking this outline will be a challenge. The division level outcomes for MS 150 statistics should probably also be usable on the MS 152 Calculus outline. In other words instructors are going to need to work together when they develop program level Type I outcomes. Two other possible Type I program level outcomes that could be used by MS 150 and MS 152 are: "To provide mathematical support to other divisions" and "To provide mathematical electives for students"
Although this closes this third monologue, there will certainly be more to come.
I have all ready learned that it may be easy to create a single course, single instructor, new from scratch program such as PE that conforms to the new standards and formats. I have also learned that in an individual course taught only by one instructor on only one campus with pre-existing well-defined measurable student learning outcomes, the change is little more than record keeping, connecting to program/division/institutional outcomes, and a slight change in physical format for the outline. Moving larger programs involving multiple instructors, courses, and, in most cases, campuses, will be difficult at best.
Lastly, I may be climbing the wrong mountain path. That is, I am feeling my way upward in the direction that I think I last saw Sue and Brent disappear into the mists above me. Core and peripheral distinctions, mid tier outcomes, and many other things are strictly figments of my own feverish imagination. At best these are my interpretation of how to go about implementing measurable student learning outcomes at all levels at the College. At worst I am simply dead wrong and have misunderstood all sorts of things. I can only hope that if I am off-track then Sue and Brent or others will put me on track.
Dana
Go Sharks!