IB3. Self Evaluation

The standard implies an assessment mechanism exists. The assessment mechanism that was deployed by the college in 1997 was the Institutional Effectiveness committee. This committee was tasked with, among other things, looking at the processes began as Program Health Indicators two years earlier.

The Institutional Effectiveness Committee and its successor the Assessment Committee would have periods of regular meetings punctuated by long periods of dormancy. More fundamentally, the membership of the committee was not the same as the Curriculum Committee. While the Institutional Effectiveness committee included members of the Curriculum Committee, the committee did not duplicate the membership of the Curriculum Committee in full. The result was that analysis of assessment and data based decisions made in the Institutional Effectiveness and later the Assessment Committee were not being fed back into the Curriculum Committee. During dormant periods assessment efforts tended to wane.

Records from 1997 and earlier are sketchy at best, only some of the minutes remain extant. Then Director of Research and Planning Marie Abram was not on board during the critical 1995 workshop in which the division chairs first learned about program health indicators.

In 1998 Director Abram left the college. This provided an opportunity for the college to send non-administrative faculty to a critical conference on assessment and student learning outcomes led by James Ratcliff, Director National Center Post-Secondary Teaching and Assessment, on Guam during May of that year (Transcriptional Report from the Conference on Institutional Assessment, 1998, http://www.comfsm.fm/comfsm/ladera.html). With turn-over at the Director of Research and Planning, this conference would pass information on assessment to rank and file faculty. This concept, of decentralizing assessment and moving assessment closer to the students, to the division chairs and their faculty is a central theme in this document.

During 1999 Greg Myers would take over as the Director of Research and Planning. Director Myers would come on board without the experience of having taken the institution through a self study and self study report. The new director would face a heavy load of daily tasks as well coming up to speed on the recommendations made in 1998.

For reasons this author cannot ascertain, the Institutional Effectiveness committee appears to have gone dormant in the spring of 2000. There exists the possibility that intense work associated with the planning council and development of the mission statement and college goals caused the Institutional Effectiveness committee to fail to convene. Institutionally, there are so many tasks delegated solely to the Director of Research and Planning that the position becomes a critical single point failure point in the college system.

The committee would be reborn in the fall of 2000 as the Assessment Committee. Whether other meetings occurred in 2000 has yet to be documented by this author. As of the writing this report there is no indication there were other meeting during this year.

At the 20 February 2001 meeting of the Assessment committee the committee was told to be prepared to discuss the purpose of the committee, with one definition offered, "Gather data and analyze all COM-FSM programs and recommend actions for improvement." The minutes for this minute appear to be missing except for a single last page including an item six. The committee membership included six directors, four of whom have since left the college (Email address list for committee members, 20 Feb 2001).

In 2002 the committee members were all provided with a copy of the recently approved Strategic Plan 2001-2006. The minutes also noted the development of a distance education plan for the college. Attendance, however, was not good. The meeting on 24 January had only 10 of 21 members present, although one absence was an off-island member. Many committees include members from the remote state campuses. Their participation has always been constrained by the complications of distance. In the past attempts were made to use the PeaceSat radio satellite network to include state directors in critical meetings. There was also a brief attempt in the 1999 time frame to use instant messenger services to include remote state campus personnel in a meeting. A transcriber typed the conversation in the meeting in real-time, while in another window the remote state campus personnel could respond or chime in. The transcriber would have to stop to read the input from the remote state campus. Neither system worked sufficiently well that either experiment was continued.

Today the only inclusion of remote state personnel in meetings in via the email traffic prior to a meeting during which issues that will come up in the meeting are discussed. This is not only imperfect, but it excludes the state campus member from the actual decision making vote on an issue.

During 2002 the Director of Research and Planning would depart. Prior to his departure he would form four committees to work on the self study. These committees would meet on three occasions in mid-2002 (Personal conversations with a participant). In August 2002 an Acting Director of Research and Planning would call a college-wide meeting and call for the formation of four new committees to work on the self study. The four new committees were not in any way related to the former four committees. Turn-over in a critical position, possibly unavoidable turn-over, once again impacted an assessment effort of the college.

Attendance is also impacted by the sense of institutional commitment or lack thereof. In an institution where all faculty are on limited term contracts that can be non-renewed with 60 day notice and without cause being given for non-renewal, full-time faculty are in the same position as a part-time faculty at other institutions. Coupled with the remote location of the college, a pay scale half that of equivalent positions in the United States, and the difficulties for many expatriates in living on small island, obtaining a sense of institutional commitment is difficult. This has translated, in part, to poor attendance for some committees. Assessment committee has had its attendance problems.

During 2002 Susan Moses began presentations and one-on-one training sessions with the academic chairs on student learning outcomes. This is cited in the above table in part because it constitutes evidence that work on assessment was continuing amongst the faculty during a time when the assessment committee was dormant. This is further evidence that decentralized assessment can proceed even in the absence of a functioning assessment committee. In section IB6 an alternative structure for assessment that fits with this new decentralized reality will be proposed.

In spring 2003 the Assessment committee reformed. The Acting Director of Research and Planning, however, would depart in early August 2003.

Complicating the ability of these committees to function was the perception by 2001 that the committee work was unfocused and primarily a talking session for the then Director of Research and Planning (Personal observations of a member and author of this section). At the 07 July 2001 meeting of the Assessment Committee only six of seventeen members were present. The following meeting had only five of seventeen members. Even once the term started attendance rose to ten of nineteen appointed members.

Assessment committee meetings in 2001 and 2002 left one participant with the impression that they were addressing the need of the moment for the Director of Research and Planning (Personal observations of a participant). A sense that the committee work would lead to an annual cycle to regular assessment was not evident.

Communication of decisions made within the assessment committee to those affected by the decisions was not accomplished. The 10 March 2003 meeting of the Assessment Committee passed a resolution that the development of standards for program level outcomes would be the responsibility of the Assessment Committee. This pronouncement, however, lacked input from critical non-members who were, at the same time, developing program learning outcomes based on dialog in their divisions and based on the recommendations of national organizations in their field. This isolation and detachment of the assessment committee from planning and deploying committees such as curriculum is addressed in the planning section below and in the planning section for IB6.