1. Institutional Mission and Effectiveness
   1. Mission
      1. Alignment
      2. Approved
      3. Reviewed
      4. Centrality
   2. Improving Institutional Effectiveness
      1. Dialog
      2. Goals
      3. Progress
      4. Evidence
      5. Communication
      6. Planning
      7. Mechanisms

IA.  Mission

IA.  The institution has a statement of mission that defines the institution's broad educational purposes, its intended student population, and its commitment to achieving student learning.

IA. Descriptive Summary

Mission statement of the College of Micronesia-FSM

The mission statement is published on page five of the 2003-2005 College of Micronesia General Catalog and reads:

Historically diverse, uniquely Micronesian, and globally connected, the College of Micronesia-FSM is the national institution of higher education of the Federated States of Micronesia. Originally established to develop teacher education, its current mission is to provide educational opportunity – academic, vocational and technical – for all people. Aimed at nourishing individual growth and national unity, scholarship and service, COM-FSM is dedicated to developing integrity, critical thinking skills, a breadth of vision, and the habit of reflection in an educational environment enriched by cultural traditions. [IA(1)]

Perhaps the most important quality of the college mission statement is that it is a dynamic organism.  In essence, since it is the collective thoughts and beliefs and goals of many people, it is alive.  It lives and breathes throughout the college.  Like any other living organism, it changes and evolves over time.  Thus, through the process of its construction as described below, the mission statement continues to clearly define the mission of the college.

Educational purposes

As laid out in the mission statement of the college, its broad educational purposes are to provide educational opportunities – academic, vocational, and technical – for all people.  Specific broad aspects of these purposes include emphases on individual growth, national unity, scholarship, and service.

Appropriateness

Six of the basic elements of the purposes in the mission statement are common to post-secondary educational institutions throughout the world: academics, vocational training, technical skills enhancement, individual growth, scholarship, and service.  The remaining element is unique to an institution of higher education which serves as the only one of its kind for an entire country – that of national unity.  The addition of attention paid to cultural traditions sets the college apart as specifically appropriate for the students it serves.

Students

The mission statement indicates that educational opportunity is provided by the college to all people.  Thus, while it is true that the core nucleus of the current college population is comprised of traditionally-aged college students (immediately post-high school), applications are accepted from non-traditional and returning students for degree-bearing programs.  Additionally, continuing education and community programming is provided to adults from many walks of life, including development of our own staff and faculty.  Finally, children are served through several programs, including teachers trained by the the Division of Education and by programs run in the state campuses.  [IA(2)]

Determination of intended population

In many cases, funding sources define those whom the college can target as potential students.  For instance, a grant-related project may specify certain criteria that must be met by participants in a particular program.  When such limitations are not present, programs are more widely advertised and information encouraging participation is disseminated to the community at large.

Match between served population and institution

The College of Micronesia-FSM is a multi-campus institution with the National Campus located in Palikir, Pohnpei and branch campuses located in each state.  The college system also includes the FSM Fisheries and Maritime Institute, which is located in Yap.  The area most directly served by the college is the Federated States of Micronesia which includes about two million square miles of the western Pacific Ocean and a population of over 110,000.  Thus, while a few international students are enrolled each semester, the vast majority of students are FSM citizens.  Hence, the deliberate inclusion of FSM national unity and Micronesian cultural traditions in the college's mission statement reflects the specifically unique qualities that engender the match between this institution and the students it serves.

Processes to foster college-wide commitment to student learning

The student is the central figure in all college processes and actions.  Student support units exist to serve the students.  Courses are developed and delivered by faculty to meet the academic needs of the students.  Policies and procedures are designed to protect the interests of the students.

Expression of commitment in mission statement

Specific terminology within the mission statement which expresses the commitment to student learning includes:  educational opportunity, individual growth, scholarship, service, developing integrity, critical thinking skills, breadth of vision, habit of reflection, educational environment, and cultural traditions.  Note that the mission statement acknowledges the history of the college as a place to develop teacher education, but that its current focus is on the development of human resources.

IA. Self Evaluation

As noted at the commencement of this section, the college views the mission statement as a living document and subject to change as necessary to keep pace with the evolution of the college.

Mission Statement Survey

In January 1997 the director of research and planning ran a survey of the mission statement extant at that time. The 1997 mission statement differed from the present mission statement:

The College of Micronesia-FSM is the national college of the Federated States of Micronesia. Its immediate mission is to: provide an equal opportunity for quality higher education; assist with the job training needs of the Nation; promote economic development; foster a sense of national unity; preserve and enhance the artistic and cultural heritage of the community; and encourage the growth of the individual as a person, as a citizen of his or her community, nation, and the world, and as an enthusiastic and competent participant in his or her civilization and culture. [IA(3)]

The 1997 survey asked the following questions:

1. I am familiar with the COM-FSM mission statement Statement
2. The mission is clear, direct, and realistic Statement
3. COM-FSM implements its mission successfully Statement
4. The present COM-FSM mission addresses the FSM national mission for the year 2001 [IA(4)]

The original analysis was:

Data indicated agreement... to statements one, two, and four. Here respondents indicated that they were familiar with the COM-FSM mission statement; they believed it to be clear, direct, and realistic; and they felt the present mission statement is in tune with the FSM national mission for 2001.

Data were mixed on the third statement: COM-FSM implements its mission successfully. This indicates there was a wide range of opinions on this issue. [IA(4)]

The new study modified question four only slightly:

1. I am familiar with the mission statement
2. The mission is clear, direct, and realistic
3. COM-FSM implements its mission successfully
4. The present COM-FSM mission addresses the FSM national mission

As in the 1997 study, respondents had the option of choosing strongly agree, agree, neutral, disagree, strongly disagree. These responses were then later coded using the following numerical values:

 2 Strongly Agree
 1 Agree
 0 Neutral
-1 Disagree
-2 Strongly Disagree

Although returns were low (n = 18), confidence intervals using a t-distribution could be calculated (n > 5). All returns were from faculty, staff, administration, and a single student.

The following chart depicts the mean for each question at the middle of each vertical bar. The extent of the bar reflects the 95% confidence interval.

The graph suggests that for the college community question one and two continue to generate high levels of agreement. In contrast to the 1997 study, the third question has moved from generating a wide range of opinions to a narrower range of agreement with the statement. The confidence interval is larger than for question one and two, suggesting that there is still more range in opinions on question three than on questions one and two.

For question four there was the widest range of responses and the lowest level of agreement. The confidence interval nearly includes the possibility of a neutral response as a possible population value. Interpreting the responses is problematic. Either the respondents are reflecting a sense the college does not meet the national mission or the they are expressing uncertainty in the meaning of the phrase "national mission."

Comments by Respondents

Some of the comments received provide an insight into the thinking of the respondents. Two respondents indicated that the phrase a breadth of vision, and the habit of reflection was problematic. One respondent noted that the phrases "may seem to be so broad. I have difficulty nailing them down." Another respondent said, "I really don't know what this means and if it is realistic to know whether or not the college can achieve it."

Meeting the Needs of the Federated State of Micronesia

Two respondents did not answer question four, asking "What is the FSM national mission?" The question presupposes that an FSM national mission exists and that faculty would be familiar with that mission. Although there is no mission statement per se that the college can turn to, there have been statements made in national forums that can guide the college.

The law that created the college directed the college to provide post-secondary education including teacher training, continuing education, cultural education, adult basic education, vocational education, extension services, and post-high school college preparatory instruction. The college was to develop and offer certificate and degree programs that will meet the development and manpower needs of the states and nation. The college was directed to conduct and support research relevant to the needs of the states and national government with a particular focus on assessing ongoing training, educational, and technological needs. [IA(5)]

The August 1995 Human Resource Development in Micronesia report prepared for the Asian Development Bank on behalf of the government of the Federated States of Micronesia called on the college to be involved in the development of a modular multi-model teacher preparation program. [IA(6)] The document also called on the college to design and provision a program of induction of new teachers. [IA(7)]

The March 1996 FSM MegaConference: The Who, When, and How of Educational Improvement directed all schools in the FSM to develop a results based system. Schools were ordered to develop the human resources to support sustainable economic and social development. The college was also directed to develop cultural education programs, require a Micronesian history course, and reinstate the Island Arts program. State and national scholarship programs were told that the college should have greater involvement in their programs. The college was tasked with reducing attrition rates and improving graduation rates. The report recognized that improving the skills of the secondary school teachers would help.

The one area the report overlooked is that an estimated 15,000 FSM citizens are abroad. [IA(8)] Estimates made by the Micronesian Seminar are that ten of eleven new jobs "found" by young FSM citizens were found overseas in the 1990s. [IA(9)] Anecdotally FSM citizens can be found in minimum wage jobs from Tumon Bay, Guam, through Waikiki, Honolulu, to Virginia Beach, Virginia. These statistics suggest that the college could play a critical role in providing post-secondary education that might allow young FSM citizens to apply for jobs above the minimum wage bracket.

The August 1997 Strategic Plan for the Improvement of Education in the Federated States of Micronesia was based on the 1996 MegaConference and issued the following mission statement:

The Federated States of Micronesia educational system recognizes its shared participation with parents, extended family, and broader social structures in the intellectual, emotional, physical and social development of children. It will deliver a quality, sustainable basic education system which provides all students with basic skills, thinking skills, and personal qualities; provides for the manpower needs of the Nation; develops a literate population based on the revitalization of local languages and cultures while ensuring high competence in English and other international languages; and collaborates with all sectors of the government and community to fully utilize available human and financial resources in developing the educational foundation required for sustainable economic growth and social development. [IA(10)]

This report tasked the college with a number of responsibilities. Among other recommendations, the college was asked to develop local language programs and provide teacher training and curricular support.

Another national mission statement that impacts the college is phrased as a policy statement in the Federated States of Micronesia Language Policy. This policy developed as result of the FSM Language Policy workshop held in January 1997. The policy is:

The language policy of the Federated States of Micronesia is to enhance the economic growth and social development of the Nation through recognition of language as the carrier of the values and cultures which make us unique as a people and as the medium through which we communicate across the FSM and with the world. [IA(11)]

The Language Policy workshop called on the college to determine the need for an FSM languages requirement at the college.

More recently Dr. Kenneth Rehg at the University of Hawaii, in a draft of an article titled Thoughts on Training Teachers of the Indigenous Languages of the Federated States of Micronesia, noted that there are fifteen unique languages in the Federated States of Micronesia. Offering courses to students on their own languages is not a trivial undertaking. Dr. Rehg also noted that "Language shift is precipitated by a variety of factors, most of which are beyond the control of the school system..." Later in the article, Dr. Rehg "noted language shift is very difficult to impose from without. As a corollary to this observation, it is also clear that outsiders, however well meaning, will be powerless to stop it." [IA(12)]

Thus while the nation looks to the college to assist with conserving language, the reality is that many factors precipitating the language shift are beyond the control of the college. This does not absolve the college of doing what it can in its mission and programs to conserve local culture, this only tempers what the college can expect to accomplish. The national campus is home to the National Language and Culture Center (NLCI). The NLCI has assisted the education and languages and literature divisons in offering a linguistics course, a Pohnpeian orthography course, and a Pohnpeian language course to be offered spring 2004. The proposed bachelor's of elementary education will encorporate bilingual education into many of its courses. Language and culture are directly supported by programs such as the Micronesian studies program and by the integration of language and culture into other courses such as SC/SS 115 Ethnobotany. [IA(13)]

In September 1999, the president drafted a list of highlights and college mandates by sector from the second FSM economic summit. [IA(14)] These mandates included developing and expanding programs to support tourism, language and culture conservation, teacher education, nursing, expansion of sports and recreation, and fisheries. Business, hospitality and tourism, vocational education, marine science, agriculture, media studies, and the Fisheries Maritime Institute were all developed to meet national development and manpower needs. The Micronesian studies degree was developed to meet cultural education mandates. [IA(15)]

Commitment to Achieving Student Learning

The survey asked for comments on the requirement that the mission statement include a commitment to achieving student learning. The present mission statement does not include such a commitment. The question the survey asked was:

The new accreditation standards require institutional mission statements to now include a statement of commitment to achieving student learning in contrast, for example, to providing educational opportunity. Any comments on this new requirement?

The question was intended to be leading in order to focus responses on the specific area that arose in during dialog during an August 2003 student learning outcomes workshop.

The comments indicate that this is a dialog the college needs to have with itself. The comments are included primarily because there are those who disagree with meeting the required standard. Given the differences of opinion, the following are quotes from the surveys in order to preserve all the nuances present in the dialog. While some agree with the standard, others only agree because they feel it is an external requirement and not an internally desired result. The number of comments are not to be taken as representative of the number of people holding a particular position.

Pro: Fundamental agreement

I agree to the inclusion of the statement commitment to achieving student learning because this statement focuses on "a learner centered environment" rather than a "trainer centered environment."

As for "commitment to achieving student learning" to be included in our mission statement. I feel that there is no problem with that. In my opinion, our good instructors are doing that any way. We provide educational opportunity to all. We also work hard or are committed to make sure that students learn as much as they can. We are very much aware that we are not a university, but a community college dedicated to nurturing our students.

commitment to achieving student learning pretty much says it all and it should be part of our mission statement. For our program [Student Services Support Program], commitment to achieving student learning is already our mission.

Perhaps the statement about "student learning" could be placed right after "academic, vocational, and technical" as follows: "Originally establish to develop teacher education, its current mission is to provide educational opportunity – academic, vocational, and technical – with a commitment to achieving student learning – for all people."

In the new context I would just mess with the verbs – develop or produce (or some such) – provide and nourish are the main ones that need a verb change to reflect student achievement. Please no year to change this – and I guess it is the Planning Council wherever those guys are play with this - no?

Pro: External Requirement

There is a need... to include reference to student learning outcomes to be in compliance with the new standards.

If it's a requirement, let's add it in.

Con: Disagreement with the standard

This takes too much responsibility away from the student. We may be committed to the students learning, but if they aren't equally committed, prepared, and willing to work, learning won't happen.

If the new wording implies that we guarantee the students will learn, I do not agree with the change. We provide the opportunity, but responsibility of learning is with the student; we are committed to providing that opportunity.

Does this mean part of our job is forcing people to learn, whether they want to or not?

I disagree with the accrediting body on this one. I believe that learning is an internal process and that there's no way we can measure, let alone guarantee, that any student who comes to this institution WILL learn. The current wording on the mission statement is more in keeping with what we can reasonably do – provide opportunities. Then, it's up to the students whether they learn or not. It's the old saw about how we can lead a horse to water but can't make it drink. The accreditors are essentially saying that we have to guarantee, warrant, covenant, whatever that the horse will drink. That's ridiculous.

The college as a community will have to engage in a dialog on this aspect of the standards. Acceptance of a student learning phrase in the mission statement will have to emphasize that it is a commitment to student learning that is being made and not a guarantee of student learning.

IA. Planning Agenda

As a result of the self study, questions and concerns have arisen about the mission statement's wording. There are at least four dialogs that should occur. The first should center on the meaning and measurability of the phrases, "a breadth of vision, and the habit of reflection." The second should tackle whether the mission statement lacks a clear commitment to national manpower and development needs. The third is a dialog on whether a statement of commitment to student learning ought to be included, as opposed to an opportunity to learn. The fourth dialog, discussed later, is a dialog on whether the institution is "uniquely Micronesian" and does enough to support local language and culture.

The recommendation will be made that the Curriculum Committee be the starting point for a dialog on the mission statement. Revision options could then be drawn up, discussed, and the final revisions passed on to Cabinet for review. The Cabinet would then turn over the proposed revisions to the Planning Council in either an annual or biannual meeting. The Planning Council would make recommendations on the revisions which would then be forwarded back through the Cabinet to the Board of Regents for final approval. This would make much better and more efficient use of the Planning Council's limited time. The concept of a larger, more inclusive Planning Council that meets less often in order to provide strategic advice rather than tactical advice is covered in the planning agenda for IA3.

IA1.  The institution establishes student learning programs and services aligned with its purposes, its character, and its student population.

Descriptive Summary

Both the law that created the college and the mission statement call on the college to development academic, vocational, and technical education. The college has established student learning programs in academic, vocational, and technical areas. The college continues to development new academic, vocational, and technical programs. The college has developed degree programs in Hotel and Restaurant Management, Media Studies, and Micronesian Studies. Certificates have been established in a number of fields including electricity, carpentry, and cabinet making/furniture making. The college responds to national manpower and development needs with these and other programs. The certificates and degrees developed are in keeping with the character of the institution as a two-year post-secondary institution.

Those students who are not planning to remain in the Federated States of Micronesia after graduating from the college are often planning to attend four year institutions and complete a bachelor's degree. The college maintains articulation agreements with other institutions to ensure that courses transfer. For students traveling abroad to work, the education provided by the college provides the alumni with stronger skills in language, mathematics, sciences, and social sciences along with a college degree.

Requests from the community have come to the college for specific professional training programs (e.g., the request for a trial counselors program from the office of the country's Chief Justice of the Supreme Court, the request for a health assistants' training program from the local and state hospitals) as well as for a wide variety of non-credit continuing education programs.  Program participation is evidence that community members find value in state campus offerings. Rising system-wide enrollments indicate students find value in the programs the college offers.

Relevance of mission statement to student learning

As the college focuses on full implementation of student learning outcomes, questions have arisen concerning the use of the phrase "to provide educational opportunity" rather than a phrase which would more clearly state what the student will gain from the opportunities provided.  In other words, what WILL the student learn versus what MIGHT the student learn?

Certain voices within our college community are raising concerns that too little emphasis is being placed on the Micronesian cultural aspects of the college.  The mission statement currently indicates that, as an institution, the college is "uniquely Micronesian" and that it is dedicated to "an educational environment enriched by cultural traditions."  However, many within our ranks, as well as those in other constituent stakeholder groups, point out that language and culture are synonymous and cannot be separated.  There is significant concern that the trend toward English-only in terms of both the language of instruction and the language of production (both written and oral) by the students flies in the face of the intent of the mission statement.  Thus, there is a movement to include student learning of and/or in Micronesian languages (and, by extension, culture) as a revision of the mission statement.

Student learning in the mission statement

Statements specific to student learning include these phrases:

• to provide educational opportunity
• nourishing individual growth and national unity
• scholarship
• service 
• integrity
• critical thinking skills
• breadth of vision
• habit of reflection
• educational environment enriched by cultural traditions.

Purposes of the institution made explicit

Here are some explications and examples linking the words and phrases listed above to the purposes of the college:

• to provide educational opportunity:  As noted above, revision of this phrase is currently under dialog to reflect a movement toward the end product of the educational opportunity rather than merely the provision thereof;
• nourishing individual growth and national unity:  Student learning outcomes are geared toward both personal development and national interdependence;
• scholarship:  Scholarship implies rigor and standards within the provision of learning opportunities and the expectations for student learning outcomes;
• service:  Institutions of higher education are increasingly implementing service learning.  Some programs within the college are already including service learning as part of the formalized curriculum; others plan to incorporate service learning components soon.  College-wide, informal and extra-curricular opportunities abound for students to engage in volunteerism and other service learning contexts.  For instance, our Student Body Association gives participants opportunities to serve their school and peers through student government.  Another example is the regularly-scheduled campus clean-up days and "trash-a-thons," which foster unity toward the common good of reduced litter and improved aesthetics; the pride students feel in their campus is only one aspect of this service learning activity;. 
• integrity:  Implementation and enforcement of the college's Academic Honesty Policy compels students to behave in a manner reflecting integrity while they learn;
• critical thinking skills:  Learning how to think critically is considered by many educators the crux of learning at the college/university level;
• breadth of vision:  Most of our students come from small islands yielding extremely limited backgrounds which can be described as narrow in terms of experience and exposure.  Many students state that the most important thing they've learned at the college is the enormous breadth of the world in terms of ideas, cultures, art, music, religion, philosophy, etc.  In short, students at the college are learning to broaden their world views;
• habit of reflection:  The reflective learning process is valued and taught.  Students learn how to engage in self-evaluation and how to use the resulting information to replicate successes and avoid making the same mistakes in the future.  Many courses, for instance, involve audio- and/or video-taping student presentations and performances so students can learn not only the content-area skills but also this reflective process to apply to many areas of their lives; and
• educational environment enriched by cultural traditions:  As noted earlier, a concern specific to the lack of Micronesian language learning has been raised in this area.  As part of the ongoing dialogue on that issue, many faculty have been pointing out the myriad ways that cultural traditions have been integrated into the curriculum – as topics upon which students prepare essays in writing classes, as indigenous dances taught by the students to one another in physical education instruction classes, and as folk songs taught by students to one another in music classes.  Courses such as ethnobotany include learning local language vocabulary for plants and their uses. The exercise course in running and juggling also includes exposing the students to such basic terms such as run, walk, and juggle in their own language [IA1(1)]. The word "juggle" is sufficiently unusual that not all students know that word in their native language. The ongoing development of cultural related assignments, activities, and courses is evident that the college is indeed enriched by cultural traditions.

It is worthy of note that one of the greatest values of the self-study process for the college has been the discovery of the many ways the mission statement is being concretely applied at all levels – individual classes, programs, campus-wide matters, and systemic issues.

Addressing the needs of the student population

There are student representatives from both national and state campuses on the Planning Council, which is the body that reviews and revises the mission statement and other goals for the college.  Thus, students have direct input into ensuring that their needs are being addressed.  Additionally, faculty, staff, and administrators constantly seek student feedback both formally and informally to allow students' needs to be identified and addressed. 

Self Evaluation

The programs at the college are well aligned with its purposes, character, and student population as well as with the institutional mission statement. The mission statement reflects community needs and national mandates while providing for the academic growth of the students. The mission statement reflects the character and nature of the college. There is need to examine what it means to be "uniquely Micronesian" in an time of local language loss and the loss of traditions and culture. The extent to which an English language institution can conserve, or even promote, local language and culture is an area that remains to be explored.

Planning Agenda

The college will continue to develop and deploy degrees, certificates, and programs that are aligned with the mission and purpose of the college. The college has begun to offer local language and orthography classes, these offerings will be expanded to the extent that resources, personnel, and support material allow.

IA2. The mission statement is approved by the governing board and published.

Descriptive Summary

The mission statement was unanimously approved by the Board of Regents during a teleconference at 11:00 on July 16, 1999, at which point it formally became the institution's mission statement. [IA2(1)] The mission statement is published on page five of the 2003 - 2005 catalog. The mission statement is linked directly to the college home web page. [IA2(2)]

Self Evaluation

The mission statement is approved and published, both in the catalog and on the college web site.

Planning Agenda

Maintain publication of the mission statement in all appropriate media.

IA3.Using the institution's governance and decision-making processes, the institution reviews its mission statements on a regular basis and revises it as necessary.

Descriptive Summary

Development Process

The mission statement was initially developed through an intense collaborative process in which representatives of college and community stakeholders met repeatedly to come to consensus on precise wording that reflects all views and that most completely and accurately states the mission of the college.  Work on the mission statement was begun by the Planning Council on October 29, 1998 [IA3(1)].

From the college, persons on the Planning Council included representatives from each of these areas:  administration, faculty, staff, Staff Senate, students, and Board of Regents.  Additionally, those representing the college came from all four state campuses as well as the national campus.  From the community, persons representing each of these areas participated in the development of the mission statement:  business, churches, women, youth, persons with disabilities, traditional cultural leadership, and governmental bodies of the four states as well as the national government.

The specific composition of the Planning Council in 1998 and 1999 is important to understand the make-up of the participants in the construction of the mission statement. Any significant change to the mission statement would have to return to an equally broad and inclusive group of participants. [IA3(2)]

1. Participants external to the college:
   1. Traditional leader Iso Nahnken Nett Salvador Iriarte
   2. FSM National Government (Health, Education) Catalino Cantero
   3. FSM National Government (Fisheries) Bernard Thoulag
   4. Chuuk State Government Manny Mori
   5. Kosrae State Government Rinson Edmund
   6. Pohnpei State Department of Education Director Damian Sohl
   7. Yap State Representative John Mangefel
   8. Private business owner Loren Peterson
   9. Father Francis X. Hezel
   10. Emiliana Musrasrik (community leader in women's issues in the community)
   11. Lily Jean Iriarte (representing children with special needs and disabilities in the community)
   12. Caleb Ludwig (community leader in youth issues in the community).
2. College of Micronesia-FSM:
   1. Regent John Sohl
   2. Regent Brian Isaac
   3. President Susan Moses
   4. Staff Senate President Eddie Haleyalig
   5. Student Body Association President Xavier Yarofmal
   6. Vice President Spensin James
   7. Vice President Ringlen Ringlen
   8. Vice President Yasuo Yamada
   9. Director of Research and Planning Greg Myers
   10. Director Chuuk Campus Graceful Enlet
   11. Director Kosrae Campus Kalwin Kephas
   12. Director Pohnpei Campus Penny Weilbacher
   13. Director Yap Campus Lourdes Roboman
   14. Director of Academic Programs Joe Habuchmai
   15. Comptroller Robert Epstein
   16. Business Office Manager Pelma Palik
   17. Personnel Officer Linda Maradol
   18. Chair Agriculture Kyoshi Phillip
   19. Chair Education Richard Womack
   20. Chair Language and Literature Jonathan Gourlay
   21. Pohnpei Campus Instructor Phyllis Silbanuz
   22. Pohnpei Campus Counselor Maria Dison
   23. National campus student Jonathan Mathau
   24. National Campus student Xavier Yarofmal
   25. Pohnpei Campus student Penias B-More.

Between 29 October 1998 and 11 February 1999 the Planning Council met seven times. In meetings on 12 November 1998, 26 November 1998, 10 December 1998 the Planning Council worked on defining the identity of the college and who the college served. [IA3(3)]

A survey instrument was given to 60 people. The top eleven adjectives used to describe the college were, in descending order, excellent, friendly, prestigious, interesting, best, responsible, innovative, academic, and cultural. The top needs that the college should provide for the FSM was led by professional training, vocational programs, and well-trained, knowledgeable teacher/staff. [IA3(4)]

The survey asked how faculty and staff at the college can best help students, business, government, community, and other stakeholders. Top responses included getting involved in the community, serving public governmental needs, providing training for business and government, and preparing students to meet the student's own needs. The survey concluded by having the participants list the five values the college should represent. The list was led by quality education, honesty, customs and cultural values, integrity, loyalty, and being knowledgeable.

Among other missions that the survey indicated the college should undertake, there was a clear indication that one of the differentiating values the college has is the attention paid to traditional customs and cultural values. The college is uniquely Micronesian.

On January 14, 1999, the Planning Council produced four draft mission statements. On January 28, 1999, the group worked on producing a single mission from the four draft versions. At their February 11, 1999, meeting the mission in its present form was roughed out.

At the February 25, 1999, Planning Council the words "and national unity" were added to the mission statement. This wording reflected the under-current of thinking among national campus faculty and staff that another unique role for the college, specifically the national campus, was in bringing together the youth of all four states on the national campus. In working together, playing together, and living together in the dormitories, the students come to know each other as friends. The FSM is a nation of mutually unintelligible languages and underlying cultural differences. Both the national campus and sponsored programs such as Upward Bound and Talent Search (through summer inter-island student exchanges) promote and foster communication and understanding between the peoples of the FSM.

The proposed mission statement was sent through the Cabinet to the Board of Regents and was ratified unanimously. [IA2(1)]

The college communicates its mission statement in a variety of ways.  The mission statement is prominently featured in many of the printed publications of the college, including on page five of the current catalog [IA(1)].

The mission is also on a web page on the college's web site. The mission statement is read at the start of each board meeting. [IA3(5)] The mission statement is displayed in the administration buildings at each of the state campuses as well as at the national campus.  The mission statement is read aloud at orientation sessions for all incoming students. It is submitted with each funding proposal or request to granting agencies and government bodies.  More importantly, the college's mission is communicated beyond just the words of its mission statement.  The mission is communicated through the actions of the college, the programs it offers the community, and the education it provides its students.  Thus, the mission is communicated through the daily life of the institution, the participants in programs, and the graduates of the college.

Three factors lead the college to believe its mission, through the mission statement, is fully understood.  First, the wording of the statement was agreed upon by a wide variety of stakeholders and college officials. Thus each of the representatives in the creation of the mission statement, including those from within the college community, believed the ideas could be understood by members of the groups represented.  Second, the mission statement is purposefully phrased in plain language to optimize its comprehension.  Third, the mission statement is widely communicated in a variety of modalities (print, electronic media, oral readings, etc.) to enhance understanding regardless of a person's ability to access the information, disabilities, and/or learning style. 

The college views the mission statement as a living document and subject to change as necessary to keep pace with the evolution of the college.

Effectiveness of periodic review

There has been no documented periodic review of the mission statement. Many of the original participants in the process are still centrally involved in the college. The original process was so thorough, and in some cases exhausting, that there has been a reticence to re-enter the process too quickly. [IA3(6)] The mission continues to be viewed as relevant, as evidenced in part by its reading at the start of each Board of Regents meeting with the intent to "focus" the board on the mission of the college. [IA3(5)] Since the departure of the director of research and planning mid-2002, no Planning Council meetings have been held. The acting director of research and planning 2002-2003 did not convene the Planning Council.

Incorporation of the interests of stakeholders

The mission statement emanates from the Planning Council.  From the college, persons on the Planning Council include representatives from each of these areas:  administration, faculty, staff, Staff Senate, students, and Board of Regents.  Additionally, those representing the college come from all four state campuses as well as the national campus and the Fisheries & Maritime Institute.  From the community, persons representing each of these areas participate in the ongoing evolution of the mission statement:  business, churches, women, youth, persons with disabilities, traditional cultural leadership, and governmental bodies of the four states as well as the national government.  Should another legitimate constituent group emerge, a representative would certainly be invited to join the Planning Council.

Evidence of effectiveness

While no formal study has been undertaken in this regard, the ongoing support and involvement of the various constituent groups in the life of the college lend credence to the assertion that they are satisfied with the mission of the institution. 

Circumstances prompting changes

Noting that the Planning Council is a standing committee and will be reconvened, any member of the college community or a constituent group can communicate his/her concerns to the appropriate representative on the Planning Council.  The representative, in turn, will bring the recommendation to the next Planning Council meeting.  Something as dramatic as a reorganization of the entire institution would certainly prompt a major revision of the mission statement.  On a more commonly occurring basis, issues being discussed collegially through the faculty governance system may lead to recommendations for mission statement changes.  Most recently, the extensive and intensive accreditation renewal self-study process has given rise to some specific issues noted herein. The mission remains a touchstone for many committees.

Development of Goals from Mission

On March 11, 1999, the Planning Council turned its attention to the development of goals from the mission statement (March 11, 1999, Planning Council minutes). By mid-March the Planning Council was discussing the "centralization" issue: the role of the state campuses and their relationship to the national campus. These issues were raised initially by a faculty member at the Kosrae campus.

The March 25, 1999, Planning Council discussed the role of state campuses and the impact of resource limitations. These discussions continued in the next two meetings. The Planning Council is a broad and inclusive body including key business and governmental leaders. The biweekly meeting schedule was demanding for people with numerous other responsibilities. At some point the discussion moved from strategic planning to tactical discussions of what programs should be run at which campus. The biweekly schedule was orally reported to be onerous. Undocumented sidewalk discussions at that time suggested that many of these decisions could and should be made by the other standing committees of the college, the results of which could then be reported to the Planning Council. This would allow the Planning Council to function in a manner in which less time is required of the participants.

IB3. Self Evaluation

The mission statement has not been re-evaluated on a regular basis. Two factors have led to the perception that changing the mission statement would be a significant undertaking. The broad, multi-island group that was assembled to develop the mission statement in 1998 to 1999 would require significant investments of time, resources, and money to duplicate. Any lesser group would be seen as uninclusive and would not have the mandate of the original group.

The other factor was that the process in 1998 became protracted as constituencies argued over specific words or phrases in the mission. Individual meetings ran to over two hours, the process was painful in terms of time and energy for the participants. While all mission statements are necessarily dynamic, some missions may enjoy a longer shelf life than others due to the process through which they were formed. The formation of the present college mission statement was extraordinarily inclusive and carefully considered.

The February 4, 1999, minutes of the Institutional Effectiveness committee noted the lack of returns on a survey done to assess community perceptions of the college mission statement. [IA3(7)] Note that the Institutional Effectiveness committee was succeeded by the Assessment Committee on November 14, 2000. [IA3(8)(9)]

The Institutional Effectiveness committee of February 18, 1999, reviewed the mission statement. A recommendation was made to include "COM-FSM is dedicated to fostering national unity." [IA3(10)] The March 4, 1999, meeting of the Institutional Effectiveness committee recommended the addition of "to provide educational opportunity – both academic, vocational, and technical." [IA3(11)]

On the 18th of March 1999 the Institutional Effectiveness committee noted that the Planning Council had decided to retain the wording without including further wording on economic development. Questions arose as to who was on the planning council and to whom the council was beholden. Clearly there was a desire on the part of the committee to alter wording in the mission statement, but the lack of further discussion suggests a decision on the part of the chair to support the Planning Council's decision. This may have been the first discussion of whether the mission statement should be internally generated within the college or externally specified by the broader community.

This discussion will likely occur again over the matter of wording concerning achievement of student learning. Does the college have to add the wording as the result of a mandate or does the college have an option to differ? There might be a need for a deeper and broader dialog as to who actually decides the mission of the college. Should it align to the national mission? What if the national mission is not congruent with the desires of the students? Do student needs (desires to obtain skills not useful in the FSM but in work abroad for example) trump national needs? Or is it more complex: perhaps the college can serve both national needs and student needs, even when they diverge.

The minutes of the Institutional Effectiveness committee do not indicate any further review of the mission statement. The Assessment Committee appears to have met irregularly in spring 2002 and possibly not all during fall 2002. During this time the college did not have a full-time director of research and planning, although an acting director was appointed August 12, 2002.

In a May 22, 2002, meeting of the Planning Council a suggestion was made to change the word mission in the second sentence to purpose. [IA3(12)] This was one of final meetings of the Planning Council, there is no record of subsequent meeting. That summer the director of research and planning departed the college. An acting director was appointed in August and served during the 2002 - 2003 school year. The acting director did not reactivate the Planning Council nor undertake any further review of the mission statement.

During oral discussions of student learning outcomes at a workshop on August 5, 2003 the use of the word opportunity in the mission statement was questioned. The question asked whether the phrase "...its current mission is to provide educational opportunity..." might be replaced with a phrase such as "...its current mission is to facilitate and demonstrate achievement of student learning outcomes. [IA3(13)]

In addition, no comprehensive evaluation has been undertaken concerning the college community's or constituent stakeholders' understanding of the mission of the college.

Planning Agenda

The mission statement was forged via an inclusive and arduous process. Undertaking a major change in the mission statement would divert limited resources from other critical areas that must be addressed. There is a need, however, to begin a discussion of whether the college only provides educational opportunities or actually guarantees student learning.

Concordant with the dialog on the mission statement, under the leadership of a director of research and planning, the Institutional Research and Planning Office should perform a more formal and thorough survey review of the mission statement.

There is an additional need to reactivate the Planning Council, but to also make membership less onerous. Planning Council should be a very broad and inclusive annual event with participants from all sectors. Planning Council should be tasked with strategic planning as opposed to tactical planning and mission implementation. The annual meeting could be published in the catalog. Greatly reducing the number of meetings could also allow the college to bring in leaders from the other islands. The cost of bringing together people in this nation cannot be underestimated. Flying state and community leadership from Yap costs on the order of $1000 round trip per person. This is no small obstacle to convening a truly inclusive Planning Council. The college, however, spans six sites in four states on four different islands. The Planning Council can no longer claim nationwide participation utilizing only traditional and community leaders from Pohnpei.

IA4.The institution's mission is central to institutional planning and decision-making.

Descriptive Summary

As part of the college's strategic plan, the mission statement is an intrinsic part of all planning and decision-making at the college.  Additionally, the mission statement is posted in the meeting rooms where planning meetings are generally held.

Effectiveness in prompting planning and decision making

Because of the direct connection between the mission statement and the strategic plan, the former is highly effective in influencing the latter.  Thus, in essence, all decisions of importance affecting the college filter through the mission statement.  Conversely, the mission statement itself can be the catalyst to prompt planning action and decision-making on the college's behalf. 

Centrality

By making the mission statement an intrinsic part of the strategic plan, by using the mission as a guideline for budgeting, and by placing the mission statement and the living proofs of the mission of the college (programs, students, graduates, etc.) in the forefront of the way the college represents itself on every occasion, the college has ensured that its mission remains central to decision-making and to planning.

Self Evaluation

Having the mission statement as a central part of the strategic plan and other organizational activities of the college guarantees the influence of the mission statement on planning and decision-making.  To "test" the effectiveness by withdrawing the mission statement from these activities would be foolhardy and poor research design at best, and disastrous at worst, for it would cause the institution to lose its lodestone.  Thus, the self-evaluation that is in place – that of nurturing the living organism of the mission statement – is best.

Planning Agenda

The College of Micronesia-FSM has a well-constructed mission statement which was created by the inclusion of all constituencies, is central to the college, and is part of a constantly dynamic symbiotic process in which the mission statement influences policy, decision-making and planning while at the same time being responsive to changes in the institution.  Thus, the planning agenda is to continue cultivating this process for the indefinite future.

IB. Improving Institutional Effectiveness

IB. Descriptive Summary

Production and Support of Student Learning Outcomes

The core of the effort to produce and support student learning consists of utilizing a highly qualified and professional faculty who are given the necessary resources and support to deliver courses built around measurable student learning outcomes. Measurement of that learning is primarily done by faculty.

Courses are designed around outlines consisting of measurable student learning outcomes. These outlines are prepared by faculty who teach the courses. These course outlines are reviewed by division chairs and the curriculum committee. The curriculum committee ensures that outlines are written using measurable student learning outcomes. Production and review of outlines often involves referring to similar courses at comparable institutions.

The responsibility for the process of assessment, analysis, and redevelopment to improve student learning rests first and foremost on faculty. Each is a specialist in the area they are teaching, either via academic qualifications or through a combination of academic qualifications and experience.

The faculty who teach a course develop the student learning outcomes for the course, usually referencing both college-level texts and, in many cases, equivalent courses at other colleges.

Over the past year a model of assessment has been developed. Originally developed in one division, the model is merely a diagram that shows visually how a division achieves and measures learning.

The original core of the model consists of qualified faculty delivering courses based on student learning outcomes. The actual model exists as a web portal, a web page that contains links to supporting documents.

IB.1 Diagram of Web Portal with Faculty and Outline links

Courses based on Student Learning Outcomes		Qualified faculty
Outlines	Sample Syllabi	Instructor
MS 150	MS 150	Dana Lee Ling

Although the above diagram is static, the online version includes links to outlines and faculty qualifications. The "Outlines" item links to a list of outlines that are available online. The left "MS 150" links to the MS 150 outline. The name "Dana Lee Ling" links to the résumé for that member of the faculty. The actual portal includes other faculty and course outlines.

The link between faculty and the courses that they teach are their course syllabi. In the actual portal there are links to online syllabi. This permits anyone either inside or outside of our organization to access information on our faculty, our courses, and how those courses are implemented. This purports to produce results, student achievement data in the form of grades, that can be trusted.

IB.2 Diagram of Web Portal with Faculty, Outline, and Grade links

Courses based on Student Learning Outcomes		Qualified faculty		Grades
Outlines	Sample Syllabi	Instructor
MS 150	MS 150	Dana Lee Ling

On the actual portal page "Grades" links to a report on grades given in the division.

This system has two shortcomings. The first is that this system does not provide a way to assess program level student learning outcomes. The second is that this system relies wholly on measures that are internal to the classroom and within the scope and control of single individual. The college hires qualified faculty and trusts the faculty implicitly. An outsider, however, might want some form of external evaluation of the system. An outsider would probably ask what a given grade might mean.

Each division is in the process of making its own decisions on how to validate their grades to an outside observer. In the example shown above, the mathematics division has developed and deployed an in-house test that is to be given to a sample of post-MS 100 students.

IB.3 Diagram of Web Portal with Faculty, Outline, Grade, and Program Evaluation links

Courses based on Student Learning Outcomes		Qualified faculty		Grades		Program outcomes evaluation
Outlines	Sample Syllabi	Instructor
MS 150	MS 150	Dana Lee Ling

On the actual portal page, the item " Program outcomes evaluation" links to a report on pilot run of the mathematics program evaluation instrument in the spring of 2003. The program outcomes evaluation is referred to as an external measure for the system. The development and subsequent assessments of program level student learning outcomes is only just beginning at the college. The intent is that the external validation of grades will support program level student learning outcomes.

The following is the web portal with the program learning outcomes link depicted.

IB.4 Diagram of Mathematics Web Portal with internal and external support for Program Learning Outcomes

					Math Program Learning Outcomes Link to Performance Based Budget
Up to degree level outcomes, goals, and mission
		Internal Measures				External Measures
Courses based on Student Learning Outcomes		Qualified faculty		Grades		Program outcomes evaluation
Outlines	Sample Syllabi	Instructor
MS 150	MS 150	Dana Lee Ling

The above is merely a diagram, a way of depicting the inter-relationships among the systems that measure student learning at the college. The genesis of the diagram was in discussions in the fall of 2003 on the inter-relationships between course level student learning outcomes, program level student learning outcomes, and grades. Of course the heart of any college is the students, the diagram can only depict the structures that are involved with measuring student learning. The online portal provides easy access to the documents that report on student learning. The diagram has been referred to in some documents as a student learning outcomes nexus or student learning outcomes hub.

While other divisions may not physically deploy a web portal, all divisions are developing systems that could be depicted in such a manner. Each division is making their own decisions on what constitutes external validation of student learning outcomes. Some divisions are publishing rubrics and sample student work with the associated grade. [IB(1)] Other divisions are planning to use binding exit examinations. Discussion of external measures includes dialog on portfolios, projects, capstone experiences, and other ways of knowing students have achieved program level student learning outcomes.

The web portal does not depict the full assessment cycle, there are no circular arrows from program outcomes evaluation to course outlines. The diagram would become unnecessarily complex and unwieldy with the addition of arrows depicting the closure of the assessment cycle.

Support for student learning also includes a number of divisions and programs that are outside of the classroom.

The Learning Resource Center is an important support facility providing both texts and Internet access. This facility is covered in Standard IIC.

Other programs, some institutionalized and others supported by grants, provide support to students. These programs include tutoring programs and college-run grants that serve pre-college students.

IB. Key Processes and Resource Allocation

Resource allocation is separately and independently covered in Standard III later in this document.

Resource allocation is decentralized, including facilities design, facilities usage, and financial resources.

The facilities were designed with input from a number of constituencies, both at the national campus and at the state campuses. The exception to this is the Chuuk State Campus which does not currently own its own buildings.

The classrooms at the national campus, built in the mid-1990's, were designed with input from the chairs and faculty of the respective divisions of the college at that time. The Learning Resource Center was designed under the aegis of the library committee and guided by a vision statement prepared by that committee. Design details down to and including the location and type of shelves and computers were specified by a faculty-staff team.

Facilities usage is directed by facility directors and division chairs. The division chairs prepare schedules of classroom usage. Future facility plans are made in consultation with faculty and staff. Committees such as curriculum, the weekly chairs meetings, and ongoing email communications are all involved in providing input to these decisions.

Individual facility directors and division chairs prepare budgets for their area. These budgets are submitted to the finance committee, a committee consisting of faculty, staff, and administrators, for inclusion in the college budget.

The extent to which an individual facility director, unit chair, or division chair shares their budget planning and preparation with others in the facility, unit or division varies from unit to unit. Concern has also been expressed as to the transparency of budgets in the state campuses, especially with respect to the faculty and staff who work in the state campuses.

IB. Demonstrations of Effectiveness

Almost all actively taught courses are based on course outlines that utilize measurable student learning outcomes. Those few outlines that have been found to not use student learning outcomes are being rewritten. Measurement of these outcomes rests with faculty. At present there are no external mechanisms that provide evidence of achievement of student learning outcomes. All evidence of achievement resides with individual professors and instructors.

The methods of measurement vary from course to course, with each instructor specifying the measurement systems most appropriate to their course. Measurement methods include, but are not limited to, homework, quizzes, tests, midterm and final examinations, projects, presentations, papers, and performances.

The college has used student achievement data and other indicators to perform what have been called program health reviews. Key indicators have included enrollment and student seat costs. None of these indicators, however, carries any information as to whether learning is occurring in the classroom. Demonstrating that programs reviewed only by indicators are effective programs from the perspective of student learning is not possible.

IB. Self Evaluation

The college has begun to build program level assessment of student learning outcomes. This effort has begun with a dynamic and active dialog on what constitutes important program level outcomes in each program. This is an ongoing conversation that is built on the knowledge of what works in our classrooms, the learning styles of our students, and on what is seen as important for our students to take away from a program. Program learning outcomes have been adopted in mathematics and exercise sport science. Program learning outcomes for the English general education core and science have been proposed.

The effort to deploy program evaluations that do reflect learning, the external evaluations noted in the web portal diagrams above, has only just begun. The college has not had a year-to-year cycle of any external evaluation system by which to make true program effectiveness measures.

Assessment of learning at the program level has only begun at the national campus and in the tourism and hospitality program at the Pohnpei State Campus. Efforts at the other state campuses have yet to begin. Workshops have been planned to help these campuses perform program level assessment. Workshops were conducted by the Vice President for Instructional and Academic Affairs in Chuuk and Yap during late October 2003.

IB. Planning Agenda

The college is conducting a review of course outlines to determine whether there are courses using out of date outlines or outlines that do not utilize measurable student learning outcomes.

The college is continuing to develop program level student learning outcomes and assessment instruments. The VPIA has proposed a time-line for this work. However, as of the writing of this document the time-line has yet to be discussed by the curriculum committee.

Even while the curriculum committee discusses time-line issues, divisions are forging ahead with the development of program level student learning outcomes. This development work is lengthy as it must involve dialog inside each division leading to general agreement on the learning outcomes for that particular program.

IB1. The institution maintains an ongoing, collegial, self-reflective dialogue about the continuous improvement of student learning and institutional processes.

IB1. Descriptive Summary

The college had recommended the use of student learning outcomes in course outlines for many years. Under the leadership of the then new Vice President for Instructional and Academic Affairs, all outlines produced after fall 1992 had to be in a student learning outcome format.

In the fall of 1995 the college embarked on the development of program health indicators as a way to measure the performance of all programs, not just academic programs. Many of these program health indicators were just that: single point in time indicators or measurements such as faculty to student ratios, freshmen intake to graduation rates, numbers of students counseled, number of books checked out, or number of air conditioners cleaned. While many indicators were internal to the college, a few were external. An accounting program might look at the number of graduates who successfully obtained jobs in their field.

In 1996 and 1997, as part of the development of a five year strategic plan, the college developed program goals, objectives, and outcomes. These outcomes were not necessarily student learning outcomes, and in many divisions the previously developed program health indicators drove the development of the outcomes. That is, outcomes were developed that would be measured by the program health indicators.

Two examples of this structure can be seen below.

The structure seen above of goal-objective-indicator was developed for each division or area of the college by representatives from that division or area. These were developed not just for academic divisions but also for the Financial Aid Office, Office of Admissions and Records, the Business Office, Maintenance and Facilities, the Learning Resource Center, and all areas of the college.

This structure or portions of it can be seen in a variety of documents including the strategic plan and some of the indicators reported as part of the performance based budget. [IB(2)] Indicators or indicators modified into outcomes can still be found in a number of documents.

There is not, however, complete congruence between the Strategic Plan of 2001-2006 and the above reported table. This is in part due to the evolution and construction of the documents. The above table began life as a result of a workshop on program health indicators conducted in 1995. At that workshop divisions developed program health indicators by which to evaluate the "health" of their programs. These were student achievement data indicators. This effort was a "bottom-up" effort and included the generation of program goals and objectives.

The Strategic Plan 2001-2006 arose out of mission statement review and the setting of ten institutional goals by the Planning Council from 1998 to 1999. This process did not include all of the division chairs, those who had worked on the program health indicators effort three years earlier. Thus while the chairs were utilizing program health indicators, the Planning Council worked on a superstructure that was not directed by the existing structures. This was a "top-down" effort external to the divisions and generated goals on a different basis.

Work on bringing the program health indicators and their associated goals and objectives in line with the work done by the Planning Council was done by the Institutional Research and Planning Office in internal documents and by work done in the Assessment Committee in 2001. [IB(3)]

The goals and objectives (now renamed activities) would be further modified in 2002 and 2003 principally by the acting director of research and planning 2002-2003 and published May 2003 in the Performance-Based Budget Institution-Level and Program Outcomes and Outcomes Measures Fiscal Year 2004 (Performance Based Budget 2004 or PBB 2004).

This document would differ from the Strategic Plan 2001-2006 due to work done in the Assessment Committee and due to work done by the acting director of research and planning. A program-by-program analysis of differences has not been formally performed, but change is inevitable in an organism as dynamic as the college. Notable differences include the omission of the Micronesian Studies Program from the PBB 2004 document and the inclusion of Physical Education in the PBB 2004 document. The PBB 2004 document represents the most current published statement of the status of the college's program goals.

To understand the nature of the changes from the above program health indicator based model seen in Table IB1.1, the following is an excerpt from the physical education section of the Performance Based Budget 2004 document. The budget data is omitted from this table.

Table IB1.2 reveals a hybrid nature that includes student learning outcomes as measured by performance indicators. The next step in the evolution of assessment at the college is seen as measuring the student learning outcomes directly rather than using indicators. The Performance Based Budget, however, had to use measures that are already extant and not those the college is in the process of developing and deploying.

Evolution from a goal-objective-indicator structure to a student learning outcomes structure at the program and institutional level is now occurring. This evolution occurs through self-reflective dialog, although not necessarily the quiet reverie that the word self-reflection conjures up. This is an energetic discussion that permeates the campus. Specific models, formats, and implementations remain under discussion. Although some program learning outcomes have been approved, any examples presented should be considered unofficial and unapproved. These examples are meant to reflect the nature of dialog occurring.

Table IB1.3 derives from a physical education course outline and includes only a subset of all the student learning outcomes on the actual outline.

In the above model the institutional and program outcomes are measured through accomplishment of the course outcomes. Separately there would need to be survey assessments done to provide external measurements of the institutional and program outcomes.

The above outline is the result of a dialogue during the fall of 2002 that is documented in part by the following reports and documents:

Student Learning Outcomes Nexus (Web Portal)
SLO Part I: Grade book and grading impact, core and peripheral outcomes
SLO Part II: One-to-many and format matters
SLO Part III: Mid-Tier, Type I indicators, Type II outcomes
Part IV: Program Outcomes for Mathematics
Part V: Layout issues in MS 150 Statistics
Part VI: Course Intentions, outline contents, values outcomes
Part VII: MS 150 in General Objective and Specific Objectives format
Triple Tier is an enhanced variant of current format: Side by Side Comparison
System Wide Competencies
Division Objectives & Indicators; Program Learning Outcomes
Math Prog Evaluation Instrument
Math Program Outcomes Evaluation 21 March 2003

Ongoing Continuous Dialog

These reports trace an evolution in thinking as a result of conversations in curriculum committee and the ongoing conversations held on the sidewalks and in the cubicles of the college. The college does not set aside a specific time or place to discuss specific topics or assessment cycles. That was in part the downfall of the Assessment Committee (detailed in IB3 and IB6) - it was not seen as being relevant to what was happening out in the hallways and classrooms. The college is engaged in an ongoing, continuous discussion of learning in our classrooms. [IB1(1)]

Our small island environment means that faculty and administrators meet and talk nearly daily in a series of ongoing impromptu meetings and dialogs. There are only a limited number of stores on Pohnpei - impromptu meetings occur in the hallways of the college, in the aisles of a supermarket, between tables at a local restaurant, or on a Saturday at a picnic. This is probably unique to small island schools and to our environment. The island is a far more closed and insular place to live than even a small rural town. The result is that faculty are more aware of what is happening in other classrooms and other divisions than one would likely be at a mainland school.

IB1. Self Evaluation

The college is engaged in a discussion of measuring student learning. This discussion is centered at the national campus, but does include state campus personnel as well. The discussion includes the issues surrounding the difficulties of including remote island state campus personnel. Many of these personnel are part-time employees literally passing through our system, teaching for a term or two. In other instances there are permanent faculty, but communication with them can be difficult.

Due to the distances, and the cost of something as simple as a telephone call, communication with remote island faculty relies on email. Part-time faculty, however, rarely have email addresses, and when they do have an email address, the national campus and its division level personnel are rarely aware of the email address. Some campuses, notably Chuuk, appear to have difficulty providing faculty with sufficient access to Internet connected computers, complicating the communication effort immensely.

Thus the ongoing, self-reflective dialog is missing voices. Faculty from the remote state campuses are less likely to be a part of the discussion, and support staff at remote state campuses are likely to have no ability to join the dialog via email.

Thus while there is an ongoing dialog on student learning at the core, the periphery is less participative. The provision of computers and Internet access would not necessarily solve this communication problem: part-time personnel, as are often found teaching at the state campuses, are temporary employees. They are not likely to have any particular commitment to the institution. Involving them in the conversation may be difficult at best. Since such an employee might be literally gone by next term, there is little leverage to force them to contribute and, quite frankly, little they are likely to be able to contribute due to the briefness of their time with the college.

There are also communication issues even at the national campus. The college is quite unique in its extreme remoteness and in the size of the community in which the college exists. Coupled with no tenure track available, a large expatriate faculty, and no possibility under FSM law to remain in the FSM except under a work permit, the college will always have a high turn-over. There are faculty who are here for a three-year contract, and who have no intention of remaining beyond their contract. Other faculty might work a contract or two, but have no long term commitment to the institution or to living out their lives in the FSM.

The result is that the college has both institutional memory problems and institutional commitment problems. Faculty who are here for a brief time may not share the same level of commitment to working on institutional goals. Some faculty may be doing a superb job in their classrooms, but have little interest beyond the four walls of their classroom. Due to the combination of contracts that can be non-renewed without cause being given, and the restrictions of the work regulations in the FSM, the college cannot guarantee any faculty more than a four year contract. This is true for all positions at the college, administrative, faculty, and staff. The result is potentially both a lack of commitment to the institution by some and an inability of the institution to utilize tenure track faculty as key leaders in the institution.

Despite the above, there are faculty who have spent their working life at the college. The longest serving faculty members predate any current member of administration. While faculty turn-over will continue to impact institutional memory, the faculty remain the key to institutional memory over the long haul.

IB1. Planning

Time-lines for the implementation of program learning outcomes will be developed and adopted by the curriculum committee during the fall of 2003. The mathematics program and the Micronesian studies program will be conducting a second annual review of their programs during the spring of 2004. Depending on time-line negotiations, other programs will also have program learning outcomes evaluations during 2004.

Communication with the remote state campuses has gone from zero prior to the existence of Internet connectivity to extant. At this point, further improvement depends in part on facilities improvement at the state campuses. Kosrae State Campus built a faculty office which permitted the campus to provide each faculty member with a desk and a computer continuously connected to the Internet. The result is that personnel at the national campus can be in regular contact with personnel at the Kosrae campus. Pohnpei State Campus also provides desks and Internet connected computers to each faculty member, providing communication resources for their faculty. At this point, better communication with Chuuk and Yap would require similar improvements in their facilities.

IB2. The institution sets goals to improve its effectiveness consistent with its stated purposes. The institution articulates its goals and states the objectives derived from them in measurable terms so that the degree to which they are achieved can be determined and widely discussed. The institutional members understand these goals and work collaboratively toward their achievement.

IB2. Descriptive Summary

The goals of the college are stated on page five of the 2003-2005 college catalog:

• Instill and maintain pride in the unique heritage of linguistic and cultural diversity in Micronesia.
• Promote national economic self-sufficiency/self-reliance
• Develop individual's capacity for reflection, vision, critical thinking, and analytic problem solving in relation to local, national and global issues.
• Provide a safe, healthy, and attractive environment that promotes a sense of achievement and belonging for students, faculty, and staff.
• Obtain an adequate, stable form of financial support for COM-FSM.
• Demonstrate fiscal, management, and programmatic accountability.
• Provide relevant, up-to-date, quality, technical, and vocational degree and certificate programs, continuing education activities, and services enhanced by state-of-the-art technology to facilitate satisfying careers or transfer to schools in any geographic setting.
• Attract, employ, and retain highly competent faculty and staff who pursue professional development and contribute to a positive institutional culture which encourages honesty, appropriate moral values, and respect for human and cultural diversity.
• Establish partnerships with educational institutions, national and state governments, businesses, and other community agencies to identify and address common goals.
• Conduct necessary and appropriate research.

In the Performance Based Budget Institution Level and Program Outcomes and Outcomes Measures Fiscal Year 2004 (PBB 2004) document the institution reported five institutional level outcomes that were measured by performance indicators. The five institutional outcomes are effectively another set of goals for the college:

• The college will provide educational opportunity to all qualified students within the FSM as demonstrated by the diversity of students representative of the demographic make up of the general population and ethnic groups within the FSM.
• Students will be academically successful as measured by retention rates, completion rates, persistence rates, and graduation rates by college major system-wide
• Students will demonstrate their readiness for further studies as measured by transfer rate to other educational institutions of higher learning.
• Graduates will demonstrate their job readiness and competency through gaining employment within six months of graduation.
• The college will maintain the high standards of institutional capacity and educational effectiveness as demonstrated by maintaining its accreditation with the Western Association of Schools and Colleges.

While the above are forms of overall institutional goals, programs also have goals. The goals given in the Strategic Plan 2001-2006 and those in the Performance Based Budget Institution Level and Program Outcomes and Outcomes Measures Fiscal Year 2004 (PBB 2004) are not identical. The table below is only a small sample to show the shift in the goals and the focus of the goals between the Strategic Plan 2001-2006 and the Performance Based Budget for Fiscal Year 2004 that was produced in May, 2003.

There are other difficulties in mapping these two documents against each other. Items that were goals in the strategic plan are, in some instances, demoted to an activity in the later document. There are more activities in the PBB 2004 document than are listed below. In some instances the strategic plan reflects the top-down heritage that created that document with the director of research and planning central to the creation of that document. The Performance Based Budget 2004 was developed in consultation with the program heads and reflect their work on developing program outcomes for their respective divisions. Thus there is far more ownership in the Performance Based Budget document.

Note that program level goals are dynamic and continue to evolve. When the mathematics program found that the program goals for developmental mathematics and college level mathematics were essentially the same, the division opted to consider all of their math courses as part of a single program. The original division was artificial in that faculty regularly teach across the two areas. The result was a compression from five program level goals to four program student learning outcomes banks. That particular division has since mapped their Performance Based Budget assessments by indicators against their program level student learning outcomes that are assessed using an external instrument and a sample of students.

The above table represents the continued development and evolution from an indicator based system of assessment to a student learning based system of assessment in a single division.

Description of Strategic Plan Development

Institutional goals and purposes are captured in the strategic plans that are redeveloped on roughly a five year basis. Although the Strategic Plan 2001-2006 was developed primarily by the Institutional Research and Planning Office, the modified plan found in the Performance Based Budget for Fiscal Year 2004 was developed with extensive input from the program leads. Members of the college and other constituencies all have had input, through various mechanisms, into the plans of the college found in the Performance Based Budget. Thus these plans are not unknown to the members of the college, they typically represent their collective thinking. Through a variety of formal and informal mechanisms, these goals and purposes are then communicated back to the members of the college.

There remains work to be done in shifting this process from looking at indicators to the use of measurable student learning outcomes. Much of this data is couched in the language of numbers of students in sections, pass rates in courses, and other spot indicators.

Utilization of measurable student learning outcomes at the program and institutional level will require the development of new methods and instruments of measurement. The college is only beginning to engage in an internal conversation of what measurable skills a student will carry away from the college at a program and institutional level. Skills at the course level are well defined by course outlines grounded in measurable student learning outcomes that are developed by the instructors who teach those courses.

IB2. Self Evaluation

Performance Based Budget 2004

This document was prepared by the departing director of research and planning in coordination with a small but representative team of administrators at the college. There are specific goals and performance targets that the college has committed to meeting. The departure of the author of this document prior to a full year's cycle of living with the document and prior to any assessment demanded by the document leaves the college in a difficult position.

The college must report on each and every indicator, yet there is probably no one other person who fully comprehends this document in its entire scope and breadth. Any new director of research and planning will have to become intimately familiar with this document. The new director will bear the primary responsibility for coordinating the follow-on reports to this document.

The upside is that the Performance Based Budget for Fiscal Year 2004 was developed out of input from those who are leading programs. Each program or unit lead had input on the contents of the document. With many of the same leads still in place, each lead should be able to handle evaluation of the goals they specified. This argues for the importance of a decentralized assessment system.

Bachelor's Degree in Business

References to developing a bachelor's degree in business are made in the Strategic Plan and in the Performance Based Budget 2004. The college, however, is now aware that taking a second program to four year status would require full accreditation with the Accrediting Commission for Senior Colleges and Universities. The college will have to revisit these goals with this knowledge in mind.

Micronesian Studies Program

In the strategic plan for 2001-2006, under the single goal "Deliver a social science curriculum that meets the needs of FSM citizens," objective two was to "successfully implement the Micronesian Studies Program." [IB2(1)] Although the Performance Based Budget for Fiscal Year 2004 did not use the term "objective," the Micronesian Studies Program was omitted from the activities of the Social Science division. The cause for this omission is unclear and does not reflect the absence of this program. This omission may indicate that the performance based budget document was not thoroughly reviewed. This may constitute indirect evidence that the performance based budget is not well understood or widely read. The college presently does offer an Associate's degree in Micronesian Studies.

IB2. Planning Agenda

Growth from Strategic Plan to Performance Based Budget and Beyond

Any future director of research and planning will have to thoroughly understand the changes made in goals and activities between the strategic plan and the performance based budget 2004. The later document represents part of an evolution from an indicator, institution centered perspective, to a student learning outcome, student centered perspective. This evolution is not yet complete as some of the goals and outcomes in the Performance Based Budget are still being measured by indicators as opposed to outcomes.

Assessment of goals specified in the Performance Based Budget will have to be done by the program leads who helped generate those goals.

Bachelors in Business

The college will have to revisit the goal of developing a bachelor's degree in business with the new knowledge that this would require a change in accreditation. This discussion will begin within the business division and the curriculum committee. Decisions made will have to be reported for advice and consent from the cabinet and the Planning Council. Note that the proposed process begins internally and then reports results to the Planning Council. This would allow the Planning Council to meet annually to consider changes being proposed at the strategic level at the college.

Micronesian Studies Program

The Micronesian Studies Program should be included in the activities of the social science division in the Performance Based Budget for Fiscal Year 2004.

IB3. The institution assesses progress toward achieving its stated goals and makes decisions regarding the improvement of institutional effectiveness in an ongoing and systematic cycle of evaluation, integrated planning, resource allocation, implementation, and re-evaluation. Evaluation is based on analyses of both quantitative and qualitative data.

IB3. Descriptive Summary

The overall effectiveness and evaluation of programs or institutional sectors remains an area in which improvement can occur.

Some areas regularly evaluate themselves, such as the Learning Resource Center. Through surveys and indicators, the Learning Resource Center evaluates the extent to which it is meeting or not meeting the needs of its constituencies and then modifies its plans to better meet those needs. The Learning Resource Center has gone from an institutional weak point in 1994 to an institutional strength at present.

Evaluation of faculty occurs on a regular basis, both by supervisors and by students. This data is then shared with the affected faculty member in order to assist them in their own self-evaluation.

While some areas regularly evaluate themselves through established instruments or processes, other areas are less well evaluated. The result is often discussions as to the effectiveness of a program without clear data.

One example of this is the Intensive English Program (IEP). The IEP was begun in 1996 as an English as a Second Language program run at the state campuses. The program was designed to replace a remedial English program that suffered from a poor retention rate.

Although the IEP assisted in retaining students, questions have arisen at the national campus over the perceived uneven quality of the students coming from the different campuses. Some instructors feel that there are quality problems in all four state campuses. The difficulty with the discussion is that there is little in the way of non-anecdotal data to look at, nor any comparison data to equivalent programs at other institutions.

A study of the first IEP students at the Pohnpei campus in 1996, that had unavoidable design flaws, suggested that retention and subsequent graduation rates were higher for IEP alumni versus students in the remedial English program. This study was unable to come to any statistically significant conclusion as the sample sizes were too small.

Part of the complication was a high loss rate of students in the so-called control group who had taken remedial English courses instead of enrolling in the IEP program. There were too few students left to produce statistically significant comparisons of student performance in college level English courses. Other complications included the non-random nature of the study.

Virtually all of the IEP students were Pohnpei state students who lived at home and returned to their home environment and mother tongue at night. The students in the remedial English courses were principally Chuukese, Kosraen, and Yapese students (students from remote islands) who stayed in the college dormitory. These student were returning to a multi-lingual environment in which English provided a potential medium of common communication.

Ultimately the higher apparent retention rate for the IEP students drove its adoption.

Initially students were placed directly into college level English post-IEP. Instructors in those courses noted that many of these IEP students were not capable of handling college level reading and writing courses. As a result, students were later placed into what used to be remedial English courses post-IEP. This has led to questions as to whether the IEP is performing as designed or whether it could ever perform as designed. Yet discussions remain at the level of opinion and anecdotal observation in the absence of comprehensive effectiveness data or formal studies.

Another complication with this program is that, due to the organizational structure of the college, no one other than the directors of the state campuses and the president of the college has any authority over the IEP programs. When the chair of the Language and Literature Division was asked to be the coordinator for the IEP programs, he rightfully refused noting he would have no authority over nor input with respect to key aspects of the program including hiring, number of students admitted at each campus, and local curricular choices. These are all issues within the program and the program as implemented clearly does not meet the current American Association of Intensive English Programs standards.

Evaluative studies are often driven by anecdotal observations. As a result, many studies are done outside of the office of research and planning. When a new placement system for developmental mathematics that was introduced in 2000 appeared to be non-predictive of success, an informal study was undertaken by the chair of the division. [IB3(1)] The result showed that the new system was no better than random at placing students. The anecdotal observations of the developmental math instructors were confirmed: students were in the wrong courses.

This led to the development of a new placement instrument in 2001. The instrument was piloted in spring 2002. A study of the results showed high rates of student success. Coupled with the observations of the instructors involved, the new system was deemed a success.

The instrument was used in fall 2002 and the results were studied again later that fall. Preliminary results of that study confirm the test is placing students at a level in mathematics at which they can succeed. There was also the suggestion of a slight underplacement of students in the fall. [IB3(2)] The mathematics division is now making minor modifications in the instrument to correct the underplacement effect.

In a small two-year college the task of evaluating effectiveness, following up on anecdotal data, and redeveloping systems will often depend on individual units engaging actively in self-study.

When the business community indicated that the accounting graduates did not have the skills to be effective employees in local businesses, the business division undertook the development of a third year in accounting program to provide the necessary skills.

When the need arose to develop a social sciences program that would focus on the cultures and traditions of Micronesia, the social science division developed the Micronesian Studies program.

When the dormitory needed evening opportunities for recreation and shopping; recreation and maintenance teamed up to provide bus service three times a week to town for the dormitory students.

Each unit, formally or informally, identifies needs, assesses solutions, and implements improvements on an ongoing basis.

All units, however, could benefit from better data on the effectiveness of programs implemented.

IB3. Self Evaluation

The standard implies an assessment mechanism exists. The assessment mechanism that was deployed by the college in 1997 was the Institutional Effectiveness committee. This committee was tasked with, among other things, looking at the processes began as Program Health Indicators two years earlier.

The Institutional Effectiveness Committee and its successor the Assessment Committee would have periods of regular meetings punctuated by long periods of dormancy. More fundamentally, the membership of the committee was not the same as the Curriculum Committee. While the Institutional Effectiveness committee included members of the Curriculum Committee, the committee did not duplicate the membership of the Curriculum Committee in full. The result was that analysis of assessment and data based decisions made in the Institutional Effectiveness and later the Assessment Committee were not being fed back into the Curriculum Committee. During dormant periods assessment efforts tended to wane.

Records from 1997 and earlier are sketchy at best, only some of the minutes remain extant. The college did not have a director of research and planning during the critical 1995 workshop in which the division chairs first learned about program health indicators. [IB3(3)] There was a period of 191 days between July 28, 1997, and February 4, 1998, for which there are no records of a meeting occurring.

In 1998 the director of research and planning left the college. This provided an opportunity for the college to send non-administrative faculty to a critical conference on assessment and student learning outcomes led by James Ratcliff, director of the National Center for Post-Secondary Teaching and Assessment, on Guam during May of that year. [IB3(4)] With turn-over at the director of research and planning, this conference would pass information on assessment directly to faculty. This concept, of decentralizing assessment and moving assessment closer to the students, to the division chairs and their faculty is a central theme in this document. [IB3(3)]

During the time between directors in 1998, curriculum committee continued to meet, but the institutional effectiveness committee appears to have gone dormant for 309 days. No record of meetings for this time can be found. [IB3(3)]

During 1999 the college hired a new director of research and planning. The director would come on board without the experience of having taken the institution through a self study and self study report. The new director would face a heavy load of daily tasks as well coming up to speed on the recommendations made in 1998.

The Institutional Effectiveness Committee did not meet from November 1999 to November 2000. [IB3(3)] There exists the possibility that intense work associated with the Planning Council and development of the mission statement and college goals caused the Institutional Effectiveness committee to fail to convene. There are so many tasks delegated solely to the director of research and planning that the loss of the director has become a single point failure for the institution.

The committee was reconstituted in fall 2000 as the Assessment Committee. There were no other documented meetings in spring and summer 2000.

At the February 20, 2001 meeting of the Assessment committee the committee was told to be prepared to discuss the purpose of the committee, with one definition offered, "Gather data and analyze all COM-FSM programs and recommend actions for improvement." The minutes for this minute appear to be missing except for a single last page, including an item six. The committee membership included six directors, four of whom have since left the college. [IB3(5)]

During 2001 the Assessment committee would have periods in which it did not meet of 98 days, 129 days, and 48 days. [IB3(3)] There may have been as few as seven meetings during the year. Curriculum committee, in contrast, has historically met biweekly.

In 2002 the committee members were all provided with a copy of the recently approved Strategic Plan 2001-2006. The minutes also noted the development of a distance education plan for the college. Attendance, however, was not good. The meeting on January 24 had only 10 of 21 members present, although one absence was an off-island member. Many committees include members from the remote state campuses. Their participation has always been constrained by the complications of distance. In the past attempts were made to use the PeaceSat radio satellite network to include state directors in critical meetings. There was also a brief attempt in 1999 to use instant messenger services to include remote state campus personnel in a meeting. A transcriber typed the conversation in the meeting in real-time, while in another window the remote state campus personnel could respond or chime in. The transcriber would have to stop to read the input from the remote state campus. Neither system worked sufficiently well that either experiment was continued.

Today the only inclusion of remote state personnel in meetings in via the email traffic prior to a meeting during which issues that will come up in the meeting are discussed. This is not only imperfect, but it excludes the state campus member from the actual decision making vote on an issue.

During 2002 the director of research and planning departed. Prior to his departure he would form four committees to work on the self study. These committees would meet on three occasions in mid-2002. In August 2002 an acting director of research and planning would call a college-wide meeting and call for the formation of four new committees to work on the self study. The four new committees were not in any way related to the former four committees. Turn-over in a critical position, possibly unavoidable turn-over, once again impacted an assessment effort of the college.

Attendance is also impacted by the sense of institutional commitment or lack thereof. In an institution where all faculty are on limited term contracts that can be non-renewed with 60 day notice and without cause being given for non-renewal, full-time faculty are in the same position as a part-time faculty at other institutions. Coupled with the remote location of the college, a pay scale half that of equivalent positions in the United States, and the difficulties for many expatriates in living on small island, obtaining a sense of institutional commitment is difficult. This has translated, in part, to poor attendance for some committees. Assessment committee has had its attendance problems.

During 2002 a member of faculty began presentations and one-on-one training sessions with the academic chairs on student learning outcomes. This is cited in the above table in part because it constitutes evidence that work on assessment was continuing amongst the faculty during a time when the assessment committee was dormant. This is further evidence that decentralized assessment can proceed even in the absence of a functioning assessment committee. In section IB6 an alternative structure for assessment that fits with this new decentralized reality will be proposed.

In spring 2003 the Assessment committee reformed after a 361 day hiatus due again to a change in the director of research and planning position. The reformed committee would meet irregularly, with one gap of 54 days, during the spring. The acting director of research and planning departed in early August 2003 and the committee went dormant.

Complicating the ability of these committees to function was the perception by 2001 of some members that the committee work was unfocused and primarily a talking session for the then director of research and planning. At the July 7, 2001, meeting of the Assessment Committee only six of seventeen members were present. The following meeting had only five of seventeen members. Even once the term started attendance rose to only ten of nineteen appointed members.

Assessment committee meetings in 2001 and 2002 left one participant with the impression that they were addressing the need of the moment for the director of research and planning. A sense that the committee work would lead to an annual cycle of regular assessment was not evident.

Communication of decisions made within the assessment committee to those affected by the decisions was not accomplished. The March 10, 2003, meeting of the Assessment Committee passed a resolution that the development of standards for program level outcomes would be the responsibility of the Assessment Committee. This pronouncement, however, lacked input from critical non-members who were, at the same time, developing program learning outcomes based on dialog in their divisions and based on the recommendations of national organizations in their field. This isolation and detachment of the assessment committee from planning and deploying committees such as curriculum is addressed in the planning section for IB6.

Although the obvious recommendation might be to recommend that the Assessment committee meet regularly, ultimately even regular meetings will not solve the fundamental problem that planning and deployment of courses and programs occur in the Curriculum Committee while assessment is reported to the Assessment Committee. This builds into place a permanent separation of planning and deployment from assessment. If the assessment cycle is to be closed, assessment must be reported where planning and deployment occur: the Curriculum Committee.

Another factor is driving assessment away from an assessment committee led by the Institutional Research and Planning Office. This factor is that program assessment is being done in the academic divisions. Where student achievement data was the forté of the director of research and planning, the new assessment regime is being built in each division. This decentralization of assessment, and the planning recommendations that these appear to entail, are detailed in the IB6 planning agenda.

The issue of a lack of system-wide assessment of the Intensive English Program, as mentioned above, will have to be tackled by the president's office through the Institutional Research and Planning Office due to the peculiar organization of the college wherein neither the vice president for instructional affairs nor the academic division chairs have any authority over academic programs in the state campuses.

There is a need to find a way to preserve records and minutes of meetings. The effort to find minutes for the Institutional Effectiveness committee and the Assessment committee involved searching file cabinets, notebooks, hard drives, and floppy disks, all in multiple locations.

One option would be to post all minutes to the web server for the college, deleting out those materials that are strictly internal or are governed by privacy policies and regulations. Web server based storage would provide access to these materials on all six campuses and appears to be a stable and useful way to store information over the long term. Unlike paper based products, web pages do not physically degrade nor are they easily lost or misplaced. As long as the item remains linked to the rest of the college web site, an Internet search engine can find the item with relative ease.

IB3. Planning Agenda

The recommendation has been made to terminate the Assessment committee and integrate its functionality into other committees. Academic assessment will be reported into the Curriculum committee. Student support services assessment will be reported into the student support services committee. Other units will report assessment to the committees associated with that unit. This is detailed in section IB6. New efforts must be made to find ways to include the active participation of state campus personnel on other islands who are assigned to committees at the national campus on Pohnpei. The college continues to work toward an infrastructure that would support full video conferencing. This will be covered in the section on technological resources.

IB4. The institution provides evidence that the planning process is broad-based, offers opportunities for input by appropriate constituencies, allocates necessary resources, and leads to improvement of institutional effectiveness.

IB4. Descriptive Summary

The planning process at the college is driven by a wide variety of formal and informal structures.

Ongoing strategic planning and guidance is provided by the planning council, a council consisting of members of the college system at all levels, members of the community at large including traditional leaders, and members of government. [IB4(1)] The present college of Micronesia-FSM Strategic Plan 2001-2006 was developed by the planning council beginning in 1998. [IB4(2)]

Degree granting academic programs at the college also have a program advisory council consisting of members of the college who work under that program and community members who work in that field. Business/Accounting, Computer Information Systems, and Tourism and Hospitality program advisory councils have been formed and have met to provide input on those degrees. Councils are not yet formed for every degree, divisions in charge of degrees without councils are being tasked with forming councils.

Planning also occurs as a direct function of the director of research and planning.

Other input comes from a variety of committees that tackle issues on a more tactical level. Occasionally, however, these committees drive planning on a larger scale. Data that indicated the students at the college were unfit in terms of body fat is one factor driving the development of a physical education program. [IB4(3)] The development of this program came from internal processes at the college. The college recognizes that physical education alone cannot be shown to impact body fat levels. In a nation with high rates of life style diseases such as diabetes and heart disease, physical education is one of many approaches that educational institutions ought to implement.

Planning can also be driven by outside constituencies. The college is the only post-secondary institution in the Federated States of Micronesia. Hence when the national government calls on post-secondary institutions to train citizens in tourism, marine sciences, and agriculture, it is the college that must respond. Each of these calls arose from national or state strategic plans, and each call led to the development of a program at the college. The most recently added was the Hotel and Restaurant Management program.

Planning and the processes that develop new programs also occurs informally as members of the college interact with the broader community. Individual members of the college often see community needs and as a result develop programs to meet those needs. These programs are then routed through formal channels for approval, but the start of the planning process was a conversation held off campus in the evening between a member of the college community and the broader community at large.

Once a need enters the system, resource deployment depends on the personnel tasked with meeting that need. For the most part the college develops and implements programs or processes to meet the need.

There are limitations on resources at the college. If meeting a need has high costs and the demand is low, the college might not be able to meet that need.

In some cases where the demand is low and the cost is high, but the benefits are real, the process will proceed but at a slower and more affordable pace. For a number of years there has been the realization that our software systems in the Office of Admissions and Record, the Financial Aid Office, and the Business Office need to be integrated. The original designer of the database currently used in the Office of Admissions and Records was in contact with an outside consultant in 1997 and informed of this potential need. [IB4(4)]

Software that could handle all three areas has proven prohibitively expensive to implement, and may not remove all double entry of data for the units involved. Not all of this is within the control of the college: the Financial Aid Office uses required United States Department of Education software for their financial aid work. This software does not currently appear to be designed to integrate with other systems software. While the demand for integrated systems is low, there would be a real benefit both to students and to constituencies such as the FSM national and state scholarship officers.

Other limitations involve the difficulties the college faces with long term financial planning. The college remains dependent on an annual allotment from the congress of the Federated States of Micronesia. This allotment is subject to change and introduces uncertainty in the college's long term financial planning.

Another major source of funding is through the United States Pell grant to students. This funding is dependent on the continuing inclusion of the citizens of the Freely Associated States in the Pell grant program.

While at the time of the drafting of this document the college lacks a director of research and planning, key administrative sectors of the college are aware of the need to start planning for the 2007 - 2012 five-year strategic plan.

IB4. Self Evaluation

The college planning process has been broad based and has offered opportunities for input by appropriate constituencies. Resources have been and will continue to be allocated to the planning process, both strategic plans and tactical plans.

Plans have changed the programs offered at the college, but evaluation of the effectiveness of those programs remains to be done. For example, the nation has placed a high priority on local agriculture, hence the college offers a degree in agriculture. The number of students who choose this program is small, the number who graduate is also small. The college is responding to a call in the national plans, but whether the students want to study agriculture is another matter.

More recently the effort to develop an integrated performance based budget has led to modifications of the strategic plan. The performance based budget is to be based on outcomes, while the 2001 - 2006 Strategic Plan was based on objectives and health indicators. This disjunction meant that in order to submit an acceptable performance based budget, the college had to reword, and in some cases restructure the strategic plan.

The time between the approval of the strategic plan and the development of the performance based budget as a time of transition for the college. Both the presidency and the director of research and planning positions were in transition at one time or another during this effort. The result was that the May 2003 performance based budget included a de facto rewrite of the 2001 strategic plan.

IB4. Planning Agenda

The college will have to continue to work on bringing future performance based budgets into alignment with the effort to assess programs based on student learning outcomes as opposed to student achievement data. This effort will have to be coordinated by the Institutional Research and Planning Office. Wherein this effort impacts the strategic plan, the college will have to report on that impact for the input of the planning council.

IB5. The institution uses documented assessment results to communicate matters of quality assurance to appropriate constituencies.

IB5. Descriptive Summary

This standard was taken as referring to communication with external constituencies. The only formal mechanism by which the college communicates results at this time is via an annual report. The annual reports were begun in 1994. The first one was written by the director of research and planning. The most recent annual report was a combined two-year annual report for October 1, 1999 to September 30, 2000 and October 1, 2000 to September 30, 2001. The most recent annual reports were completed by the college president.

The annual reports began as a four page highlighting of events over the past year. The last report, for fiscal years 2000 and 2001, was twenty pages and included reports on instructional programs, program costs based on student seat costs from divisional budgets, enrollment statistics, program evaluation based on completion rates and graduation rates, student services, accreditation, finances, and facilities (College of Micronesia Annual Reports October 1, 1999 - September 30, 2000 and October 1, 2000 - September 30, 2001).

The Annual Report was distributed to national and state leadership, both political leadership and educational leadership. The report is also a requirement of the FSM National Government. (Public Law 7-79 Section 21 (2) (f).)

The president periodically produces updates that are distributed both internally and externally. The updates report on a variety of matters at the college, but do not usually include documented assessment results to communicate matters of quality assurance.

Communication with the nation as a whole is complicated by distance, a lack of a developed media infrastructure, and language barriers. The only media that is distributed across the FSM is a single newspaper produced on Pohnpei in English. Radio is limited to a single island, and then only some areas of the island can receive the radio signal. Cable television is presently limited in distribution, both by island and on the islands on which it is installed. All of these factors contribute to difficulty in communicating.

Some divisions are posting student achievement data and student learning outcomes assessment results to the college web site. Although Internet access remains limited, and the location of this information is not obvious, this does represent an effort to expose internal assessment data to a broader global community.

IB5. Self Evaluation

The annual report is the only broadly communicated output of the college. The amount of assessment information in it is minimal and is limited to broad student achievement data. This report was not completed for fiscal year 2002: October 1, 2001 to September 30, 2002. According to a former president of the college, another report is due within 120 days of October 1, 2003 for fiscal year 2003.

As few units at the college have yet assessed program or degree level student learning outcomes, the Annual Report cannot at this point report on learning to external constituencies. At this point the Annual Report can only report student achievement data. For those programs that have initial student learning outcomes assessment systems and data, the systems and the data can be reported.

IB5. Planning Agenda

The college will complete an annual report for fiscal year 2003. The college president will take lead responsibility for the production of this report.

IB6. The institution assures the effectiveness of its ongoing planning and resource allocation processes by systematically reviewing and modifying, as appropriate, all parts of the cycle, including institutional and other research efforts.

IB6. Descriptive Summary

At the center of the effort to produce a systematic cycle of process review is the director of research and planning and the Assessment Committee.

As the effort to shift from program or objective health indicators to program level and institutional level student learning outcomes has only begun, systematic review of these higher level outcomes have yet to occur.

Two issues have impacted the ability of the institutional effectiveness and assessment committees to function. The first is a lack of data, even with respect to indicators. The college used manual record systems up until 1995. In 1995 the college began using a home grown database for tracking basic student indicators, primarily grades. Conducting research on student achievement data has required the construction of database queries. In the past, these queries had to be put together by the director of research and planning. Two of the four research directors were unfamiliar with database queries.

The second issue has been turn-over in the director of research and planning position. The college now has its fourth director of research and planning since 1994. Each of the last three directors has either been new to the college and to Micronesia, or relatively new to the college system. Setting up and institutionalizing cycles of planning and assessment requires multiple years of effort.

Turnover in the lead position, and the only college employee specifically tasked with research, has meant continuous change in the direction and thrust of planning and assessment. Each new director requires approximately a year to comprehend the annual cycle of research and assessment needs. The director then usually needs a second year to implement changes, modifications, and additions. Obtaining buy-in from units also requires time and effort. By the time a regular cycle of planning and assessment is in place, the director has often moved on and a new director with new ideas and desires has come on board.

Another factor in the past is a result of both the paucity of data and a lack of information on who has what data. There has been a tendency on the part of some directors to blame personnel who are not responsible for gathering the data the director needs. This is sometimes a result of a misunderstanding as to who has the data. Data requests have also been made in the past without regard to the present load being carried by the unit or to the amount of labor involved in the data entry necessary to generate the desired data. /p>

Stability, continuity, and institutional memory in the director of research and planning position, possibly coupled with expansion of the position into a multi-employee office, would benefit the ability of the college to close the cycles of planning, implementation, assessment, and redevelopment.

IB6. Self Evaluation

Although both the most recent acting director of research and planning and the previous director spoke of a need to develop a published research calendar for the Institutional Research and Planning Office, no formally published calendar is extant at this time.

A proposal was made in September 2003 to integrate the functions of the presently dormant Assessment committee into the existing and active Curriculum committee and Student Services committee. This was as a result of the realization that to close the academic assessment loop that is made up of curriculum planning, development, approval, implementation, assessment, and redevelopment, the Assessment committee would have to be identical to the Curriculum committee in composition.

The loop is not currently closed by design: curriculum and programs are discussed in and initially approved by the curriculum committee. The assessment is reported into the Assessment committee. This creates a physical separation of the events that no amount of communication effort can fully bridge.

The proposal would also include having the student support services assessments reported into the existing student services committee.

At the core to the effort to improve courses and programs is assessment. Assessment permits data and fact driven decisions at the course level, program level, degree level, and institutional level. These assessments are assessments of student learning outcomes. Yet at the course level, and the program level, as envisioned in Language and Literature, Education, Social Science, and the Natural Science and Mathematics division, student learning outcome assessment will be done either by the faculty or by the chair.

This proposed change helps redistribute the load away from an over burdened central researcher and planner, without necessarily overloading the chairs. Assessment would move from a centralized research office out into the corridors and offices of academia.

Student achievement data - referred to in the past as Program Health Indicators (retention, graduation data, and the like) - would also be reported to the Curriculum Committee. The Curriculum Committee all ready has representation from each academic unit.

This proposal would mean that the director of research and planning would sit on the Curriculum and Student Services Committee as a voting member. The director could also report data relevant to assessment and assist units in making data based decisions.

The existing separate Assessment Committee would no longer exist under this proposal.

This proposal should also address an institutional weakness in the institution's difficulty in retaining directors of research and planning capable of performing the many tasks the college asks of them. The director of research and planning is presently responsible for organizing strategic planning, heading the Planning Council, institutional research, budget development, performance based budget reports, grant writing, accreditation liaison, sponsored programs, and the annual IPEDS report. The director also plays a key role in the production of the annual report.

Due to issues such as the remote location of the college, budget constraints that in turn constrain salary, and the complexity and load of the job, the college has seen instability in the director of research and planning position.

At the salary offered the college either gets people who are not competent at every single one of the needed skills, or the college attracts people who are competent and then they do not stay long as they can obtain better paying positions elsewhere. Directors have remained on board no more than two to three years. This creates instability in the assessment cycles as these cycles can take many years to complete.

Neither the Curriculum committee, nor the Student Services committee have had the same stability difficulties. Both are headed by vice presidents, positions that have remained stable over time at the college.

The college's current curriculum, student services, and assessment structure consists of three committees:

This proposal would result in there being two committees with assessment integrated into the existing committees:

Another problem that accompanies turn-over at the director of research and planning position is that each departure carries away critical information. The director of research and planning is tasked with attending crucial conferences and workshops on assessment and accreditation. With the departure of a director, much of the training and knowledge departs as well.

This is evidenced in part by what happens in the time gaps between directors. In 1998 the college sent a combined administrator-faculty team to an assessment conference in Guam. [IB3(4)] The team consisted of the director of academic programs and three faculty members. At that time there was no director of research and planning, otherwise the college would likely have sent only the director of research and planning and the director of academic programs.

Five years after the assessment conference on Guam, the director of academic programs has moved on, and the college has had a director of research and planning and an acting director of research and planning come and go. All three faculty members are still at the college, sharing and working with the knowledge imparted by that conference.

To help distribute institutional memory, a recommendation was made to separate the duties of the Accreditation Liaison Officer from the Institutional Research and Planning Office. The Accreditation Liaison Officer position would be moved back to faculty, where it has resided in the past. The proposal would not limit the Accreditation Liaison Officer position to faculty. The liaison officer could also be a member of the staff or the head of college unit such as the learning resource center.

There would be natural advantages to the Accreditation Liaison Officer being a faculty member. Faculty would be in the best position to share accreditation knowledge with other faculty on a daily basis. Under the new standards, student learning and its assessment are the core to accreditation. The Accreditation Liaison Officer would be in an ideal position to share knowledge of how other schools are handling student learning outcomes, program assessment, and other areas of growth and change in other schools in the Western Association of Schools and Colleges.

A faculty based Accreditation Liaison Officer who taught classes would also be seen as a teaching colleague, as opposed to an administrator. Despite the best efforts of institutions and people, those who teach in a classroom will likely always distinguish between those who teach and those who administer without ever teaching. A faculty Accreditation Liaison Officer would be able pilot new concepts in their own classrooms first, and then bring to other faculty a proven-to-work prototype rather than merely a hypothetical concept. The college is small enough that the Accreditation Liaison Officer could continue to teach and handle their accreditation duties.

The likelihood that the college would lose both an accreditation liaison officer and a director of research and planning at the same time would be reduced if the Accreditation Liaison Officer were separated. Currently the college loses an Accreditation Liaison Officer at the exact same instant it loses a director of research and planning.

IB6.Planning Agenda

As outlined above, the college will integrate assessment into the appropriate committees. This reorganization effort is being led by the president and the vice presidents and will occur during this academic year.

IB7. The institution assesses its evaluation mechanisms through a systematic review of their effectiveness in improving instructional programs, student support services, and library and other learning support services

IB7. Descriptive Summary

Program evaluation has meant, until only recently, a review of student achievement data. Although this data has helped some units improve the effectiveness of a unit, the evaluation mechanism itself, the student achievement data, has not been assessed. The effort in the academic programs is to define program student learning outcomes, to evaluate those outcomes, and then later to examine whether the assessment methods used were effective. The later is a meta-analysis that will best be done after a couple of evaluation cycles.

The faculty, through the auspices of the curriculum committee, did take a look at the faculty evaluation system. The present form used by personnel involves being rated on a scale from one to ten and has been viewed by faculty as not only having nothing to do with effectiveness but with being irrelevant to faculty. Some, however, view the proposed replacement system as equally irrelevant to improving effectiveness. Others feels improving the effectiveness of faculty is not the goal of the evaluation form or the faculty evaluation by supervisor system and prefer to handle improving faculty effectiveness outside of the realm of the supervisor evaluation. There remains a division of opinions as to whether faculty evaluation should be used as a stick to drive improvements in effectiveness or whether evaluation should remain summative while other mechanisms are deployed to improve instructor effectiveness.

IB7. Self Evaluation

Although the college has been conducting program review based on student achievement data, and, in some cases, advisory councils, evaluation from a program student learning outcomes perspective has only just begun. This effort represents a de facto assessment of our present evaluation mechanisms and a decision that these evaluations could be improved by student learning data.

IB7. Planning Agenda

Although a number of units have been working on program student learning outcomes as part of their program evaluation effort, the vice president for instructional affairs circulated two policy statements in September 2003 to formalize this process. The two polices, Policy on Instructional Programs Evaluation and Action Plan for the Program Evaluation for School Year 2003-2004 place into writing policies, guidelines, procedures, and time-lines for program assessment. The feasibility of the time lines has yet to be evaluated by the division leaders.

End Note

Link to evidence page