COMET Admissions Examination Statistical Summaries Spring 2006
Basic Statistics
The COMET admissions examination consisted of a reading section, grammar and vocabulary section, an essay, and a math section consisting of four subsections. The basic statistics for this sections and subsections are presented in the table one below.
Statistic |
m1 |
m2 |
m3 |
m4 |
SM |
Read |
Write |
Grammar |
count |
1795 |
1794 |
1795 |
1795 |
1806 |
1797 |
1783 |
1796 |
min |
0 |
0 |
0 |
0 |
0 |
1 |
0 |
3 |
max |
10 |
19 |
10 |
9 |
37 |
35 |
40 |
78 |
range |
10 |
19 |
10 |
9 |
37 |
34 |
40 |
75 |
mode |
9 |
3 |
2 |
2 |
15 |
14 |
0 |
33 |
median |
7 |
3 |
3 |
2 |
15 |
14 |
22 |
35 |
mean |
6.6897 |
3.6856 |
3.5064 |
2.4345 |
16.2148 |
14.8114 |
20.6932 |
37.0434 |
stdev |
2.2997 |
2.1324 |
2.1969 |
1.4336 |
6.2405 |
5.1457 |
12.4327 |
14.6300 |
coef var. |
0.34 |
0.58 |
0.63 |
0.59 |
0.38 |
0.35 |
0.60 |
0.39 |
low95 |
6.58 |
3.59 |
3.40 |
2.37 |
15.93 |
14.57 |
20.12 |
36.37 |
mean |
6.69 |
3.69 |
3.51 |
2.43 |
16.21 |
14.81 |
20.69 |
37.04 |
high95 |
6.80 |
3.78 |
3.61 |
2.50 |
16.50 |
15.05 |
21.27 |
37.72 |
stand error |
0.05 |
0.05 |
0.05 |
0.03 |
0.15 |
0.12 |
0.29 |
0.35 |
skew |
-0.45 |
0.79 |
0.70 |
0.69 |
0.52 |
0.67 |
-0.27 |
0.40 |
kurtosis |
-0.57 |
1.29 |
0.02 |
0.88 |
0.09 |
0.77 |
-1.04 |
-0.49 |
Table one.
m1 is the first math subsection and contains ten arithmetic level problems.
m2 is the second math subsection and contains ten prealgebra level problems.
m3 is the third math subsection and contains ten elementary algebra problems.
m4 is the fourth math subsection and contains ten college level algebra problems.
SM is the sum of the math subsections and is out of forty.
Reading is a reading score out of 40.
Writing (essay) is the essay score out of 40.
Grammar is the grammar and vocabulary score out of 80.
The reading section average is separated from random by less than one standard deviation. While significant due to the sample size, the low average reinforced analysis in the language and literature division that the reading section might be problematic.
Correlations
The correlations between the subsections is given in table two below.
Correlations |
SM |
Read |
Write |
Grammar |
SM |
1.00 |
0.55 |
0.55 |
0.60 |
Read |
0.55 |
1.00 |
0.59 |
0.64 |
Write |
0.55 |
0.59 |
1.00 |
0.69 |
Grammar |
0.60 |
0.64 |
0.69 |
1.00 |
Table two.
Although the first two subsections of the math test were rewritten to require less reading ability, the math subsections sum remains, given the sample size, correlated moderately to moderately-strong with the English sections.
Writing correlated more strongly with the grammar section that with the reading section. Past experience in the language and literature division indicated that the writing section, the essay, was a trustworthy indicator of student ability. The strong correlation between the trusted writing section and the grammar section and the lower correlation of both to reading led the language and literature division to request that the reading score not be used in the final admissions decision this year. The division plans to redesign this section.
Distributions
The distribution of scores on each section is given in chart one. The score is on the horizontal axis, the vertical axis is the number of students with that score.
Chart
one.
With the exception of the writing section, the other sections distributed in a manner which is considered statistically reasonable. That is, the frequency (the number of students) rises with the score towards the mean, and then falls as the score continues to rise past the mean to the maximum possible score. If the distribution (the shape of the curve) is a symmetrical heap, the most common score is also the mean score as well as the median score. This distribution is sometimes called a "bell curve" and is mathematically described by the normal or student's t-distribution.
In the past, the writing section, the essay, has in performed in a statistically reasonable fashion except for the number of students who score a sum of zero. This was true again this year with 235 scores of zero. Yet the removal of those 235 scores would not return the distribution this year to a bell curve.
The most common score on the essay, except for zero, was the maximum possible score of 40 with seventy-four students attaining a score of 40. The frequency rises steadily from eight students with a score of 2 to fifty-nine students with a score of 20. From a score of 20 to 35 the distribution is fairly flat. Then the distribution drops slightly until a score of 40.
The behavior of the essay section suggests that the section did not distinguish well among the seventy-four students with a score of 40. Bear in mind that a score of forty represents a paper with some errors and omissions of grammar, vocabulary, organization, cohesion, and content coverage. The distribution argues that the rubric should now include a score of five in each of the five areas. This should spread the seventy-four scores of 40 across the ten-point range from 40 to 50. This might also provide enough information to potentially place students in a writing course at the college.
Rank order
With the recommendation to remove the reading section from the calculations, the admissions board approved a rank-order formula using z-scores on the mathematics section, the grammar and vocabulary section, and the writing (essay) section. The formula placed a 50% weight on the essay, 25% on grammar and vocabulary, and a 25% weight on mathematics. The individual weighted z-scores were added to generate a sum of z-scores. The resulting z-score was linearly transformed to produced the final COMET score. The basic statistics for the z-score sum and the COMET score are given in table three.
Statistic |
z-score sum |
COMET |
count |
1806 |
1806 |
min |
-8.46 |
84.35 |
max |
8.23 |
997.78 |
range |
16.69 |
913.43 |
mode |
-5.56 |
243.17 |
median |
0.05 |
549.95 |
mean |
-0.06 |
544.34 |
stdev |
3.53 |
193.05 |
cv |
-62.51 |
0.35 |
Table three.
A COMET score of 700 was approved as the cut-off for regular admission to an associate degree program. A COMET score of 250 was established as the minimum score for admission to a certificate program. Students who have scored between 250 and 700 will be eligible for either the Intensive English Instruction sequence or a certificate program based on their individual position on a rank order by state. Decisions on the exact nature of the IEI admissions mechanism are still to be determined.
Bear in mind that the college now operates based on fixed resources and specific projections of the number students that the college can serve. Each site has an target enrollment number and an maximim enrollment capacity limit. Like many smaller colleges, the college opts to admit a specific number of students.
The result is that the target admissions number is a fixed number. Put another way, there is not the possibility at present of all high schools increasing the number of students attaining regular admission to the college. If one high school sees an increase in the number of students who have attained regular admission, then some other high school (or high schools) must have seen a decrease in the number of students who gained regular admission. Thus admission is competitive and is mathematically termed a zero-sum game.
High schools that are using increases in regular admissions as an indicator do so at their own peril. If another high school improves by a larger amount, then that first high school could see a drop in admissions despite real improvement in their own educational processes.
Table four records the distribution by state and high school of the students into the three categories delineated above.
State |
School |
Non-admitted |
Certificate/IEI |
Associate |
Total |
Chuuk |
(empty) |
1 |
|
|
1 |
Berea |
2 |
19 |
3 |
24 |
|
CHS |
65 |
204 |
5 |
274 |
|
CSC |
7 |
71 |
1 |
79 |
|
CSDA |
|
10 |
1 |
11 |
|
Faichuk |
16 |
30 |
|
46 |
|
Mizpah |
|
21 |
1 |
22 |
|
Nukuno |
1 |
10 |
1 |
12 |
|
PLHA |
4 |
42 |
1 |
47 |
|
Saramen |
|
30 |
25 |
55 |
|
SNHS |
5 |
41 |
|
46 |
|
WHS |
15 |
72 |
2 |
89 |
|
Xavier |
|
|
35 |
35 |
|
Chuuk |
|
116 |
550 |
75 |
741 |
Kosrae |
KHS |
|
86 |
55 |
141 |
Kosrae Result |
|
|
86 |
55 |
141 |
Pohnpei |
(empty) |
2 |
|
|
2 |
? |
|
1 |
|
1 |
|
CCA PNI |
|
|
10 |
10 |
|
CCA PNI Tchrs |
|
4 |
3 |
7 |
|
Madolehnihmw |
|
69 |
29 |
98 |
|
OHWA |
|
8 |
1 |
9 |
|
PICS |
|
269 |
152 |
421 |
|
PNI campus |
|
82 |
2 |
84 |
|
SDA PNI |
|
9 |
28 |
37 |
|
Pohnpei |
|
2 |
442 |
225 |
669 |
Yap |
(empty) |
|
2 |
|
2 |
DOE |
|
|
1 |
1 |
|
NICHS |
|
45 |
5 |
50 |
|
OIHS |
|
26 |
8 |
34 |
|
YCS |
|
3 |
|
3 |
|
YHS |
4 |
70 |
42 |
116 |
|
YMH |
|
6 |
1 |
7 |
|
YSC |
|
30 |
3 |
33 |
|
YSDA |
|
2 |
7 |
9 |
|
Yap Result |
|
4 |
184 |
67 |
255 |
Total |
|
122 |
1262 |
422 |
1806 |
Table four.
Note that the table above reports the high school name reported by those taking the test. The decision on the IEI admissions mechanism was still pending at the time this report was written.
The non-admitted status is not equivalent to the the "limbo" status discussed in prior years. "Limbo" was an arbitrary term used to refer to students who had a z-score sum on reading, writing, and grammar which was less than negative three. The nearest equivalent this year would be students below a COMET score of 328. Bear in mind that the calculation this year involves mathematics, grammar, and a weighted essay score.
The average COMET score seen in table five represents a measure of the relative performance on the writing (essay) section, the grammar section, and the mathematics section.
School |
Mean COMET |
School |
Mean COMET |
Xavier |
878.35 |
Overall |
544.34 |
CCA PNI |
853.2 |
Overall |
544.34 |
YSDA |
775.26 |
Mizpah |
536.25 |
SDA PNI |
771.19 |
NICHS |
534.41 |
CCAP tch |
685.67 |
PSC |
504.34 |
Saramen |
680.05 |
YCS |
503.63 |
PICS |
657.22 |
Berea |
488.63 |
Kosrae |
650.08 |
YMH |
473.03 |
KHS |
650.08 |
Nukuno |
442.68 |
Pohnpei |
641.51 |
Chuuk |
421.8 |
Madol HS |
635.48 |
PLHA |
417.05 |
YHS |
613.82 |
SNHS |
398.43 |
CSDA |
601.12 |
CSC |
389 |
OIHS |
595.03 |
WHS |
370.87 |
OHWA |
591.4 |
CHS |
341.51 |
Yap |
587.06 |
Faichuuk |
296.32 |
YSC |
556.51 |
|
|
Overall |
544.34 |
|
|
Table five.
A COMET average below 400 is exceptionally weak and below 328 represents essentially random performance on all included subsections. As this is the first year with a non-admission category, a conservative cut-off at 250 was established.
While the individual high school scores are of interest to individual schools, state educational leadership might consider the mean performance of all of the schools in their state. Note that Kosrae reported results from only a single school, thus the state mean is the mean for that school.
Where a performance is strong, no one effort can be credited with the success. Where a performance is weak, no one lack can be faulted as the cause of the poor performance. Education is a set of canoes with many people paddling. Students, parents, teachers, administrators, state leadership, and special programs should all share in the credit where credit is due, in the blame where performances are weak.
Table five is depicted graphically in chart two.
95% Confidence intervals for the COMET mean score by high school
Determining whether one high school is statistically significantly stronger in rank order than another high school requires determining whether the mean COMET scores are sufficiently separated. The following table provides the exact values for the 95% confidence interval for the COMET means by high school.
School |
n |
stdev |
low95% |
high95% |
mean |
Xavier |
35 |
56.61 |
858.91 |
897.8 |
878.35 |
CCA PNI |
10 |
85.11 |
792.32 |
914.09 |
853.2 |
YSDA |
9 |
88.13 |
707.51 |
843 |
775.26 |
SDA PNI |
37 |
132.17 |
727.12 |
815.25 |
771.19 |
Saramen |
55 |
84.58 |
657.19 |
702.92 |
680.05 |
PICS |
421 |
131.04 |
644.66 |
669.77 |
657.22 |
KHS |
141 |
134.72 |
627.65 |
672.51 |
650.08 |
Madolehnihmw |
98 |
114.72 |
612.48 |
658.48 |
635.48 |
YHS |
116 |
176.86 |
581.3 |
646.35 |
613.82 |
CSDA |
11 |
81.95 |
546.06 |
656.17 |
601.12 |
OIHS |
34 |
121.71 |
552.56 |
637.5 |
595.03 |
OHWA |
9 |
93.84 |
519.28 |
663.53 |
591.4 |
YSC |
33 |
106.97 |
518.58 |
594.44 |
556.51 |
Overall |
1806 |
193.05 |
535.43 |
553.25 |
544.34 |
Mizpah |
22 |
110.9 |
487.08 |
585.42 |
536.25 |
NICHS |
50 |
125.75 |
498.67 |
570.15 |
534.41 |
PNI campus |
84 |
132.18 |
475.65 |
533.02 |
504.34 |
Berea |
24 |
182.21 |
411.69 |
565.57 |
488.63 |
YMH |
7 |
169.43 |
316.34 |
629.73 |
473.03 |
Nukuno |
12 |
177.65 |
329.81 |
555.56 |
442.68 |
PLHA |
47 |
124.03 |
380.63 |
453.46 |
417.05 |
SNHS |
46 |
108.2 |
366.3 |
430.57 |
398.43 |
CSC |
79 |
117.02 |
362.79 |
415.21 |
389 |
WHS |
89 |
127.22 |
344.07 |
397.67 |
370.87 |
CHS |
274 |
121.3 |
327.09 |
355.94 |
341.51 |
Faichuk |
46 |
98.31 |
267.13 |
325.52 |
296.32 |
Table six.
Thus, for example, while PICS has a higher mean COMET score than KHS, KHS high school's 95% confidence interval spans 22 points on either side of the KHS mean COMET of 650.08. The PICS mean of 657.22 is only 7.14 points away. As the separation does not exceed 22 points, the two schools are in a statistical dead heat. Table six above is graphically shown in chart three on the next page.
The vertical bars show the extent of the 95% confidence interval for the mean score. Where a bar for one school overlaps the mean ("the ball") for another school, the former is not statistically separated from the mean of the later. Note that the upper limit for the possible mean for Faichuuk high school is less than 328, thus confirming that in terms of the admissions examination, Faichuuk scored no different from students randomly choosing answers.
Chart
three.
Writing (essay) section
The writing section is considered by both the language and literature division and by those from other divisions who mark this section to possibly be singularly indicative of the capacity of students to academically succeed at the college – if there is such a thing as a single indicator. The maximum possible score is a forty, and there is a general sense that a score above 32 indicates a potential to succeed at the college. Scores of twenty and under are considered to be significantly weak and raise questions as to whether these students can tackle any program that requires the ability to communicate in written English. Table six records the mean essay scores by state and by high school.
School |
essay mean |
School |
essay mean |
CCA PNI |
38.7 |
Overall |
20.69 |
CCAP tch |
37.43 |
Mizpah |
20.64 |
Xavier |
36.86 |
YCS |
17.33 |
SDA PNI |
34.89 |
Berea |
16.79 |
YSDA |
34.11 |
PSC |
16.77 |
Saramen |
33.56 |
Nukuno |
15.5 |
Kosrae |
28.48 |
SNHS |
14.49 |
KHS |
28.48 |
Chuuk |
13.73 |
OIHS |
27.12 |
YMH |
13 |
CSDA |
27.09 |
CSC |
12.91 |
YHS |
26.86 |
WHS |
12.52 |
OHWA |
26.11 |
PLHA |
12.15 |
PICS |
25.56 |
CHS |
7.33 |
Pohnpei |
25.06 |
Faichuuk |
5.14 |
Yap |
24.98 |
|
|
Madol HS |
24.42 |
|
|
YSC |
23.03 |
|
|
NICHS |
21.63 |
|
|
Overall |
20.69 |
|
|
Table seven.
The data in table seven is shown in chart four on the next page.
The mean essay scores suggest that Kosrae, Pohnpei, and Yap states are, to differing degrees, accomplishing the task of instructing students in the skills necessary to compose written English essays. As a state, despite the impact of Xavier and Saramen Chuuk Academy, the Chuuk state average is abysmal. Looking at the individual public high schools in Chuuk indicates an almost complete lack of ability to accomplish a written composition in the English language.
While some fields such as science have significant support costs, English language and literature is likely the least expensive program on a per student basis. This is not likely a failure due to a lack of funding alone.
The 95% confidence interval for the mean writing (essay) score allows one to determine whether the means for two schools are statistically significantly separated.
State |
School |
n |
mean |
stdev |
95% low |
95% high |
mean |
Chuuk |
Xavier |
35 |
36.86 |
3.66 |
35.6 |
38.11 |
36.86 |
Chuuk |
Saramen |
55 |
33.56 |
5.56 |
32.06 |
35.07 |
33.56 |
Chuuk |
CSDA |
11 |
27.09 |
6.95 |
22.42 |
31.76 |
27.09 |
Chuuk |
Mizpah |
22 |
20.64 |
9.2 |
16.56 |
24.72 |
20.64 |
Chuuk |
Berea |
24 |
16.79 |
12.83 |
11.37 |
22.21 |
16.79 |
Chuuk |
Nukuno |
12 |
15.5 |
11.29 |
8.33 |
22.67 |
15.5 |
Chuuk |
SNHS |
45 |
14.49 |
9.46 |
11.65 |
17.33 |
14.49 |
Chuuk |
Chuuk |
728 |
13.73 |
12.7 |
12.8 |
14.65 |
13.73 |
Chuuk |
CSC |
79 |
12.91 |
10.06 |
10.66 |
15.16 |
12.91 |
Chuuk |
WHS |
86 |
12.52 |
9.91 |
10.4 |
14.65 |
12.52 |
Chuuk |
PLHA |
47 |
12.15 |
10.37 |
9.1 |
15.19 |
12.15 |
Chuuk |
CHS |
270 |
7.33 |
8.95 |
6.26 |
8.4 |
7.33 |
Chuuk |
Faichuuk |
42 |
5.14 |
8.56 |
2.48 |
7.81 |
5.14 |
Kosrae |
Kosrae |
141 |
28.48 |
8.09 |
27.13 |
29.83 |
28.48 |
Kosrae |
KHS |
141 |
28.48 |
8.09 |
27.13 |
29.83 |
28.48 |
Overall |
Overall |
1783 |
20.69 |
12.43 |
20.12 |
21.27 |
20.69 |
Pohnpei |
CCA PNI |
10 |
38.7 |
1.57 |
37.58 |
39.82 |
38.7 |
Pohnpei |
CCAP tchrs |
7 |
37.43 |
2.94 |
34.71 |
40.14 |
37.43 |
Pohnpei |
SDA PNI |
36 |
34.89 |
4.68 |
33.31 |
36.47 |
34.89 |
Pohnpei |
Ohwa |
9 |
26.11 |
7.69 |
20.2 |
32.02 |
26.11 |
Pohnpei |
PICS |
420 |
25.56 |
8.14 |
24.78 |
26.34 |
25.56 |
Pohnpei |
Pohnpei |
665 |
25.06 |
9.45 |
24.35 |
25.78 |
25.06 |
Pohnpei |
Madol HS |
97 |
24.42 |
9.1 |
22.59 |
26.26 |
24.42 |
Pohnpei |
PSC |
84 |
16.77 |
10.53 |
14.49 |
19.06 |
16.77 |
Yap |
YSDA |
9 |
34.11 |
6.07 |
29.44 |
38.78 |
34.11 |
Yap |
OIHS |
34 |
27.12 |
9.04 |
23.96 |
30.27 |
27.12 |
Yap |
YHS |
111 |
26.86 |
11.19 |
24.75 |
28.96 |
26.86 |
Yap |
Yap |
249 |
24.98 |
10.71 |
23.64 |
26.32 |
24.98 |
Yap |
YSC |
33 |
23.03 |
7.33 |
20.43 |
25.63 |
23.03 |
Yap |
NICHS |
49 |
21.63 |
9.43 |
18.92 |
24.34 |
21.63 |
Yap |
YMH |
7 |
13 |
14.71 |
-0.6 |
26.6 |
13 |
Table eight.
Chart five depicts the 95% confidence interval for the mean writing (essay) scores, with the high schools sorted by state and then in descending writing score order. The vertical bars are the extent of the 95% confidence interval for the sample mean. Where a vertical line for one school overlaps the mean (the center ball) of a another school, the first school is not statistically significantly separated from the second school's mean.
Chart
five.
Changes in writing (essay) performance 2005 to 2006
During the marking process some graders noted that the essays appeared to be slightly stronger this year versus last year. Although year-to-year comparisons are always risky due to potential differences in the grading teams and other confounding factors, showing that there was no significant change would disconfirm the possibility the essays improved.
The mean scores by high school were compared year-to-year. The results are seen in table nine. NS indicates that the year-to-year change is not significant.
|
|
2005 Results |
2006 results |
|
||||
State |
School |
n |
mean |
stdev |
n |
mean |
stdev |
Significance |
Chuuk |
Chuuk |
620 |
11.67 |
11.11 |
728 |
13.73 |
12.7 |
Yes, change: 2.06 |
Chuuk |
Berea Christian High School |
20 |
18 |
10.46 |
24 |
16.79 |
12.83 |
NS |
Chuuk |
Chuuk High School |
224 |
7.71 |
9.61 |
270 |
7.33 |
8.95 |
NS |
Chuuk |
Chuuk SDA |
11 |
19.36 |
10.81 |
11 |
27.09 |
6.95 |
NS |
Chuuk |
Chuuk State Campus |
43 |
11.37 |
8.56 |
79 |
12.91 |
10.06 |
NS |
Chuuk |
Mizpah Christian High School |
13 |
17.15 |
8.3 |
22 |
20.64 |
9.2 |
NS |
Chuuk |
Nukuno Christian High School |
10 |
11.2 |
6.75 |
12 |
15.5 |
11.29 |
NS |
Chuuk |
Pentecostal Lighthouse Academy |
52 |
8.44 |
7.61 |
47 |
12.15 |
10.37 |
Yes, change: 3.71 |
Chuuk |
Saramen Chuuk Academy |
47 |
22.98 |
8.93 |
55 |
33.56 |
5.56 |
Yes, change: 10.58 |
Chuuk |
Southern Nomwoneas HS |
59 |
11.07 |
8.87 |
45 |
14.49 |
9.46 |
NS |
Chuuk |
Weno High School |
117 |
8.85 |
8.17 |
86 |
12.52 |
9.91 |
Yes, change: 3.67 |
Chuuk |
Xavier High School |
24 |
37.71 |
3.22 |
35 |
36.86 |
3.66 |
NS |
Kosrae |
Kosrae (KHS) |
191 |
23.24 |
8.13 |
141 |
28.48 |
8.09 |
Yes, change: 5.24 |
Kosrae |
Kosrae High School |
191 |
23.24 |
8.13 |
141 |
28.48 |
8.09 |
Yes, change: 5.24 |
Pohnpei |
Pohnpei |
643 |
23.44 |
8.48 |
665 |
25.06 |
9.45 |
Yes, change: 1.62 |
Pohnpei |
Calvary Christian Academy P |
15 |
32.47 |
5.76 |
10 |
38.7 |
1.57 |
Yes, change: 6.23 |
Pohnpei |
Ohwa Christian High School |
7 |
26.86 |
6.23 |
9 |
26.11 |
7.69 |
NS |
Pohnpei |
Pohnpei Islands Central School |
505 |
22.63 |
8.07 |
420 |
25.56 |
8.14 |
Yes, change: 2.93 |
Pohnpei |
Pohnpei SDA |
32 |
31.47 |
6.26 |
36 |
34.89 |
4.68 |
Yes, change: 3.42 |
Pohnpei |
Pohnpei State Campus |
60 |
19.78 |
8.64 |
84 |
16.77 |
10.53 |
NS |
Yap |
Yap |
228 |
23.96 |
9.76 |
249 |
24.98 |
10.71 |
NS |
Yap |
Neighboring Islands High School |
41 |
19.54 |
6.98 |
49 |
21.63 |
9.43 |
NS |
Yap |
Outer Islands High School |
31 |
19.65 |
9.18 |
34 |
27.12 |
9.04 |
Yes, change: 7.47 |
Yap |
Yap High School |
121 |
25.37 |
10.48 |
111 |
26.86 |
11.19 |
NS |
Yap |
Yap Medical Hospital |
19 |
24.05 |
5.4 |
7 |
13 |
14.71 |
NS |
Yap |
Yap SDA |
12 |
32.42 |
5.74 |
9 |
34.11 |
6.07 |
NS |
Yap |
Yap State Campus |
4 |
34 |
8.16 |
33 |
23.03 |
7.33 |
NS |
Table nine.
The only changes which were large enough to attain significance were positive changes. No school dropped by a statistically significant amount.
Mathematics section
Year-to-year mathematics subsection performance
Table ten records the year-to-year performance from 2005 to 2006 on the mathematics subsections. The table also shows the averages for the top four-hundred rank order students on each subsection.
Column one |
2005 |
4.00 |
2006 |
6.69 |
|
2006 top 400 |
8.57 |
|
Column two |
2005 |
3.25 |
2006 |
3.69 |
|
2006 top 400 |
5.74 |
|
Column three |
2005 |
3.08 |
2006 |
3.51 |
|
2006 top 400 |
5.74 |
|
Column four |
2005 |
2.25 |
2006 |
2.43 |
|
2006 top 400 |
3.27 |
Table ten.
Each subsection showed a small improvement, with the largest improvement seen in the first subsection. This improvement is thought to be due to the removal of English, to the extent that it could be removed, from the first subsection. Failure by a student to succeed in the first subsection is now a strong indicator of a failure of the student to perform basic arithmetic operations correctly.
The redesign of the mathematics subsections with the accompanying improvement in scores, coupled with a curriculum realignment within developmental mathematics, led to a redesign of the mathematics placement function.
Mean Mathematics Scores by High School
The mean score by high school for the mathematics section represents the average number correct out of forty for a high school.
School |
Math |
School |
Math |
Xavier |
25.54 |
Mizpah |
14.32 |
CCA PNI |
23.8 |
NICHS |
14.3 |
SDA PNI |
21.08 |
OIHS |
14.29 |
PICS |
20.58 |
PLHA |
14.28 |
KHS |
20.06 |
Berea |
13.5 |
Madol HS |
19.88 |
CCAP Tchrs |
12.57 |
YSDA |
16.22 |
YMH |
12.43 |
Overall |
16.21 |
CHS |
12.04 |
YHS |
16.21 |
SNHS |
11.28 |
PSC |
16.01 |
CSC |
11.11 |
YSC |
15.88 |
Nukuno |
10.75 |
OHWA |
15.33 |
WHS |
10.38 |
CSDA |
15 |
Faichuuk |
8.93 |
Saramen |
14.6 |
|
|
Table eleven.
A student blindly guessing on all forty five-option multiple choice questions should get at least eight correct. Given that the first few problems involve basic arithmetic, scores around twelve or under are extremely weak performances. Table eleven is depicted graphically in chart six.
The performance at Nukuno was statistically indistinguishable from a random score of eight. Weno high school and Faichuuk high school performed so poorly in both mathematics and English that there is merit to an argument which would shift all of whatever resources are being spent on mathematics education to the teaching of English. These schools will not do significantly worse by not teaching mathematics – there is no where down for them to go from the their current status.
Chuuk high school, while exceptionally weak in performance, still outperformed the other public high schools in Chuuk on the mathematics section. Given the poor performance of Faichuuk on the essay and mathematics sections, there is scant evidence at present that dispersing the students away from Weno will lead to improved academic results.
Chart seven on the next page depicts the 95% confidence interval for the mean mathematics scores, with the high schools sorted by state and then in descending mathematics score order. The vertical bars are the extent of the 95% confidence interval for the sample mean. Where a vertical line for one school overlaps the mean (the center ball) of a another school, the first school is not statistically significantly separated from the second school's mean.
Chart
seven.
Mathematics Placement Data
Incoming freshmen placement over the past three fall terms is given in the table below:
Course |
Fall 03 |
Fall 05 |
Fall 06 |
MS 090 |
162 |
184 |
75 |
MS 095 |
94 |
80 |
144 |
MS 098 |
101 |
77 |
34 |
MS 100 |
70 |
105 |
139 |
MS 101/150 |
25 |
17 |
30 |
Table twelve.
The mathematics placement by high school is given in the table below:
|
|
Non-admit |
Certificate/IEI |
Regular admission |
Sum |
||||||||||||||||
State |
School |
90 |
95 |
90 |
95 |
98 |
100 |
90 |
95 |
98 |
100 |
101 |
|
||||||||
Chuuk |
(empty) |
1 |
|
|
|
|
|
|
|
|
|
|
1 |
||||||||
Berea |
2 |
|
18 |
1 |
|
|
1 |
2 |
|
|
|
24 |
|||||||||
CHS |
65 |
|
190 |
14 |
|
|
3 |
1 |
|
1 |
|
274 |
|||||||||
CSC |
7 |
|
62 |
8 |
|
1 |
1 |
|
|
|
|
79 |
|||||||||
CSDA |
|
|
8 |
2 |
|
|
1 |
|
|
|
|
11 |
|||||||||
Faichuk |
16 |
|
30 |
|
|
|
|
|
|
|
|
46 |
|||||||||
Mizpah |
|
|
20 |
1 |
|
|
1 |
|
|
|
|
22 |
|||||||||
Nukuno |
1 |
|
10 |
|
|
|
1 |
|
|
|
|
12 |
|||||||||
PLHA |
4 |
|
32 |
8 |
2 |
|
|
|
1 |
|
|
47 |
|||||||||
Saramen |
|
|
28 |
1 |
|
1 |
15 |
9 |
|
1 |
|
55 |
|||||||||
SNHS |
5 |
|
39 |
2 |
|
|
|
|
|
|
|
46 |
|||||||||
WHS |
15 |
|
69 |
3 |
|
|
2 |
|
|
|
|
89 |
|||||||||
Xavier |
|
|
|
|
|
|
3 |
6 |
1 |
24 |
1 |
35 |
|||||||||
Kosrae |
KHS |
|
|
46 |
36 |
|
4 |
4 |
19 |
1 |
23 |
8 |
141 |
||||||||
Pohnpei |
(empty) |
1 |
1 |
|
|
|
|
|
|
|
|
|
2 |
||||||||
? |
|
|
1 |
|
|
|
|
|
|
|
|
1 |
|||||||||
CCA PNI |
|
|
|
|
|
|
|
6 |
|
4 |
|
10 |
|||||||||
CCA PNI Tchrs |
|
|
4 |
|
|
|
2 |
1 |
|
|
|
7 |
|||||||||
Madolehnihmw |
|
|
19 |
48 |
|
2 |
1 |
20 |
1 |
7 |
|
98 |
|||||||||
OHWA |
|
|
3 |
5 |
|
|
|
1 |
|
|
|
9 |
|||||||||
PICS |
|
|
97 |
135 |
24 |
13 |
11 |
39 |
23 |
60 |
19 |
421 |
|||||||||
PNI campus |
|
|
43 |
31 |
5 |
3 |
|
|
|
1 |
1 |
84 |
|||||||||
SDA PNI |
|
|
7 |
|
1 |
1 |
8 |
5 |
3 |
12 |
|
37 |
|||||||||
Yap |
(empty) |
|
|
2 |
|
|
|
|
|
|
|
|
2 |
||||||||
DOE |
|
|
|
|
|
|
1 |
|
|
|
|
1 |
|||||||||
NICHS |
|
|
32 |
13 |
|
|
1 |
1 |
|
3 |
|
50 |
|||||||||
OIHS |
|
|
18 |
8 |
|
|
5 |
3 |
|
|
|
34 |
|||||||||
YCS |
|
|
3 |
|
|
|
|
|
|
|
|
3 |
|||||||||
YHS |
4 |
|
40 |
29 |
1 |
|
7 |
28 |
4 |
2 |
1 |
116 |
|||||||||
YMH |
|
|
5 |
1 |
|
|
1 |
|
|
|
|
7 |
|||||||||
YSC |
|
|
17 |
12 |
1 |
|
2 |
|
|
1 |
|
33 |
|||||||||
YSDA |
|
|
1 |
1 |
|
|
4 |
3 |
|
|
|
9 |
|||||||||
Total |
|
121 |
1 |
844 |
359 |
34 |
25 |
75 |
144 |
34 |
139 |
30 |
1806 |
Table thirteen.
Comments
The college admissions examination is an instrument that has been refined over the years towards the sole purpose of determining admissions at the college. The examination is not intended as either a diagnostic or summative measure of the effectiveness of high schools or state departments of education.
The admissions test can indicate, at the extreme ends, whether a school is performing strongly on these admissions measures or performing in a manner indistinguishable from random. Comments above have been made where performances are extraordinarily weak.
Diversity
The differences in the performances by state have repercussions on the make-up of the regular admissions. Table fourteen compares the composition of the regular admissions to the state shares of the national population. The table reflects the state of the high school, not the cultural identity of the individual test takers.
State |
Associate |
RF |
Pop 2000 |
Pop RF |
Factor |
Chuuk |
75 |
18% |
53595 |
50% |
0.35 |
Kosrae |
55 |
13% |
7686 |
7% |
1.81 |
Pohnpei |
225 |
53% |
34486 |
32% |
1.65 |
Yap |
67 |
16% |
11241 |
11% |
1.51 |
|
422 |
100% |
107008 |
100% |
0 |
Table fourteen.
Thus in terms of the high school's state share of the national population, students from high schools in Kosrae are the most over-represented group in the regular admissions category with 1.81 times their share of the national population, while those from high schools in Chuuk are under-represented at a factor of about a third of their share of the national population. Each high school has students from every state, however the data is still likely roughly reflective of the underlying cultural diversity the college can expect among freshmen in the fall of 2006.
Addendum: Subgroups of Interest
The college admissions examination is neither a diagnostic nor comprehensive measure of learning. The college admissions examination is designed for a single purpose: admitting students to the appropriate programs for their abilities in English and mathematics. Despite the focus of the examination, many sponsored programs utilize the admissions results and statistics in their reports. Other programs in the high schools also choose to use the college admissions examination as one of their many indicators. The following are analyses of the essay and mathematics section with some of these subgroups broken out separately.
Writing (essay) section
The following table details the performance of the high schools in descending mean essay score order, repeating an earlier table but including some of the subgroups of interest. Explanation of the various subgroups is given after the table. The writing section had a total possible of 40.
School |
n |
stdev |
low |
high |
mean |
CCA |
10 |
1.57 |
37.58 |
39.82 |
38.7 |
Xavier |
35 |
3.66 |
35.6 |
38.11 |
36.86 |
KHS au |
12 |
2.93 |
34.8 |
38.53 |
36.67 |
KHS adv |
10 |
4.64 |
31.68 |
38.32 |
35 |
PSDA |
36 |
4.68 |
33.31 |
36.47 |
34.89 |
YSDA |
9 |
6.07 |
29.44 |
38.78 |
34.11 |
Saramen |
55 |
5.56 |
32.06 |
35.07 |
33.56 |
PICS ub |
22 |
5.03 |
30.95 |
35.41 |
33.18 |
KHS ub |
6 |
5.6 |
22.96 |
34.71 |
28.83 |
KHS |
141 |
8.09 |
27.13 |
29.83 |
28.48 |
KHS reg |
99 |
7.48 |
26.03 |
29.02 |
27.53 |
PICS tsp |
89 |
6.93 |
25.80 |
28.72 |
27.26 |
OIHS |
34 |
9.04 |
23.96 |
30.27 |
27.12 |
CSDA |
11 |
6.95 |
22.42 |
31.76 |
27.09 |
YHS |
111 |
11.19 |
24.75 |
28.96 |
26.86 |
OHWA |
9 |
7.69 |
20.2 |
32.02 |
26.11 |
PICS |
420 |
8.14 |
24.78 |
26.34 |
25.56 |
MHS tsp |
36 |
8.41 |
21.93 |
27.63 |
24.78 |
PICS reg |
309 |
8.29 |
23.6 |
25.46 |
24.53 |
Madol HS |
97 |
9.1 |
22.59 |
26.26 |
24.42 |
MHS reg |
61 |
9.54 |
21.77 |
26.65 |
24.21 |
KHS x |
14 |
10.97 |
17.09 |
29.76 |
23.43 |
YSC |
33 |
7.33 |
20.43 |
25.63 |
23.03 |
NICHS |
49 |
9.43 |
18.92 |
24.34 |
21.63 |
Mean |
1783 |
12.43 |
20.12 |
21.27 |
20.69 |
Mizpah |
22 |
9.2 |
16.56 |
24.72 |
20.64 |
Berea |
24 |
12.83 |
11.37 |
22.21 |
16.79 |
PSC |
84 |
10.53 |
14.49 |
19.06 |
16.77 |
Nukuno |
12 |
11.29 |
8.33 |
22.67 |
15.5 |
SNHS |
45 |
9.46 |
11.65 |
17.33 |
14.49 |
CSC |
79 |
10.06 |
10.66 |
15.16 |
12.91 |
WHS |
86 |
9.91 |
10.4 |
14.65 |
12.52 |
PLHA |
47 |
10.37 |
9.1 |
15.19 |
12.15 |
CHS |
270 |
8.95 |
6.26 |
8.4 |
7.33 |
Faichuk |
42 |
8.56 |
2.48 |
7.81 |
5.14 |
Table fifteen.
KHS au refers to the Kosrae Upward Bound students who were also in the KHS advanced class one.
KHS adv refers to students in the KHS advanced class one but who were not also in Upward Bound.
PICS ub refers to the Pohnpei Upward Bound students at PICS.
KHS ub refers to the KHS Upward Bound students who were not in the advanced class one.
KHS refers to all KHS students.
KHS reg refers to the "regular" students at KHS, that is, all of the non-Upward Bound, non-advanced class students.
PICS refers to all PICS students.
PICS reg refers to the "regular" students at PICS, that is, all of the non-Upward Bound students.
KHS x refers to test takers who took the test at KHS but who were not seniors this year in KHS. These test takers were, typically, older former students of KHS.
tsp refers to Talent Search Program students at PICS and Madolehnihmw High School (MHS).
A chart of some of the table above is provided on the next page.
Chart
eight.
The mean essay score is marked by the central ball. The extent of the 95% confidence interval for the mean is given by the vertical bars. Where the 95% confidence interval for one school or subgroup overlaps the ball of another school or subgroup, then the scores are not sufficiently separated to ensure statistical significance.
Of some interest is that a class of 22 students at KHS working in an academically challenging curriculum and referred to as the advanced class performed on par with the top private schools in the nation. Of note is that while twelve of these students benefitted from the Upward Bound program, ten others were not a part of Upward Bound at the time of the admissions examination. These ten outperformed students who were in the Kosrae Upward Bound program but not in the advanced class in Kosrae. The ten also outperformed the Pohnpei Upward Bound participants.
Mathematics Section
The following table details the performance of the high schools in descending mean mathematics score order, repeating an earlier table but including some of the subgroups of interest. The mathematics score is the sum of the four subsections, the total number correct out of forty.
School |
n |
stdev |
low |
high |
math mean |
PICS ub |
22 |
3.31 |
28.62 |
31.56 |
30.09 |
KHS adv-ub |
12 |
3.88 |
26.37 |
31.3 |
28.83 |
KHS ub |
6 |
2.94 |
25.24 |
31.42 |
28.33 |
Xavier |
35 |
4.31 |
24.06 |
27.02 |
25.54 |
KHS adv |
10 |
4.24 |
22.17 |
28.23 |
25.2 |
CCA |
10 |
4.54 |
20.55 |
27.05 |
23.8 |
PICS tsp |
90 |
5.53 |
21.75 |
24.07 |
22.91 |
Madol HS tsp |
36 |
2.71 |
20.3 |
22.14 |
21.22 |
PSDA |
37 |
5.56 |
19.23 |
22.94 |
21.08 |
PICS |
421 |
5.8 |
20.02 |
21.13 |
20.58 |
KHS |
141 |
6 |
19.06 |
21.06 |
20.06 |
PICS reg |
309 |
5.13 |
18.65 |
19.79 |
19.22 |
Madol HS reg |
62 |
4.06 |
18.07 |
20.13 |
19.10 |
Madol HS |
98 |
3.75 |
19.13 |
20.63 |
19.88 |
KHS reg |
99 |
4.89 |
17.53 |
19.48 |
18.51 |
KHS x |
14 |
4.85 |
13.56 |
19.15 |
16.36 |
YSDA |
9 |
3.31 |
13.68 |
18.77 |
16.22 |
Mean |
1806 |
6.24 |
15.93 |
16.5 |
16.21 |
YHS |
116 |
5.61 |
15.17 |
17.24 |
16.21 |
PSC |
84 |
5.01 |
14.92 |
17.1 |
16.01 |
YSC |
33 |
3.79 |
14.54 |
17.22 |
15.88 |
OHWA |
9 |
2.4 |
13.49 |
17.18 |
15.33 |
CSDA |
11 |
2.86 |
13.08 |
16.92 |
15 |
Saramen |
55 |
4.13 |
13.48 |
15.72 |
14.6 |
Mizpah |
22 |
2.42 |
13.25 |
15.39 |
14.32 |
NICHS |
50 |
4.05 |
13.15 |
15.45 |
14.3 |
OIHS |
34 |
3.36 |
13.12 |
15.47 |
14.29 |
PLHA |
47 |
4.64 |
12.91 |
15.64 |
14.28 |
Berea |
24 |
4.36 |
11.66 |
15.34 |
13.5 |
YMH |
7 |
4.5 |
8.26 |
16.59 |
12.43 |
CHS |
274 |
3.4 |
11.63 |
12.44 |
12.04 |
SNHS |
46 |
2.61 |
10.51 |
12.06 |
11.28 |
CSC |
79 |
3.24 |
10.39 |
11.84 |
11.11 |
Nukuno |
12 |
4.65 |
7.79 |
13.71 |
10.75 |
WHS |
89 |
3.19 |
9.71 |
11.05 |
10.38 |
Faichuk |
46 |
2.47 |
8.2 |
9.67 |
8.93 |
Table sixteen.
The subgroups above are explained in the previous section. Of note in the above table is that top three mean mathematics scores were attained by Upward Bound students. Both the Upward Bound students at PICS and the Upward Bound students in the advanced class at KHS were statistically significantly separated from the strongest performance by a school, which was Xavier. The Upward Bound students at KHS not in the advanced class were not statistically separated from Xavier, but in rank order they were higher.
Also of note is that the non-Upward Bound students in the Kosrae advanced class were statistically significantly stronger than the Pohnpei SDA students, and they performed on par with the Calvary Christian Academy students on Pohnpei. In mathematics and on the essay, the KHS advanced class performed competitively against the top private schools.
In mathematics, the Upward Bound students are generally as strong as or stronger than students benefitting from a private high school education.
A chart of the table above is provided on the next page.
Chart
nine.
Mathematics Placement Data
The following table shows the break-down of placement by the subgroups. Code "r" refers to a regular, that is, non-Upward Bound, non-advanced class student.
State |
School |
code |
90 |
95 |
98 |
100 |
101/150 |
Sum |
Chuuk |
(empty) |
r |
1 |
|
|
|
|
1 |
Berea |
r |
21 |
3 |
|
|
|
24 |
|
CHS |
r |
258 |
15 |
|
1 |
|
274 |
|
CSC |
r |
70 |
8 |
|
1 |
|
79 |
|
CSDA |
r |
9 |
2 |
|
|
|
11 |
|
Faichuk |
r |
46 |
|
|
|
|
46 |
|
Mizpah |
r |
21 |
1 |
|
|
|
22 |
|
Nukuno |
r |
12 |
|
|
|
|
12 |
|
PLHA |
r |
36 |
8 |
3 |
|
|
47 |
|
Saramen |
r |
43 |
10 |
|
2 |
|
55 |
|
SNHS |
r |
44 |
2 |
|
|
|
46 |
|
WHS |
r |
86 |
3 |
|
|
|
89 |
|
Xavier |
r |
3 |
6 |
1 |
24 |
1 |
35 |
|
Kosrae |
KHS |
adv |
1 |
3 |
|
5 |
1 |
10 |
|
au |
1 |
1 |
|
6 |
4 |
12 |
|
|
r |
39 |
45 |
1 |
11 |
3 |
99 |
|
|
ub |
1 |
|
|
5 |
|
6 |
|
|
x |
8 |
6 |
|
|
|
14 |
|
Pohnpei |
(empty) |
r |
1 |
1 |
|
|
|
2 |
? |
r |
1 |
|
|
|
|
1 |
|
CCA PNI |
r |
|
6 |
|
4 |
|
10 |
|
CCA PNI Tchrs |
r |
6 |
1 |
|
|
|
7 |
|
Madolehnihmw |
r |
20 |
68 |
1 |
9 |
|
98 |
|
OHWA |
r |
3 |
6 |
|
|
|
9 |
|
PICS |
r |
108 |
174 |
44 |
64 |
9 |
399 |
|
|
ub |
|
|
3 |
9 |
10 |
22 |
|
PNI campus |
r |
43 |
31 |
5 |
4 |
1 |
84 |
|
SDA PNI |
r |
15 |
5 |
4 |
13 |
|
37 |
|
Yap |
(empty) |
r |
2 |
|
|
|
|
2 |
DOE |
r |
1 |
|
|
|
|
1 |
|
NICHS |
r |
33 |
14 |
|
3 |
|
50 |
|
OIHS |
r |
23 |
11 |
|
|
|
34 |
|
YCS |
r |
3 |
|
|
|
|
3 |
|
YHS |
r |
51 |
57 |
5 |
2 |
1 |
116 |
|
YMH |
r |
6 |
1 |
|
|
|
7 |
|
YSC |
r |
19 |
12 |
1 |
1 |
|
33 |
|
YSDA |
r |
5 |
4 |
|
|
|
9 |
|
Sums |
|
|
1040 |
504 |
68 |
164 |
30 |
1806 |
Students who place into MS 100 or MS 101/101 are deemed college level students.