Status : Verified
| Personal Name | Villas, Jonas P. |
|---|---|
| Resource Title | Evaluation of Angoff and Mapmark standard-setting procedures and considerations in setting cut scores of college admission test |
| Date Issued | January 2019 |
| Abstract | Using a combination of explanatory-sequential and transformative mixed method research design, this study evaluates the Angoff and Mapmark standard-setting procedures in setting cut scores of a college admission test. The elements of evaluation focused on internal, procedural and external validity, and modeled the facet effects by means of Many-Facet Rasch Model (MFRM) using 3282 students' data. The study also looked into the considerations of the 63 members of the standard-setting panel and the management committee in recommending and finalizing the cut score. The findings of the study shows that the Mapmark method has better inter-rater reliability than the Angoff method and the cut scores produced by both methods are statistically different with a decreasing trend, observed in rounds 1 to 3. The Mapmark method derived a final cut score that is lower than that of the Angoff method thus, facilitating a higher passing rate. Both the Angoff and Mapmark method are procedurally sound. The Angoff method has a better fit than the Mapmark method to the MFRM and the separation statistics revealed the effects of age, length of experience, and college affiliation facet to one's judgment in the standard-setting process. The findings of the study supported the objectivity of standard-setting process exemplified by the Angoff and Mapmark methods, exhibiting comparable validity. However, pragmatic policy considerations such as enrollment quota, internally generated income, and fiscal viability of the educational institution, influence the finalization of the cut score of the college admission test which leads to the notion of accommodation, and the revelation that cut score setting is a mediated process. |
| Degree Course | Doctor of Philosophy in Education (Research and Evaluation |
| Language | English |
| Keyword | Rasch model; mapmark methods; Angoff methods; entrance examinations; universities and colleges |
| Material Type | Thesis/Dissertation |
Preliminary Pages
2.38 Mb
Category : F - Regular work, i.e., it has no patentable invention or creation, the author does not wish for personal publication, there is no confidential information.
Access Permission : Open Access