Status : Verified
| Personal Name | Salubayba, Teodora M. |
|---|---|
| Resource Title | Determining differential item functioning in an achievement test using Mantel-Haenszel, item response theory, and logical data analysis |
| Date Issued | March 2013 |
| Abstract | Differential item functioning procedures, namely the Mantel-Haenszel (MH) and Item Response Theory (IRT) one-parameter logistic (1PL) model were investigated in the contexts of reading comprehension and math application among the Grade Six pupils in the six mixed-gender private schools and three all-girl schools run by religious congregation in Luzon and Visayas. MH and IRT-1PL are tests for critical evaluation of items; a means for determining the presence of DIF in the items. These two test DIF procedures having different assumptions were found both effective and sensitive in detecting DIF in the items. They were compared with regard to their effectiveness and sensitivity in detecting DIF in the test items to help test practitioners make informed decisions regarding the choice of DIF procedure, to continue improving the process of test evaluation, to ensure fair tests and assessments, and to review test development practices. In DIF procedures in the study, grouping variables like gender and school type were deemed to influence the performance of pupils in reading comprehension and math application. These two tests both measured ability and required the pupil’s understanding of the passages and inferential skills in reading comprehension; and the application of concepts learned and analysis in math application. IRT-1PL results showed large and consistent differences between the performance of the focal group and the reference group in reading comprehension and math application tests by gender and school-type compared to MH. (Focal group in gender-based DIF in reading comprehension comprised the boys, in math application — the girls; focal group in school type-based DIF in reading comprehension and in math application comprised the girls in mixed-gender schools) IRT-1PL detected more DIF items that were potentially biased against the focal group in all the analyses across comparison groups in the two tests. MH detected fewer DIF items in the two tests across comparison groups thus leaving the reading comprehension and math application tests with large number of “DIF-free” items. The logical data analysis of the responses of pupils during the focus group discussion and interviews with the psychometricians, counselors, and teachers involved with the pupils provided an emerging picture of the causes of gender-based and school type-based DIF in the test items. Information provided from the focus group discussions and interviews lent support to the causes of DIF in the items and helped validate the interpretations derived from the DIF analyses. DIF in favor of each gender showed agreement with the sex-role stereotypes in the literature; DIF in favor of each school type conformed to the findings of earlier studies about differences in the experiences of the girls in the all-girl schools and their counterparts in the mixed-gender schools. Findings showed that the advantage of a particular group on the reading comprehension was Iimited to specific subskills, literal recognition and inferential comprehension, while in math application on complex problem solving involving the application of two or more steps in basic operation including computations of fractions and percentages. Conclusions and implications were drawn for uses of DIF procedures in the improvement of achievement tests that entail less burden to the examinees yet reliable measures of performance of specific skills; for the test specialists and curriculum developers to strive to come up with unbiased test items using the more acceptable, feasible, and appropriate DIF procedures in evaluating test items. |
| Degree Course | Doctor of Philosophy in Education (Research and Evaluation) |
| Language | English |
| Keyword | educational tests and measurements, sex differences in education, reading comprehension, mathematical ability, sex differences, academic achievement, Item response theory |
| Material Type | Thesis/Dissertation |
Preliminary Pages
12.06 Mb