The utility of modified item response theory (IRT) models in small sample testing applications was studied. The modified IRT models were modifications of the one- and two-parameter logistic models. One-, two-, and three-parameter models were also studied. Test data were from 4 years of a national certification examination for persons desiring certification in personal financial planning. Sample sizes for the 4 years were 173, 149, 106, and 159, respectively. The stability of the item parameters over the 4 years was investigated, and the utility of the models was examined. All IRT analyses used the MULTILOG computer program. Item parameter stability was not exhibited for one-, two-, or three-parameter models or the modifications of the one- and two-parameter logistic models. Other results neither support nor reject the statement that IRT cannot be used with sample sizes smaller than 200 examinees. Five tables and four graphs are included. Two appendixes show MULTILOG input for two models. (SLD)
Authors
- Peer Reviewed
- F
- Publication Type
- ['Reports - Evaluative', 'Speeches/Meeting Papers']
- Published in
- United States of America
Table of Contents
- Stephen G. Sirecil 2
- The test data analyzed in this study were part of a national the examination over a four-year period. Because the requirements to sit 4
- The One- Two- and Three-Parameter IRT Models 5
- The three IRT models used in this study were the one- two- and Steinberg 1986 and so they are not described in detail here. The equations for he 1PL 2PL and 3PL respectively are presented below 5
- All IRT analyses reported here were conducted using the equal to the difference between the number of free parameters estimated in each model. If the additional parameters in the more general 6
- Procedure and Results 6
- Assessing dimensionality. To determine whether the test items matrix was computed for the 13-item data set based on the aggregated 6
- The one-factor model accounted for over 66 of the variance in the data 7
- 3PL models were fit to the aggregated data set to determine the most parameters were set at .25 which was the reciprocal of the number of 7
- 2PL analyses were conducted. These modified models added a fixed 7
- Table 2 8
- Parameters 8
- Sireci 1991. The model asymptote parameters. These models as 9
- Parameters 9
- IRT item parameters based on aggregated data. To investigate the 10
- To test whether this procedure would result item numbers are printed in samples Group 4 n159 and in the Appendix B. The commands 12
- 4 Data 13
- 1.5 15
- The differences observed between the standard errors and RMSE are among the estimates for the 1PL. Further analyses were planned to 16
- RMSE and Standard Errors of 1PL and MOD-MIX Models 16
- RMSE 16
- One avenue for future research may be to increase the number of 16
- References 18
- D.C. American Council on Education pp. 508600. Reprinted by 18
- Gorsuch R.L. 1983. Factor Analysis. Hillsdale N.J. Lawrence 18
- Measurement 11 263-277. 19
- Birnbaums three-parameter logistic model. Educational and 19
- Sireci S.G. 1991. Sample-independent item parameters An 19
- Stone C.A. Lane S. 1991. Use of restricted item response theory time. Applied Measurement in Education 4 125-141. 19
- Ed. Computerized adaptive testing a primer. Hillsdale N.J. 19
- Hillsdale N.J. Lawrence Erlbaum Associates pp. 161-186. 19
- Thissen D. 1991. MULTILOG Version 6.0 Users Guide. 19
- Thissen D. Steinbeg L. Gerrard M. 1986. Beyond group-mean 19
- N.J. Lawrence Eribaum Associates. 20
- Issues and 20
