Portal de Periódicos da CAPES

Use of Item Models in a Large-Scale Admissions Test: A Case Study

2008; Taylor & Francis; Volume: 8; Issue: 3 Linguagem: Inglês

10.1080/15305050802262019

1532-7574

Sandip Sinharay, Matthew Johnson,

Educational Assessment and Pedagogy

Item models (LaDuca, Staples, Templeton, & Holzman, 1986 LaDuca, A., Staples, W. I., Templeton, B. and Holzman, G. B. 1986. Item modeling procedure for constructing content-equivalent multiple-choice questions. Medical education, 20: 53–56. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) are classes from which it is possible to generate items that are equivalent/isomorphic to other items from the same model (e.g., Bejar, 1996 Bejar, I. I. 1996. Generative response modeling: Leveraging the computer as a test delivery medium, Princeton, NJ: ETS. (ETS RR-96-13) [Google Scholar], 2002 Bejar, I. I. 2002. “Generative testing: From conception to implementation”. In Generating items from cognitive tests: Theory, Edited by: Irvine, S. H. and Kyllonen, P. C. 199–217. Mahwah, NJ: Lawrence Erlbaum Associates. [Google Scholar]). They have the potential to produce large numbers of high-quality items at reduced cost. This article introduces data from an application of item models for the generation of items for a large-scale assessment and investigates several research questions associated with the data. We begin by reviewing calibration techniques for the analysis of data involving item models; one method assumes that the items are isomorphic, while the other treats items generated from the same item model as distinct but related. A major question for these types of data is whether these items are isomorphic; that is, if they behave the same psychometrically. This article describes a number of rough diagnostic measures and a statistical diagnostic to assess the extent of isomorphicity in the items generated from an item model. Finally, this article discusses the issue of scoring—an area that needs more research—with data involving item models.