Modelling figural matrix items and specification of guidelines for their construction
Supporting Agencies
- Subsidies from the University of Buenos Aires (UBACyT 2011-2014 Code Nb. 20020100100346)
- and MINCYT ANPCyT PICT Code Nb. 2011-0826 (2012-2015).
Abstract
This manuscript publicizes the results that concern the modelling of a Test of Figural Analogies (TFA) using Classical Test Theory (CTT) and Item Response Theory (IRT) on a joint basis, as well as the interpretation of the disadvantages and possible solutions of the test design. The theoretical background for the TFA construction is outlined, and studies accomplished after its administration to two samples is explained. One of these samples corresponds to a pilot study with Psychology students, while the other sample consisted of Art and Design college students. Main results indicate good unidimensionality and reliability, as well as acceptable discrimination (a), difficulty (b) and guessing value (c) parameters. Design disadvantages of three-rule based items are discussed, and new specifications for the 2x2 matrix item construction are given.
Downloads
References
Abad, F., Colom, R., Rebollo, I., & Escorial, S. (2004). Sex differential item functioning in the Raven’s Advanced Progressive Matrices: evidence for bi-as. Personality and Individual Differences, 36, pp. 1459–1470.
Arendasy, M. (2002). GeomGen—Ein Itemgenerator für Matrizentestaufgaben. Wien: Eigenverlag.
Arendasy, M. (2005). Automatic generation of Rasch-calibrated items: figural matrices test GEOM and Endless-Loops Test EC. International journal of tes-ting, 5(3), 197-224.
Bandeira Andriola, W. (2000). Funcionamento Diferencial dos Itens (DIF): Es-tudo com Analogias para Medir o Raciocínio Verbal. Psicologia: Reflexão e Crí-tica, 13(3), pp. 475-483.
Blum, G. D., Abal, F. J. P., Galibert, M. S. y Attorresi, H. F. (2011). Construc-ción de una Prueba de Analogias Figurales. Summa Psicológica UST, 18(1), 5-12.
Blum, G. D., Abal, F. J. P., Lozzia, G.S., Picón Janeiro, J. C. y Attorresi, H. F. (2011). Analogías de figuras: Teoría y construcción de ítemes. Interdisciplina-ria. Revista de psicología y ciencias afines, 28(1), 131-144.
Blum. G. D., Auné, S., Galibert, M. S. y Attorresi, H. F. (2013). Criterios para la eliminación de ítems de un Test de Analogías Figurales. Summa Psicológica UST, 10(2), 49-56.
Blum, G. D., Galibert, M. S., Abal, F. J. P., Lozzia, G. S. y Attorresi, H. F. (2011). Modelización de una Prueba de Analogías Figurales con la Teoría de Respuesta al Ítem. Escritos de Psicología, 4(3), 36-43.
Brown, L., Sherbenou, R. J. y Johnsen, S. K. (2000). TONI 2. Test de Inteligencia No Verbal. Apreciación de la habilidad cognitiva sin influencia del lenguaje. Manual. Madrid: TEA.
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Londres: Sage.
Cattell, R. B., & Cattell, A. K. S. (1973). Measuring Intelligence with The Culture Fair Tests. Champaign, IL: Institute for Personality and Ability Testing, Inc.
Cubillo, J. C. y González Labra, M. J. (1998). El razonamiento analógico como solución de problemas. En M. J. González Labra (Ed.), Introducción a la psico-logía del pensamiento (pp. 409-451). Madrid: Trotta.
De la Fuente Arnanz, J. y Minervino, R. A. (2004). Pensamiento analógico. En M. Carretero & M. Asensio (Coords.), Psicología del pensamiento (pp. 193-214). Madrid: Alianza.
Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for psychologists. Mah-wah: Lawrence Erlbaum Associates Inc.
Enzmann, D. (2005). Dirk Enzmann - Statistical Software (Some Useful Things). Re-cuperado de http://www2.jura.uni-ham-burg.de/instkrim/kriminologie/Mitarbeiter/Enzmann/Software/Enzmann_Software.html.
Freund, P. A., Hofer, S., & Holling, H. (2008). Explaining and controlling for the psychometric properties of computer-generated figural matrix items. Applied Psychological Measurement, 32(3), pp. 195-210.
García-Cueto, E. y Fidalgo, A. M. (2005). Análisis de los ítems. En J. Muñiz, A.M. Fidalgo, E. García-Cueto, R. Martínez, & R. Moreno (Eds.), Análisis de ítems (pp. 53-130). Madrid: La Muralla.
Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2), pp. 155-170.
Gierl, M.J., & Lai, H. (2012). The role of item models in automatic item genera-tion. International journal of testing, 12(3), 273-298. Doi: 10.1080/15305058.2011.635830.
Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive psy-chology, 12(3), pp. 306-355.
Gil Escudero, G. y Martínez Arias, M. R. (2001). Metodología de encuestas. En M. J. Navas (Ed.), Métodos, diseños y técnicas de investigación psicológica (pp. 379-436). Madrid: Universidad Nacional de Educación a Distancia.
Hanson, B. A., & Beguin, A. A. (2002). Obtaining a common scale for item re-sponse theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26(1), pp. 3-24.
Holyoak, K. J., & Koh, K. (1987). Surface and structural similarity in analogical transfer. Memory and cognition, 15(4), pp. 332-340.
Holyoak, K. J., & Thagard, P. R. (1989). A computational model of analogical problem solving. En S. Vosniadou & A. Ortony (Eds.), Similarity and analogi-cal reasoning (pp. 242-266). Cambridge University Press.
Irvine, S. (2002). The foundations of item generation for mass testing. En S.H. Irvine, & P.C. Kyllonen (Eds.), Item generation for test development (pp. 3-34). Mahwah: Lawrence Erlbaum Associates.
Kubinger, K. D. (2008). On the revival of the Rasch model-based LLTM: from constructing tests using item generating rules to measuring item administra-tion effects. Psychology science quarterly, 50(3), 311-327.
Lai, H., Alves, C., & Gierl, M. J. (2009). Using automatic item generation to ad-dress item demands for CAT. En D.J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing. Recuperado de: www.psych.umn.edu/psylabs/CATCentral.
Maller, S. J. (2000). Item invariance in four subtests of the Universal Nonverbal Intelligence Test (UNIT) across groups of deaf and hearing children. Journal of psychoeducational assessment, 18(3), pp. 240-254.
Mallows, C. L. (1973). Some Comments on CP. Technometrics, 15(4), 661–675. Doi:10.2307/1267380.
Martínez Arias, R. (1995). Psicometría: Teoría de los Tests Psicológicos y Educativos. Madrid: Síntesis.
Martínez, R., Moreno, R. y Muñiz, J. (2005). Construcción de los ítems. En J. Muñiz, A.M. Fidalgo, E. García-Cueto, R. Martínez, & R. Moreno (Eds.), Análisis de los ítems (pp. 9-48). Madrid: La Muralla.
Mulholland, T. M., Pellegrino, J. W., & Glaser, G. (1980). Components of geo-metric analogy solution. Cognitive Psychology, 12(2), pp. 252-284.
Raven, J., Raven, J. C., & Court, J. H. (1991). Manual for Raven’s Progressive Matrices and Vocabulary Scales. Sections 1, 2, 3 and 4. Oxford: Oxford Psychologists Press.
Sternberg, R. J. (1977). Intelligence, information processing and analogical reasoning: the componential analysis of human abilities. Hillsdale, NJ: Lawrence Erlbaum Asso-ciates.
Sternberg, R. J. (1987). Inteligencia humana II: Cognición, personalidad e inteligencia. Barcelona: Paidós.
Waller, N. G. (1995). MicroFact 1.1. A Microcomputer Factor Analysis Program for Ordered Polytomous Data and Mainframe Size Problems. St. Paul Minessota: As-sessment System Corporation.
Wechsler, D. (1997). Wechsler Adult Intelligence Scale-III. Nueva York: Psychologi-cal Corporation.
Whitely, S. E., & Schneider, L. M. (1981). Information structure for geometric analogies: A test theory approach. Applied Psychological Measurement, 5(3), pp. 383-397.
Wolf Nelson, N., & Gillespie, L. L. (1991). Analogies for thinking and talking. Words, pictures and figures. Tucson: Communication Skill Builders.
Yen, W. M. (1987). A comparison of the efficiency and accuracy of BILOG and LOGIST. Psychometrika, 52(2), pp. 275-291.
Zeuch, N. (2010). Rule-based item construction: analysis with and comparison of linear lo-gistic test models and cognitive diagnostic models with two item types (Tesis doctoral, Westfälische Wilhelms-Universität Münster, Germany). Recuperado de http://miami.uni-muenster.de/servlets/DerivateServlet/Derivate-6086/diss_zeuch.pdf.
Zimowski, M., Muraki, E., Mislevy, R., & Bock, R. (1996). BILOG-MGTM: Mul-tiple-group IRT analysis and test maintenance for binary items [Software]. Chicago, IL: Scientific Software International.
The works published in this journal are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) retains the property rights (copyright) of published works, and encourages and enables the reuse of the same under the license specified in paragraph 2.
© Servicio de Publicaciones, Universidad de Murcia, 2022
2. The works are published in the online edition of the journal under a Creative Commons Reconocimiento-CompartirIgual 4.0 (legal text). You can copy, use, distribute, transmit and publicly display, provided that: i) you cite the author and the original source of publication (journal, editorial and URL of the work), ii) are not used for commercial purposes, iii ) mentions the existence and specifications of this license.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
3. Conditions of self-archiving. Is allowed and encouraged the authors to disseminate electronically pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version reviewed and accepted for publication) of their works before publication, as it encourages its earliest circulation and diffusion and thus a possible increase in its citation and scope between the academic community. RoMEO Color: Green.
