Performance of an AI-Based Tool for Scoring the TTCT-Verbal in the Assessment of Creative Thinking
Supporting Agencies
- Ministerio de Ciencia, Innovación y Universidades del Gobierno de España
Abstract
Considering the specific challenges related to the assessment of the Torrance Tests of Creative Thinking – Verbal Form (TTCT-Verbal) concerning time demands, inter-rater variability, and subjectivity in the assessment of originality, this study preliminarily examines the performance of an artificial intelligence tool based on a large language model to support the scoring of fluency, flexibility, and originality, contrasting results with those of a panel of six human evaluators. A total of 47 protocols from postgraduate students at the Universitat de València were used, of which 30 were selected for the comparative analysis between the AI system and the human average. Similarly, the Intraclass Correlation Coefficient ICC (3,1), mean absolute error, root mean square error, Pearson correlations, and Bland-Altman plots were calculated. Results indicate close mean scores between both systems and low absolute error, observing the strongest association in originality, while fluency and flexibility showed lower relative consistency. Graphical analyses allowed the distribution of differences to be explored without identifying evident systematic bias, although these results should be interpreted descriptively. As such, it is concluded that the tool shows promising, but dimension-dependent performance, especially with regards to originality. Its use appears more appropriate as a complementary support within hybrid systems for assessing creative thinking than as a replacement for expert human judgment.
Downloads
-
Abstract0
-
PDF (Español (España))0
-
PDF0
References
Acar, S., Berthiaume, K., Grajzel, K., Dumas, D., Flemister, Ch., y Organisciak, P. (2023). Applying automated originality scoring to the verbal form of Torrance Tests of Creative Thinking. Gifted Child Quarterly, 67(1), 3–17. https://doi.org/10.1177/00169862211061874
Alcántara Santuario, A. (2023). La inteligencia artificial y sus implicaciones en educación. Perfiles Educativos, 45(Especial), 5–8. https://doi.org/10.22201/iisue.24486167e.2023.Especial.61687
Alfonso-Benlliure, V., Checa, I. y Meléndez, J. C. (2025). Long in the tooth for creativity? Differences in divergent thinking between young and older adults. Thinking Skills and Creativity, 57, 101847 https://doi.org/10.1016/j.tsc.2025.101847
Barragán-Giraldo, D. F., Pirela-Morillo, J. E., y Riaño Diaz, J. A. (2024). Datificación en contextos educativos. Entre subjetivación y ética. REICE. Revista Iberoamericana Sobre Calidad, Eficacia y Cambio en Educación, 22(2), 119-132. https://doi.org/10.15366/reice2024.22.2.007
Beaty, R. E., Johnson, D. R., Zeitlen, D. C., y Forthmann, B. (2022). Semantic distance and the alternate uses task: Recommendations for reliable automated assessment of originality. Creativity Research Journal, 34(3), 245–260. https://doi.org/10.1080/10400419.2022.2025720
Bland, J. M., y Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310. https://doi.org/10.1016/S0140-6736(86)90837-8
Bland, J. M., y Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical Methods in Medical Research, 8(2), 135–160. https://doi.org/10.1177/096228029900800204
Cesário, V., y Nisi, V. (2023). Designing mobile museum experiences for teenagers. Museum Management and Curatorship, 38(3), 272-292. https://doi.org/10.1080/09647775.2022.2111329
Cisternas San Martín, N., Guzman Muñoz, E., y Rivas Poblete, A. (2025). Inteligencia Artificial como asistente para desarrollar un currículum de educación contínua basado en competencias. Revista de Investigación Educativa, (43). https://doi.org/10.6018/rie.619441
Dumas, D., y Runco, M. (2018). Objectively Scoring Divergent Thinking Tests for Originality: A Re-Analysis and Extension. Creativity Research Journal, 30(4), 466–468. https://doi.org/10.1080/10400419.2018.1544601
Giavarina, D. (2015). Understanding Bland Altman analysis. Biochemia Medica, 25(2), 141–151. https://doi.org/10.11613/BM.2015.015
Goenechea, C., y Valero-Franco, C. (2024). Educación e Inteligencia Artificial: Un Análisis desde la Perspectiva de los Docentes en Formación. REICE. Revista Iberoamericana Sobre Calidad, Eficacia y Cambio en Educación, 22(2), 33-50. https://doi.org/10.15366/reice2024.22.2.002
Guilford, J. P. (1950). Creativity. American Psychologist, 5(9), 444–454. https://doi.org/10.1037/h0063487
Guilford, J. P. (1967). The nature of human intelligence. McGraw-Hill.
Guzik, E. E., Byrge, C., y Gilde, C. (2023). The originality of machines: AI takes the Torrance Test. Journal of Creative Behavior, 57(1), 45–58. https://doi.org/10.1002/jocb.546
Huerta, R. (2023). Artivismo y creatividad en la formación docente: cementerios, arte y literatura. Arte y Políticas de Identidad, 29, 65-86. https://doi.org/10.6018/reapi.598721
Huerta, R. (2025). Aprendizaje basado en proyectos (ABP). Visitas a museos y fotografía creativa en la formación del profesorado. Observar. Revista Electrònica De Didàctica De Les Arts, (19), 48–81. https://doi.org/10.1344/observar.2025.19.3
Huerta, R., y Alfonso-Benlliure, V. (2025). Museums as Catalysts for Creativity in Adolescence: A Review. Heritage, 8(8), 327. https://doi.org/10.3390/heritage8080327
Jarquín-Ramírez, M.-R., Alonso-Martínez, H., y Díez-Gutiérrez, E.-J. (2025). Chat GPT para un Sistema Educativo Justo, Democrático y Transformador. REICE. Revista Iberoamericana Sobre Calidad, Eficacia y Cambio en Educación, 23(3), 1-14. https://doi.org/10.15366/reice2025.23.3.001
Kim, K. H. (2006). Can we trust creativity tests? A review of the Torrance Tests of Creative Thinking (TTCT). Creativity Research Journal, 18(1), 3–14. https://doi.org/10.1207/s15326934crj1801_2
Kim, K. H. (2017). The Torrance Tests of Creative Thinking - Figural or Verbal: Which one should we use? Creativity. Theories – Research – Applications, 4(2), 302–321. https://doi.org/10.1515/ctra-2017-0015
Koo, T. K., y Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Mella-Mella, F. M., Calatayud Salom, M. A., y Lucas Calatayud, Á. J. (2026). Uso de la IA como Estrategia de y para el Aprendizaje en Educación Superior. REICE. Revista Iberoamericana Sobre Calidad, Eficacia y Cambio en Educación, 24(1). https://doi.org/10.15366/reice2026.24.1.007
Organisciak, P., Acar, S., Dumas, D., y Berthiaume, K. (2023). Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models. Thinking Skills and Creativity, 49, 101356. https://doi.org/10.1016/j.tsc.2023.101356
Perchtold-Stefan, C. M., Fink, A., Rominger, C., y Papousek, I. (2024). Social exclusion increases antisocial tendencies: Evidence from retaliatory ideation in a malevolent creativity task. Psychology of Aesthetics, Creativity, and the Arts, 18(6), 1014–1025. https://doi.org/10.1037/aca0000500
Reiter-Palmon, R., Forthmann, B., y Barbot, B. (2019). Scoring divergent thinking tests: A review and systematic framework. The Psychology of Aesthetics, Creativity, and the Arts, 13(2), 144-152. https://doi.org/10.1037/aca0000227
Runco, M. A. (2023). Updating the Standard Definition of Creativity to Account for the Artificial Creativity of AI. Creativity Research Journal, 37(1), 1 –5. https://doi.org/10.1080/10400419.2023.2257977
Runco, M. A., Millar, G., Acar, S., y Cramond, B. (2010). Torrance Tests of Creative Thinking as predictors of personal and public achievement: A fifty-year follow-up. Creativity Research Journal, 22(4), 361–368. https://doi.org/10.1080/10400419.2010.523393
Sanjurjo Pérez, P., Solana Domínguez, I., y Arana Cuenca, A. (2026). Un año sin móvil: Estudio de caso sobre políticas de prohibición y su influencia en el rendimiento académico. REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 24(1). https://doi.org/10.15366/reice2026.24.1.003
Sanz-Leal, M., y Orozco Gómez, M. L. (2025). Validación en español de una escala de Competencia Global en docentes en formación y en servicio. Revista de Investigación Educativa, (43). https://doi.org/10.6018/rie.564141
Silva Fuentealba, E. (2024). ChatGPT como catalizador del pensamiento creativo. [Chat GPT as a Catalyst for Creative Thinking]. European Public & Social Innovation Review, 9, 1-19. https://doi.org/10.31637/epsir-2024-410
Silva-Fuentealba, E., Valdés-León, G., y Oyarzún Yáñez, R. (2025).
Inteligencia artificial en el aula: potenciando la resolución de problemas a través del pensamiento creativo. Revista de Comunicación de la SEECI, 58, 1–19. https://doi.org/10.15198/seeci.2025.58.e927
Torrance, E. P. (1966). Torrance Tests of Creative Thinking: Norms-technical manual. Personnel Press.
Torrance, E. P. (1974). Torrance Tests of Creative Thinking: Norms-technical manual (Research edition). Personnel Press.
Torrance, E. P. (2008). Torrance Tests of Creative Thinking: Technical manual and scoring guide. Scholastic Testing Service.
Verdú-Pina, M., Serrano, V., Grimalt-Álvaro, C., y Usart, M. (2026). Perfiles docentes según la competencia digital autopercibida y el uso de la tecnología: Un análisis de clústeres. Revista de Investigación Educativa, (44). https://doi.org/10.6018/rie.672661
Welter, M. M., Jaarsveld, S., van Leeuwen, C., y Lachmann, T. (2016). Intelligence and Creativity: Over the Threshold Together? Creativity Research Journal, 28(2), 212–218. https://doi.org/10.1080/10400419.2016.1162564
Copyright (c) 2026 Revista de Investigación Educativa
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The articles and scientific documents published in RIE abide the following conditions:
1. The Servicio de Publicaciones de la Universidad de Murcia (the publisher) has the property rights (copyright) of all the documents published and allows the reuse under the user’s license indicated in point 2.
2. All documents are published in the digital edition of RIE under a Creative Commons Reconocimiento-NoComercial-SinObraDerivada 4.0 Internacional. (legal document) license. These documents can be copied, used, distributed, communicated and explained publicly if: i) the author(s) and its original source of publishing (magazine, publisher and URL of the document) are cited; ii) it is not used for commercial purpose; iii) the existence and the specifications about this license are mentioned.
3. Auto-archive’s conditions. The authors are allowed and encouraged to digitally distribute the pre-print versions (a version before evaluation) and/or post-print (a version that it is already evaluated and accepted to its publication). This promotes circulation and distribution earlier and can increase the citations and significance within the academic community.
