The use of the Spanish PISA results on scientific publications
Abstract
In the last decade, interest in PISA results has grown, both in public debate and in the use of its data for research. The aim of this study is to analyse the use of the Spanish PISA results by the scientific community. A bibliometric analysis of 119 scientific articles included in the Scopus, Eric EBSCOHost and Web of Science databases was carried out between 2002 and 2019 using frequency analysis, MDS analysis and binary logistic regressions. Results reflect a growing interest in the Spanish PISA results from 2005 to the present within university research in the field of Education, to which most of the authors of our sample pool belong. With regard to the use of PISA information, the majority of studies use data for generating knowledge, with socio-economic issues such as the family and equity being the ones with the greatest presence alongside student performance. We conclude that, despite the fact that PISA has been slow to have an impact among the Spanish academic community, its results are increasingly being used as primary source of information for carrying out detailed studies on a wide range of subjects. However, PISA is used generically as an indicator of the quality of the education system, without authors going into detail into the contextualised analysis of performance for the competences under study.
Downloads
References
Archambault, E., & Gagné, E. V. (2004). The use of bibliometrics in the Social Sciences and Humanities. Science-Matrix.
Arroyo, D., Constante, I. A., & Asensio, I. (2019). La repetición de curso a debate: un estudio empírico a partir de PISA 2015. Educación XX1, 22(2), 69-92. https://doi.org/10.5944/educXX1.22479
Bárcena-Martín, E., Budría, S., & Moro-Egido, A.I. (2012). Skill mismatches and wages among European university graduates. Applied Economics Letters, 19(15), 1471-1475. https://doi.org/10.1080/13504851.2011.633886
Cabrera, L. J. (2013). Desigualdad social, rendimiento y logro educativos en España (1990-2012): Los desequilibrios regionales aumentan. Revista de Estudios Regionales, 98, 15-49.
Domínguez, M., Vieira, M. J., & Vidal, J. (2012). The impact of the Programme for International Student Assessment on academic journals. Assessment in Education: Principles, Policy & Practice, 19(4), 393- 409. https://doi.org/10.1080/0969594X.2012.659175
El Masri, Y. H., Baird, J.-A., & Graesser, A. (2016). Language effects in international testing: The case of PISA 2006 science items. Assessment in Education: Principles, Policy & Practice, 23(4), 427–455. https://doi.org/10.1080/0969594X.2016.1218323
Ferrer, F., & Massot, M. (2005). El proyecto PISA en los medios de comunicación escrita: de la simplificación a la manipulación. Organización y Gestión Educativa, 13(1), 19-22.
Fernández-Cano, A. (2016). Una crítica metodológica de las evaluaciones PISA. RELIEVE, 22(1), art. M15. https://doi.org/10.7203/relieve.22.1.8806
Fuentes, S., & Renobell, V. (2020). La influencia del género en el aprendizaje matemático en España. Evidencias desde PISA. Revista de Sociología de la Educación-RASE, 13(1), 63-80. https://doi.org/10.7203/RASE.13.1.16042.
Gil-Flores, J., Torres-Gordillo, J. J., & Perera-Rodríguez, V. H. (2012). The role of online reader experience in explaining students’ performance in digital reading. Computers & Education, 59, 653-660. https://doi.org/10.1016/j.compedu.2012.03.014
Gingras, Y. (2016). Bibliometrics and Research Evaluation: Uses and Abuses. MIT.
González de San Román, A., & De la Rica, S. (2016). Gender gaps in PISA test scores: the impact of social norms and the mother’s transmission of role attitudes. Estudios de Economía Aplicada, 34(1), 79-108.
González-Mayorga, H., Vidal, J., & Vieira, M.-J. (2017). El impacto del Informe PISA en la sociedad española: el caso de la prensa escrita. RELIEVE, 23(1), art 3. http://doi.org/10.7203/relieve.23.1.9015
Grek, S. (2009). Governing by the numbers: The PISA “effect” in Europe. Journal of Education Policy, 24(1), 23–37. https://doi.org/10.1080/02680930802412669
Hopfenbeck, T. N., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J., & Baird, J.-A. (2018). Lessons learned from PISA: A systematic review of peer-reviewed articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research, 62(3), 333-353. https://doi.org/10.1080/00313831.2016.1258726
Jovell, A. J. (1995). Análisis de regresión logística. Centro de Investigaciones Sociológicas.
Kankaraš, M., & Moors, G. (2014). Analysis of cross-cultural comparability of PISA 2009 scores. Journal of Cross-Cultural Psychology, 45(3), 381–399. https://doi.org/10.1177/0022022113511297
Kusumastuti, F. A., Rombot, O., & Ariesta, F. W. (2019). The effect of STEM integration on primary school students’ scientific literacy. International Journal of Scientific & Technology Research, 8(12), 1551-1553.
Leunda, I., Garmendia, I., & Etxeberria, J. (2017). Statictical Matching en la práctica - Una aplicación a la evaluación del Sistema educativo mediante PISA y TALIS. Revista de Investigación Educativa, 35(2), 371-388. http://dx.doi.org/10.6018/rie.35.2.262171
Lindblad, S., Pettersson, D., & Popkewitz, T. S. (2015). International comparisons of school results: a systematic review of research on large scale assessments in education. Swedish Research Council.
López, F., García, I., & Expósito, E. (2019). Rendimiento en ciencias, concepciones epistémicas y vocaciones STEM en las comunidades autónomas españolas. Evidencias desde PISA 2015, políticas y prácticas de mejora. Revista Española de Pedagogía, 77(272), 5-27. https://doi.org/10.22550/REP77-1-2019-09
López-Yáñez, J., & Sánchez-Moreno, M. (2013). Levers for sustainable improvement of Spanish schools in challenging contexts. Journal of Educational Change, 14, 203-232. https://doi.org/10.1007/s10833-012-9198-x
Meyer, H.-D., & Benavot, A. (2013). PISA and the Globalization of Education Governance: some puzles and problems. In H.-D. Meyer & A. Benavot (Eds.), PISA, Power, and Policy: the emergence of global education governance (9-26). Symposium Books.
Ministerio de Educación (2011). PISA – ERA 2009. Programa para la Evaluación Internacional de los Alumnos. Informe español. Secretaría de Estado de Educación y Formación Profesional.
Ministerio de Educación y Formación Profesional (2019). PISA 2018. Programa para la Evaluación Internacional de los Estudiantes. Informe Español. Secretaría General Técnica.
Monereo, C. (2009). PISA como excusa. Repensar la evaluación para cambiar la enseñanza. Graó.
Murillo, F. J., Martínez-Garrido, C., & Belavi, G. (2017). Segregación escolar por origen nacional en España. OBETS. Revista de Ciencias Sociales, 12(2), 395-423. https://doi.org/10.14198/OBETS2017.12.2.04
OECD (1999). Measuring students’ knowledge and skills. A New Framework for Assessment. OCDE Publications Service. https://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/33693997.pdf
Pérez, G., Anta, C., Badera, S., García, J., Pérez, M., & Sarrate, M. (2003). Análisis Bibliométrico en Educación. Incidencia en la calidad universitaria. Ministerio de Educación y Ciencia de España.
Pérez, E., Serrano, S., & Vico, S. (2011). Cómo mejorar la comprensión lectora a partir de estrategias de aprendizaje en alumnado de 6º de Primaria. Boletín de AELFA, 11(3), 68-72. https://doi.org/10.1016/s1137-8174(11)70048-6
Pérez, A. I., & Soto, E. (2011). Luces y sombras de PISA. Sentido educativo de las evaluaciones externas. Cultura y Educación, 23(2), 171-182. https://doi.org/10.1174/113564011795944758
Pérez-Esparrells, C., & Morales, S. (2012). El fracaso escolar en España: un análisis por Comunidades Autónomas. Revista de Estudios Regionales, 94, 39-69.
Ramírez-Gutiérrez, Z., Barrachina-Palanca, M., & Ripoll-Feliu, V. M. (2019). Divulgación de rankings universitarios y eficiencia en educación superior. Análisis bibliométrico y sistémico. Revista de Educación, 384, 255-297. https://doi.org/10.4438/1988-592X-RE-2019-384-412
Rutkowski, L., & Rutkowski, D. (2010). Getting it ‘better’: The importance of improving background questionnaires in international large-scale assessment. Journal of Curriculum Studies, 42(3), 411–30. http://doi.org/10.1080/00220272.2010.487546
Rutkowski, L., & Rutkowski, D. (2016). A call for a more measured approach to reporting and interpreting PISA results. Educational Researcher, 45(4), 252–257. https://doi.org/10.3102/0013189X16649961
Salinas, J., & Santín, D. (2012). Selección escolar y efectos de la inmigración sobre los resultados académicos españoles en PISA 2006. Revista de Educación, 358, 382-405. https://doi.org/10-4438/1988-592X-RE-2010-358-083
Salmerón, L., García, A., & Vidal-Abarca, E. (2018). WebLEC: a test to assess adolescents' Internet reading literacy skills. Psicothema, 30(4), 388-394. https://doi.org/10.7334/psicothema2017.395
Sánchez, S., & Pascual, M. Á. (2019). Análisis bibliométrico de la investigación educativa sobre desventaja sociocultural/socieducativa en el periodo 2015 a 2019. Enseñanza & Teaching, 37(2), 147-164.
Santos, M. Á., Godás, A., Ferraces, M. J., & Lorenzo, M. (2016). Academic performance of native and immigrant students: A study focused on the perception of family support and control, school satisfaction, and learning environment. Frontiers in Psychology, 7, 1-12. https://doi.org/10.3389/fpsyg.2016.01560
Sanz, R., Serrano, Á., & González, A. (2020). PISA: el precio pedagógico de una evaluación internacional. REICE. Revista Electrónica de Investigación Educativa, 22, e22, 1-13. https://doi.org/10.24320/redie.2020.22.e22.2673
Saura, G., & Luego, J. (2015). Política global más allá de lo nacional. Reforma educativa (LOMCE) y el régimen de estandarización (OCDE). Bordón, 67(1), 135-148. https://doi.org/10.13042/Bordon.2015.67109
Sellar, S., & Lingard, B. (2014). The OECD ad the expansion of PISA: New global modes of governance in education. British Educational Research Journal, 40(6), 917–936. https://doi.org/10.1002/berj.3120
Silva, L. C. & Barroso, I. M. (2004). Regresión Logística. La Muralla.
Spaull, N. (2019). Who makes it into PISA? Understanding the impact of PISA sample eligibility using Turkey as a case study (PISA 2003–PISA 2012). Assessment in Education: Principles, Policy & Practice, 26(4), 397–421. https://doi.org/10.1080/0969594X.2018.1504742
Springer (s.f.). Tipos de artículos de revistas. Springer. https://www.springer.com/la/authors-editors/tutoriales-de-autores-y-revisores/writing-a-journal-manuscript/types-of-journal-articles/12022874
Stoller, F. L., & Nguyen, L. T. H. (2020). Reading habits of Vietnamese University English majors. Journal of English for Academic Purposes, 48, 1-17. https://doi.org/10.1016/j.jeap.2020.100906
Torrecilla, E. M., Olmos, S., & Martínez, F. (2019). Explanatory factors as predictors of academic achievement in PISA tests. An analysis of the moderating effect of gender. International Journal of Educational Research, 96, 111-119. https://doi.org/10.1016/j.ijer.2019.06.002
Vázquez-Cano, E. (2017). Analysis of difficulties of Spanish teachers to improve students’ digital reading competence. A case study within the PISA framework. Pedagogika, 125(1), 175-194. https://doi.org/10.15823/p.2017.13
Ventura, D. (2017). La lectura en FLE por y para el aprendizaje. Cédille, 13, 479-500.
Yanniris, C., & Huang, Y. S. (2018). Bibliometric evidence point to loci of empirical knowledge production in environmental education. Cogent Education, 5(1), 1-14. https://doi.org/10.1080/2331186X.2018.1542961
Copyright (c) 2022 Journal of Educational Research
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The articles and scientific documents published in RIE abide the following conditions:
1. The Servicio de Publicaciones de la Universidad de Murcia (the publisher) has the property rights (copyright) of all the documents published and allows the reuse under the user’s license indicated in point 2.
2. All documents are published in the digital edition of RIE under a Creative Commons Reconocimiento-NoComercial-SinObraDerivada 3.0 España (legal document) license. These documents can be copied, used, distributed, communicated and explained publicly if: i) the author(s) and its original source of publishing (magazine, publisher and URL of the document) are cited; ii) it is not used for commercial purpose; iii) the existence and the specifications about this license are mentioned.
3. Auto-archive’s conditions. The authors are allowed and encouraged to digitally distribute the pre-print versions (a version before evaluation) and/or post-print (a version that it is already evaluated and accepted to its publication). This promotes circulation and distribution earlier and can increase the citations and significance within the academic community.
