The use of the Spanish PISA results on scientific publications

Authors

DOI: https://doi.org/10.6018/rie.451201
Keywords: PISA, information dissemination, norm-referenced evaluation, competences, bibliometric analysis

Abstract

In the last decade, interest in PISA results has grown, both in public debate and in the use of its data for research. The aim of this study is to analyse the use of the Spanish PISA results by the scientific community. A bibliometric analysis of 119 scientific articles included in the Scopus, Eric EBSCOHost and Web of Science databases was carried out between 2002 and 2019 using frequency analysis, MDS analysis and binary logistic regressions. Results reflect a growing interest in the Spanish PISA results from 2005 to the present within university research in the field of Education, to which most of the authors of our sample pool belong. With regard to the use of PISA information, the majority of studies use data for generating knowledge, with socio-economic issues such as the family and equity being the ones with the greatest presence alongside student performance. We conclude that, despite the fact that PISA has been slow to have an impact among the Spanish academic community, its results are increasingly being used as primary source of information for carrying out detailed studies on a wide range of subjects. However, PISA is used generically as an indicator of the quality of the education system, without authors going into detail into the contextualised analysis of performance for the competences under study.

Downloads

Download data is not yet available.

Author Biography

María José Vieira, Universidad de León

Profesora Titular de Universidad

Métodos de Investigación y Diagnóstico en Educación

Dpto. Psicología, Sociología y Filosofía

 

References

Archambault, E., & Gagné, E. V. (2004). The use of bibliometrics in the Social Sciences and Humanities. Science-Matrix.

Arroyo, D., Constante, I. A., & Asensio, I. (2019). La repetición de curso a debate: un estudio empírico a partir de PISA 2015. Educación XX1, 22(2), 69-92. https://doi.org/10.5944/educXX1.22479

Bárcena-Martín, E., Budría, S., & Moro-Egido, A.I. (2012). Skill mismatches and wages among European university graduates. Applied Economics Letters, 19(15), 1471-1475. https://doi.org/10.1080/13504851.2011.633886

Cabrera, L. J. (2013). Desigualdad social, rendimiento y logro educativos en España (1990-2012): Los desequilibrios regionales aumentan. Revista de Estudios Regionales, 98, 15-49.

Domínguez, M., Vieira, M. J., & Vidal, J. (2012). The impact of the Programme for International Student Assessment on academic journals. Assessment in Education: Principles, Policy & Practice, 19(4), 393- 409. https://doi.org/10.1080/0969594X.2012.659175

El Masri, Y. H., Baird, J.-A., & Graesser, A. (2016). Language effects in international testing: The case of PISA 2006 science items. Assessment in Education: Principles, Policy & Practice, 23(4), 427–455. https://doi.org/10.1080/0969594X.2016.1218323

Ferrer, F., & Massot, M. (2005). El proyecto PISA en los medios de comunicación escrita: de la simplificación a la manipulación. Organización y Gestión Educativa, 13(1), 19-22.

Fernández-Cano, A. (2016). Una crítica metodológica de las evaluaciones PISA. RELIEVE, 22(1), art. M15. https://doi.org/10.7203/relieve.22.1.8806

Fuentes, S., & Renobell, V. (2020). La influencia del género en el aprendizaje matemático en España. Evidencias desde PISA. Revista de Sociología de la Educación-RASE, 13(1), 63-80. https://doi.org/10.7203/RASE.13.1.16042.

Gil-Flores, J., Torres-Gordillo, J. J., & Perera-Rodríguez, V. H. (2012). The role of online reader experience in explaining students’ performance in digital reading. Computers & Education, 59, 653-660. https://doi.org/10.1016/j.compedu.2012.03.014

Gingras, Y. (2016). Bibliometrics and Research Evaluation: Uses and Abuses. MIT.

González de San Román, A., & De la Rica, S. (2016). Gender gaps in PISA test scores: the impact of social norms and the mother’s transmission of role attitudes. Estudios de Economía Aplicada, 34(1), 79-108.

González-Mayorga, H., Vidal, J., & Vieira, M.-J. (2017). El impacto del Informe PISA en la sociedad española: el caso de la prensa escrita. RELIEVE, 23(1), art 3. http://doi.org/10.7203/relieve.23.1.9015

Grek, S. (2009). Governing by the numbers: The PISA “effect” in Europe. Journal of Education Policy, 24(1), 23–37. https://doi.org/10.1080/02680930802412669

Hopfenbeck, T. N., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J., & Baird, J.-A. (2018). Lessons learned from PISA: A systematic review of peer-reviewed articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research, 62(3), 333-353. https://doi.org/10.1080/00313831.2016.1258726

Jovell, A. J. (1995). Análisis de regresión logística. Centro de Investigaciones Sociológicas.

Kankaraš, M., & Moors, G. (2014). Analysis of cross-cultural comparability of PISA 2009 scores. Journal of Cross-Cultural Psychology, 45(3), 381–399. https://doi.org/10.1177/0022022113511297

Kusumastuti, F. A., Rombot, O., & Ariesta, F. W. (2019). The effect of STEM integration on primary school students’ scientific literacy. International Journal of Scientific & Technology Research, 8(12), 1551-1553.

Leunda, I., Garmendia, I., & Etxeberria, J. (2017). Statictical Matching en la práctica - Una aplicación a la evaluación del Sistema educativo mediante PISA y TALIS. Revista de Investigación Educativa, 35(2), 371-388. http://dx.doi.org/10.6018/rie.35.2.262171

Lindblad, S., Pettersson, D., & Popkewitz, T. S. (2015). International comparisons of school results: a systematic review of research on large scale assessments in education. Swedish Research Council.

López, F., García, I., & Expósito, E. (2019). Rendimiento en ciencias, concepciones epistémicas y vocaciones STEM en las comunidades autónomas españolas. Evidencias desde PISA 2015, políticas y prácticas de mejora. Revista Española de Pedagogía, 77(272), 5-27. https://doi.org/10.22550/REP77-1-2019-09

López-Yáñez, J., & Sánchez-Moreno, M. (2013). Levers for sustainable improvement of Spanish schools in challenging contexts. Journal of Educational Change, 14, 203-232. https://doi.org/10.1007/s10833-012-9198-x

Meyer, H.-D., & Benavot, A. (2013). PISA and the Globalization of Education Governance: some puzles and problems. In H.-D. Meyer & A. Benavot (Eds.), PISA, Power, and Policy: the emergence of global education governance (9-26). Symposium Books.

Ministerio de Educación (2011). PISA – ERA 2009. Programa para la Evaluación Internacional de los Alumnos. Informe español. Secretaría de Estado de Educación y Formación Profesional.

Ministerio de Educación y Formación Profesional (2019). PISA 2018. Programa para la Evaluación Internacional de los Estudiantes. Informe Español. Secretaría General Técnica.

Monereo, C. (2009). PISA como excusa. Repensar la evaluación para cambiar la enseñanza. Graó.

Murillo, F. J., Martínez-Garrido, C., & Belavi, G. (2017). Segregación escolar por origen nacional en España. OBETS. Revista de Ciencias Sociales, 12(2), 395-423. https://doi.org/10.14198/OBETS2017.12.2.04

OECD (1999). Measuring students’ knowledge and skills. A New Framework for Assessment. OCDE Publications Service. https://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/33693997.pdf

Pérez, G., Anta, C., Badera, S., García, J., Pérez, M., & Sarrate, M. (2003). Análisis Bibliométrico en Educación. Incidencia en la calidad universitaria. Ministerio de Educación y Ciencia de España.

Pérez, E., Serrano, S., & Vico, S. (2011). Cómo mejorar la comprensión lectora a partir de estrategias de aprendizaje en alumnado de 6º de Primaria. Boletín de AELFA, 11(3), 68-72. https://doi.org/10.1016/s1137-8174(11)70048-6

Pérez, A. I., & Soto, E. (2011). Luces y sombras de PISA. Sentido educativo de las evaluaciones externas. Cultura y Educación, 23(2), 171-182. https://doi.org/10.1174/113564011795944758

Pérez-Esparrells, C., & Morales, S. (2012). El fracaso escolar en España: un análisis por Comunidades Autónomas. Revista de Estudios Regionales, 94, 39-69.

Ramírez-Gutiérrez, Z., Barrachina-Palanca, M., & Ripoll-Feliu, V. M. (2019). Divulgación de rankings universitarios y eficiencia en educación superior. Análisis bibliométrico y sistémico. Revista de Educación, 384, 255-297. https://doi.org/10.4438/1988-592X-RE-2019-384-412

Rutkowski, L., & Rutkowski, D. (2010). Getting it ‘better’: The importance of improving background questionnaires in international large-scale assessment. Journal of Curriculum Studies, 42(3), 411–30. http://doi.org/10.1080/00220272.2010.487546

Rutkowski, L., & Rutkowski, D. (2016). A call for a more measured approach to reporting and interpreting PISA results. Educational Researcher, 45(4), 252–257. https://doi.org/10.3102/0013189X16649961

Salinas, J., & Santín, D. (2012). Selección escolar y efectos de la inmigración sobre los resultados académicos españoles en PISA 2006. Revista de Educación, 358, 382-405. https://doi.org/10-4438/1988-592X-RE-2010-358-083

Salmerón, L., García, A., & Vidal-Abarca, E. (2018). WebLEC: a test to assess adolescents' Internet reading literacy skills. Psicothema, 30(4), 388-394. https://doi.org/10.7334/psicothema2017.395

Sánchez, S., & Pascual, M. Á. (2019). Análisis bibliométrico de la investigación educativa sobre desventaja sociocultural/socieducativa en el periodo 2015 a 2019. Enseñanza & Teaching, 37(2), 147-164.

Santos, M. Á., Godás, A., Ferraces, M. J., & Lorenzo, M. (2016). Academic performance of native and immigrant students: A study focused on the perception of family support and control, school satisfaction, and learning environment. Frontiers in Psychology, 7, 1-12. https://doi.org/10.3389/fpsyg.2016.01560

Sanz, R., Serrano, Á., & González, A. (2020). PISA: el precio pedagógico de una evaluación internacional. REICE. Revista Electrónica de Investigación Educativa, 22, e22, 1-13. https://doi.org/10.24320/redie.2020.22.e22.2673

Saura, G., & Luego, J. (2015). Política global más allá de lo nacional. Reforma educativa (LOMCE) y el régimen de estandarización (OCDE). Bordón, 67(1), 135-148. https://doi.org/10.13042/Bordon.2015.67109

Sellar, S., & Lingard, B. (2014). The OECD ad the expansion of PISA: New global modes of governance in education. British Educational Research Journal, 40(6), 917–936. https://doi.org/10.1002/berj.3120

Silva, L. C. & Barroso, I. M. (2004). Regresión Logística. La Muralla.

Spaull, N. (2019). Who makes it into PISA? Understanding the impact of PISA sample eligibility using Turkey as a case study (PISA 2003–PISA 2012). Assessment in Education: Principles, Policy & Practice, 26(4), 397–421. https://doi.org/10.1080/0969594X.2018.1504742

Springer (s.f.). Tipos de artículos de revistas. Springer. https://www.springer.com/la/authors-editors/tutoriales-de-autores-y-revisores/writing-a-journal-manuscript/types-of-journal-articles/12022874

Stoller, F. L., & Nguyen, L. T. H. (2020). Reading habits of Vietnamese University English majors. Journal of English for Academic Purposes, 48, 1-17. https://doi.org/10.1016/j.jeap.2020.100906

Torrecilla, E. M., Olmos, S., & Martínez, F. (2019). Explanatory factors as predictors of academic achievement in PISA tests. An analysis of the moderating effect of gender. International Journal of Educational Research, 96, 111-119. https://doi.org/10.1016/j.ijer.2019.06.002

Vázquez-Cano, E. (2017). Analysis of difficulties of Spanish teachers to improve students’ digital reading competence. A case study within the PISA framework. Pedagogika, 125(1), 175-194. https://doi.org/10.15823/p.2017.13

Ventura, D. (2017). La lectura en FLE por y para el aprendizaje. Cédille, 13, 479-500.

Yanniris, C., & Huang, Y. S. (2018). Bibliometric evidence point to loci of empirical knowledge production in environmental education. Cogent Education, 5(1), 1-14. https://doi.org/10.1080/2331186X.2018.1542961

Published
10-01-2022
How to Cite
González-Mayorga, H., Vieira, M. J., & Vidal, J. (2022). The use of the Spanish PISA results on scientific publications. Journal of Educational Research, 40(1), 183–202. https://doi.org/10.6018/rie.451201
Issue
Section
Articles