Evolution of the Contextual Factors Predicting Spanish Students’ Competence Levels
A Comparative Study between PISA 2015 and 2018
Supporting Agencies
- Ministerio de Ciencia e Innovación del Gobierno de España
Abstract
The Programme for International Student Assessment (PISA) has been assessing student competence levels for over 20 years, while also influencing the implementation of educational policies and practices based on its results at an international level. Although PISA’s configuration does not allow for longitudinal studies, this paper proposes the design of a trend study that enables the assessment of the evolution of the sociodemographic and educational context factors that best predict student competence levels. Through a multilevel regression analysis (hierarchical linear models) with the Spanish sample from PISA 2015 and 2018 waves, consisting of 65684 students and 1873 schools, the changes in variables predicting student performance in reading, math and science can be observed. The most noteworthy findings are the reduction of the impact of migration status for first generation immigrant students, the narrowing of the gender gap in STEM subjects (and its widening in reading), or the decrease of the contextual effect of the average socioeconomic level of students in a school. The paper concludes with the need to perform deeper analyses, both at statistical and educational policy levels, to produce more detailed results that shed light on which measures are more useful in order to reduce the impact of socioeconomic, demographic and educational context factors on Spanish students’ performance.
Downloads
References
Bisquerra, R. (2004). Metodología de la investigación educativa. La Muralla.
Carabaña, J. (2015). La inutilidad de PISA para las escuelas. Los Libros de la Catarata, D.L.
Creemers, B. P. M. y Kyriakides, L. (2008). The Dynamics of Educational Effectiveness: A contribution to policy, practice and theory in contemporary schools. Routledge. https://doi.org/10.4324/9780203939185
Cohen, L., Manion, L., y Morrison, K. (2017). Surveys, longitudinal, cross-sectional and trend studies. In Research methods in education (pp. 334-360). Routledge. https://doi.org/10.4324/9781315456539
de Miguel, M. (1985). Estrategias metodológicas en los estudios longitudinales. Revista de Investigación Educativa, 3(6), 252-270. Pearson.
Ding, H. y Homer, M. (2020). Interpreting mathematics performance in PISA: Taking account of reading performance. International Journal of Educational Research, 102, 101566. https://doi.org/10.1016/j.ijer.2020.101566
Doncel Abad, D. y Cabrera Álvarez, P. (2020). Comunidades Autónomas bilingües, identidades y desempeño educativo según PISA 2015. Revista de educación, 387, 163-188. https://doi.org/10.4438/1988-592X-RE-2020-387-443
Dumay, X. y Dupriez, V. (2014). Educational quasi-markets, school effectiveness and social inequalities. Journal of Education Policy, 29(4), 510-531. https://doi.org/10.1080/02680939.2013.850536
Gamazo, A. y Martínez-Abad, F. (2020). An Exploration of Factors Linked to Academic Performance in PISA 2018 Through Data Mining Techniques. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.575167
Gamazo, A., Martínez-Abad, F., Olmos-Migueláñez, S., y Rodríguez-Conde, M. J. (2018). Evaluación de factores relacionados con la eficacia escolar en PISA 2015. Un análisis multinivel. Revista de educación, 379, 56-84.
Gaviria Soto, J. L. y Castro Morera, M. (2005). Modelos jerárquicos lineales. La Muralla.
Gómez-Fernández, N. y Mediavilla, M. (2021). Exploring the relationship between Information and Communication Technologies (ICT) and academic performance: A multilevel analysis for Spain. Socio-Economic Planning Sciences, 77, 101009. https://doi.org/10.1016/j.seps.2021.101009
Han, S. W. (2018). School-based teacher hiring and achievement inequality: A comparative perspective. International Journal of Educational Development, 61, 82-91. https://doi.org/10.1016/j.ijedudev.2017.12.004
Hayes, A. F. (2006). A Primer on Multilevel Modeling. Human Communication Research, 32(4), 385-410. https://doi.org/10.1111/j.1468-2958.2006.00281.x
Hu, X., Gong, Y., Lai, C., y Leung, F. K. S. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers & Education, 125, 1-13. https://doi.org/10.1016/j.compedu.2018.05.021
Joaristi, L., Lizasoain Hernández, L., y Azpillaga, V. (2014). Detección y caracterización de los centros escolares de alta eficacia de la Comunidad Autónoma del País Vasco mediante Modelos Transversales Contextualizados y Modelos Jerárquicos Lineales. Estudios sobre educación, 27, 37-61.
Jornet Meliá, J. M. (2016). Análisis metodológico del proyecto PISA como evaluación internacional. RELIEVE. Revista Electrónica De Investigación y Evaluación Educativa, 22(1), 1-26. https://doi.org/10.7203/relieve22.1.8293
Kameshwara, K. K., Sandoval-Hernandez, A., Shields, R., y Dhanda, K. R. (2020). A false promise? Decentralization in education systems across the globe. International Journal of Educational Research, 104, 101669. https://doi.org/10.1016/j.ijer.2020.101669
Kyriakides, L., Creemers, B. P. M., y Antoniou, P. (2009). Teacher behaviour and student outcomes: Suggestions for research on teacher training and professional development. Teaching and Teacher Education, 25(1), 12-23. https://doi.org/10.1016/j.tate.2008.06.001
Lafontaine, D., Baye, A., Vieluf, S., y Monseur, C. (2015). Equity in opportunity-to-learn and achievement in reading: A secondary analysis of PISA 2009 data. Studies in Educational Evaluation, 47, 1-11. https://doi.org/10.1016/j.stueduc.2015.05.001
Laukaityte, I. y Rolfsman, E. (2020). Low, medium, and high-performing schools in the Nordic countries. Student performance at PISA Mathematics 2003-2012. Education Inquiry, 11(3), 276-295. https://doi.org/10.1080/20004508.2020.1721256
Lee, V. E. (2000). Using Hierarchical Linear Modeling to Study Social Contexts: The Case of School Effects. Educational Psychologist, 35(2), 125-141. https://doi.org/10.1207/S15326985EP3502_6
Lenkeit, J. (2012). How effective are educational systems? A value-added approach to study trends in PIRLS. Journal of Educational Research Online, 4(2),143-173. https://hdl.handle.net/11245/1.405026
Li, H. (2016). How is formative assessment related to students’ reading achievement? Findings from PISA 2009. Assessment in Education: Principles, Policy & Practice, 23(4), 473-494. https://doi.org/10.1080/0969594X.2016.1139543
Martínez-Abad, F. (2019). Identification of Factors Associated With School Effectiveness With Data Mining Techniques: Testing a New Approach. Frontiers in Psychology, 10, 2583. https://doi.org/10.3389/fpsyg.2019.02583
Martínez-Abad, F., Gamazo, A., y Rodriguez-Conde, M.-J. (2020). Educational Data Mining: Identification of factors associated with school effectiveness in PISA assessment. Studies in Educational Evaluation, 66, 100875. https://doi.org/10.1016/j.stueduc.2020.100875
Martínez-Abad, F., Lizasoain, L., Castro, M., y Joaristi, L. M. (2017). Selección de escuelas de alta y baja eficacia en Baja California (México). REDIE: Revista Electrónica de Investigación Educativa, 19(2), 38-53. https://doi.org/10.24320/redie.2017.19.2.960
Meunier, M. (2011). Immigration and student achievement: Evidence from Switzerland. Economics of Education Review, 30(1), 16-38.
Molina Portillo, E., Contreras García, J. M., Molina Muñoz, D., y Sánchez Pelegrín, J. A. (2022). Estudio por género del impacto de factores contextuales en el rendimiento matemático del alumnado español en PISA 2018. Revista Complutense de Educación, 33(4), 645-656. https://doi.org/10.5209/rced.76428
OECD. (2006). El programa PISA de la OCDE: Qué es y para qué sirve. http://iice.institutos.filo.uba.ar/el-programa-pisa-de-la-ocde-qu%C3%A9-es-y-para-qu%C3%A9-sirve
OECD. (2019). PISA 2018 Technical Report (p. 468).
Pomianowicz, K. (2021). Educational achievement disparities between second-generation and non-immigrant students: Do school characteristics account for tracking effects? European Educational Research Journal, 22(3), 297-324. https://doi.org/10.1177/14749041211039929
Raudenbush, S. W. y Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods. SAGE.
Rutkowski, D., Rutkowski, L., Wild, J., y Burroughs, N. (2018). Poverty and educational achievement in the US: A less-biased estimate using PISA 2012 data. Journal of Children and Poverty, 24(1), 47-67. https://doi.org/10.1080/10796126.2017.1401898
Scheerens, J., Luyten, H., van den Berg, S. M., y Glas, C. A. W. (2015). Exploration of direct and indirect associations of system-level policy-amenable variables with reading literacy performance. Educational Research and Evaluation, 21(1), 15-39. https://doi.org/10.1080/13803611.2015.1008520
Sirin, S. (2005). Socioeconomic Status and Academic Achievement: A Meta-Analytic Review of Research. Review of Educational Research, 75. https://doi.org/10.3102/00346543075003417
Snijders, T. A. B. y Bosker, R. J. (2011). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. SAGE.
Sullivan, A. y Calderwood, L. (2017). Surveys: longitudinal, cross-sectional, and trend studies. The BERA/SAGE Handbook of Educational Research: Two Volume Set (Vol.2, pp.395-415). SAGE Publications Ltd. https://doi.org/10.4135/9781473983953
Willms, J. D. (2010). School Composition and Contextual Effects on Student Outcomes. Teachers College Record, 112(4), 1008-1037.
Wiseman, A. W. (2013). Policy responses to PISA in comparative perspective. En H. D. Meyer y A. Benavot (Eds.), PISA, power, and policy: The emergence of global educational governance (pp. 303-322). Symposium Books.
Wu, H., Shen, J., Zhang, Y., y Zheng, Y. (2020). Examining the effect of principal leadership on student science achievement. International Journal of Science Education, 42(6), 1017-1039. https://doi.org/10.1080/09500693.2020.1747664
Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2), 114-128. https://doi.org/10.1016/j.stueduc.2005.05.005
Yetişir, M. y Bati, K. (2021). The Effect of School and Student-Related Factors on PISA 2015 Science Performances in Turkey. International Journal of Psychology and Educational Studies, 8, 170-186. https://doi.org/10.52380/ijpes.2021.8.2.433
Copyright (c) 2024 Journal of Educational Research
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The articles and scientific documents published in RIE abide the following conditions:
1. The Servicio de Publicaciones de la Universidad de Murcia (the publisher) has the property rights (copyright) of all the documents published and allows the reuse under the user’s license indicated in point 2.
2. All documents are published in the digital edition of RIE under a Creative Commons Reconocimiento-NoComercial-SinObraDerivada 3.0 España (legal document) license. These documents can be copied, used, distributed, communicated and explained publicly if: i) the author(s) and its original source of publishing (magazine, publisher and URL of the document) are cited; ii) it is not used for commercial purpose; iii) the existence and the specifications about this license are mentioned.
3. Auto-archive’s conditions. The authors are allowed and encouraged to digitally distribute the pre-print versions (a version before evaluation) and/or post-print (a version that it is already evaluated and accepted to its publication). This promotes circulation and distribution earlier and can increase the citations and significance within the academic community.
