The systematic review in Educational Technology research: observations and advice
Supporting Agencies
- Ayuda RYC2019-028398-I financiada por MCIN/AEI/ 10.13039/501100011033 y FSE “El FSE invierte en tu futuro”.
Abstract
The use of systematic reviews started in Health Sciences as a way to synthesise and accumulate the results of different primary studies, as well as to identify research gaps that point to future lines of work. In recent years, systematic reviews in the field of educational research, and especially, in Educational Technology, seem to be on the rise. However, documentation of this methodology in Educational Technology, especially in a Spanish-speaking context, is scarce. This article therefore aims to provide guidance on initiating systematic reviews for those researching in Educational Technology. First, a clear conceptualisation of the terminology covering the umbrella term systematic review is provided and its phases are detailed. Furthermore, conceptual and digital frameworks and tools that can support such a process are presented. Using an example case in Educational Technology, the phases and tools of a systematic review are illustrated and detailed, as well as the challenges involved. Some considerations are made in relation to the methodology and advice is offered on how to deal with them. Finally, future lines of work are suggested for the development of the systematic review as a research methodology in Educational Technology.
Downloads
References
Bedenlier, S., Bond, M., Buntins, K., Zawacki-Richter, O., y Kerres, M. (2020). Learning by Doing? Reflections on Conducting a Systematic Review in the Field of Educational Technology. En O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, & K. Buntins (Eds.), Systematic Reviews in Educational Research: Methodology, Perspectives and Application (pp. 111-127). Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_7
Bond, M., Bedenlier, S., Marín, V. I., y Händel, M. (2021). Emergency remote teaching in higher education: Mapping the first global online semester. International Journal of Educational Technology in Higher Education, 18(1), 50. https://doi.org/10.1186/s41239-021-00282-x
Botella, J., y Zamora, Á. (2017). El meta-análisis: Una metodología para la investigación en educación. Educación XX1, 20(2). https://doi.org/10.5944/educxx1.19030
Buils, S., Esteve-Mon, F. M., Sánchez-Tarazaga, L., y Arroyo-Ainsa, P. (2022). Análisis de la perspectiva digital en los marcos de competencias docentes en Educación Superior en España. RIED-Revista Iberoamericana de Educación a Distancia, 25(2). https://doi.org/10.5944/ried.25.2.32349
Despujol, I., Castañeda, L., Marín, V. I., y Turró, C. (2022). What do we want to know about MOOCs? Results from a machine learning approach to a systematic literature mapping review. International Journal of Educational Technology in Higher Education, 19(1), 53. https://doi.org/10.1186/s41239-022-00359-1
Estarli, M., Aguilar Barrera, E. S., Martínez-Rodríguez, R., Baladia, E., Duran Agüero, S., Camacho, S., Buhring, K., Herrero-López, A., y Gil-González, D. M. (2016). Ítems de referencia para publicar Protocolos de Revisiones Sistemáticas y Metaanálisis: Declaración PRISMA-P 2015. Revista Española de Nutrición Humana y Dietética, 20(2), 148-160. https://doi.org/10.14306/renhyd.20.2.223
Gough, D. (2007). Weight of Evidence: A framework for the appraisal of the quality and relevance of evidence. Research Papers in Education, 22(2), 213-228. https://doi.org/10.1080/02671520701296189
Grant, M. J., y Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies: A typology of reviews, Maria J. Grant & Andrew Booth. Health Information & Libraries Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Guirao Goris, S. J. A. (2015). Utilidad y tipos de revisión de literatura. Ene, 9(2), 0-0. https://doi.org/10.4321/S1988-348X2015000200002
Gusenbauer, M., y Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods, 11(2), 181-217. https://doi.org/10.1002/jrsm.1378
Händel, M., Bedenlier, S., Marín, V. I., y Bond, M. (2020). Mapping the emerging field of research on “emergency remote teaching” in higher education due to COVID-19: Implications for education research and practice. https://osf.io/h5f8z
Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., y Reiss, K. M. (2020). The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education, 153, 103897. https://doi.org/10.1016/j.compedu.2020.103897
Kerres, M., Zawacki-Richter, O., S. Bedenlier, Bond, M., y Buntins, K. (2020). ActiveLearn Systematic Review Protocol. https://doi.org/10.13140/RG.2.2.33001.85600
Kimmons, R. (2020). Current Trends (and Missing Links) in Educational Technology Research and Practice. TechTrends, 64(6), 803-809. https://doi.org/10.1007/s11528-020-00549-6
Kimmons, R., y Rosenberg, J. M. (2022). Trends and Topics in Educational Technology, 2022 Edition. TechTrends, 66(2), 134-140. https://doi.org/10.1007/s11528-022-00713-0
Lizárraga Juárez, A., López Martínez, R. E., y López Ramírez, E. (2021). Evaluación de la calidad de la modalidad semipresencial en Educación Superior: Una revisión sistemática de la literatura. Revista Interuniversitaria de Investigación en Tecnología Educativa, 11, 131-149.
Manterola, C., Astudillo, P., Arias, E., y Claros, N. (2013). Revisiones sistemáticas de la literatura. Qué se debe saber acerca de ellas. Cirugía Española, 91(3), 149-155. https://doi.org/10.1016/j.ciresp.2011.07.009
Mertala, P., Moens, E., y Teräs, M. (2022). Highly cited educational technology journal articles: A descriptive and critical analysis. Learning, Media and Technology, 1-14. https://doi.org/10.1080/17439884.2022.2141253
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., y PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., y Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 143. https://doi.org/10.1186/s12874-018-0611-x
Newman, M., y Gough, D. (2020). Systematic Reviews in Educational Research: Methodology, Perspectives and Application. En O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, & K. Buntins (Eds.), Systematic Reviews in Educational Research (pp. 3-22). Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_1
Polanin, J. R., Maynard, B. R., y Dell, N. A. (2017). Overviews in Education Research: A Systematic Review and Analysis. Review of Educational Research, 87(1), 172-203. https://doi.org/10.3102/0034654316631117
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., Blunt, H., Brigham, T., Chang, S., Clark, J., Conway, A., Couban, R., de Kock, S., Farrah, K., Fehrmann, P., Foster, M., Fowler, S. A., Glanville, J., … PRISMA-S Group. (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z
Rodrigo Parra, J. (2021). Robótica para la inclusión educativa: Una revisión sistemática. Revista Interuniversitaria de Investigación en Tecnología Educativa, 150-171. https://doi.org/10.6018/riite.492211
Rubio-Aparicio, M., Sánchez-Meca, J., Marín-Martínez, F., y López-López, J. A. (2018). Guidelines for Reporting Systematic Reviews and Meta-analyses. Anales de Psicología, 34(2), 412. https://doi.org/10.6018/analesps.34.2.320131
Salinas, J., y Marín, V. I. (2019). Metasíntesis cualitativa sobre colaboración científica e identidad digital académica en redes sociales. RIED. Revista Iberoamericana de Educación a Distancia, 22(2), 97. https://doi.org/10.5944/ried.22.2.23238
Sánchez-Meca, J. (2010). Cómo realizar una revisión sistemática y un meta-análisis. Aula Abierta, 38(2), 53-64.
Urbina, S., Villatoro, S., y Salinas, J. (2021). Self-Regulated Learning and Technology-Enhanced Learning Environments in Higher Education: A Scoping Review. Sustainability, 13(13), 7281. https://doi.org/10.3390/su13137281
Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., y Buntins, K. (Eds.). (2020). Systematic Reviews in Educational Research: Methodology, Perspectives and Application. Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_1
Zawacki-Richter, O., Marín, V. I., Bond, M., y Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Copyright (c) 2022 Victoria I. Marín
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Those authors who have publications with this journal accept the following terms:
a. The authors will retain their copyright and guarantee the journal the right of first publication of their work, which will be simultaneously subject to the Creative Commons License. Non-commercial attribution 4.0 International that allows to share, copy, and redistribute the material in any medium or format and adapt, remix, transform and build on the material in the following terms:
Recognition - You must give the appropriate credit, provide a link to the license, and indicate if changes have been made. You may do so in any reasonable manner, but not in a way that suggests that the licensor or its use endorses it. Non-commercial - You cannot use the material for commercial purposes. Share under it - If you remix, transform, or create on the material, your contributions must be distributed under the same license as the original.
b. Authors may adopt other non-exclusive licensing agreements for the distribution of the published work (e.g. deposit it in an institutional telematic file or publish it in a monographic volume) whenever the initial publication in this journal is indicated.
c. Authors are allowed and encouraged to distribute their work through the Internet (e.g. in institutional telematic archives or on their website) before and during the submission process, which can produce interesting exchanges and increase citations of the published work. (See The effect of open access).
d. In any case, the Editorial Team understands that the opinions expressed by the authors are their exclusive responsibility.
