Machine Learning to improve the MOOC experience: the case of the Universitat Politècnica de València
Abstract
The aim of this paper is to design a proposal for automated mechanisms based on machine learning to improve the experience of participants in MOOC courses at the Universitat Politécnica de Valencia and reduce dropout rates. Following a desing based research DBR design, in which pedagogical decisions have always been prioritised over data analytics, three iterations have been carried out with different methodological patterns (systematic literature review, machine learning based on data from 260 courses and 700.000 students, and creation of automated mechanisms) that always end with the presentation of results and feedback from the university team. The main conclusions of this work indicate that, of the twenty-five pedagogical dropout indicators referred to by the literature reviews in iteration 1, only ten of them are validated with UPV courses (no automated or automatable data are available for the others), and of those finally only six of them are possible predictors of student dropout, with the data used. Finally, a set of automated mechanisms are proposed to be applied in the university's EdX platform to improve the user experience and reduce the dropout rate in the courses.
Downloads
References
Bartolomé-Pina, A., & Steffens, K. (2015) ¿Son los MOOC una alternativa de aprendizaje? Comunicar, 22(44), 91-99. http://dx.doi.org/10.3916/C44-2015-10
Bean, J., & Metzner, B. (1985). A Conceptual Model of Nontraditional Undergraduate Student Attrition. Review of Educational Research, 55(4), 485–540. https://doi.org/10.3102/00346543055004485
Billings, D. M., Connors, H. R., & Skiba, D. J. (2001). Benchmarking best practices in Webbased nursing courses. Advances in Nursing Science, 23, 41-52. https://doi.org/10.1097/00012272-200103000-00005
Bishop C.M. (2006) Pattern recognition and machine learning. Springer-Verlag.
Borrella, I., Caballero-Caballero, S., & Ponce-Cueto, E. (2019). Predict and Intervene: Addressing the Dropout Problem in a MOOC-based Program. In Proceedings of the Sixth (2019) ACM Conference on Learning@ Scale, 1-9. https://doi.org/10.1145/3330430.3333634
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE bulletin, 3(7). https://doi.org/10.1080/13562517.2013.827653
De Benito, B., & Salinas, J. (2016). La Investigación Basada en Diseño en Tecnología Educativa. Revista Interuniversitaria de Investigación en Tecnología Educativa, 0, 44-59. https://doi.org/10.6018/riite2016/260631
Escanés, G., Herrero, V., Merlino, A., & Ayllón, S. (2014). Deserción en educación a distancia: factores asociados a la elección de modalidad como desencadenantes del abandono universitario. Virtualidad, Educación Y Ciencia, 5(9), 45–55. https://bit.ly/31UD380
García-Tinizaray, D., Ordoñez-Briceño, K., & Torres-Diaz, J. C. (2015). Learning analytics para predecir la deserción de estudiantes a distancia. Campus virtuales, 3(1), 120-126. https://bit.ly/3wA06my
Halawa, S., Greene, D., & Mitchell, J. (2014). Dropout prediction in MOOC using learner activity features. Proceedings of the Second European MOOC Stakeholder Summit, Lausanne, Switzerland, 58-65.
Henderikx, M. A., Kreijns, K., & Kalz, M. (2017). Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Education, 38(3), 353-368. https://doi.org/10.1080/01587919.2017.1369006
Hmedna, B., El Mezouary, A., & Baz, O. (2020). A predictive model for the identification of learning styles in MOOC environments. Cluster Comput, 23, 1303–1328. https://doi.org/10.1007/s10586-019-02992-4
Jin, C. (2020). MOOC student dropout prediction model based on learning behavior features and parameter optimization. Interactive Learning Environments. https://doi.org/10.1080/10494820.2020.1802300
Kember, D. (1989). A longitudinal-process model of drop-out from distance education. The Journal of Higher Education, 60(3), 278–301. https://doi.org/10.1080/00221546.1989.11775036
Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & education, 104, 18-33. https://doi.org/10.1016/j.compedu.2016.10.001
Kloft, M., Stiehler, F., Zheng, Z., & Pinkwart, N. (2014). Predicting MOOC dropout over weeks using machine learning methods. Proceedings of the EMNLP 2014 Workshop on Analysis of Large Scale Social Interaction in MOOC, 60-65. https://doi.org/10.3115/v1/w14-4111
Lee, Y., & Choi, J. (2011). A Review of Online Course Dropout Research: Implications for Practice and Future Research. Educational Technology Research and Development, 59(5), 593–618. https://doi.org/10.1007/s11423-010-9177-y
León-Urrutia, M., Vázquez-Cano, E., & López-Meneses, E. (2017). MOOC learning analytics using real-time dynamic metrics. @tic. Revista d'innovació educativa, 18(3). https://doi.org/10.7203/attic.18.10022
Li, W., Gao, M., Li, H., Xiong, Q., Wen, J., & Wu, Z. (2016). Dropout prediction inMOOCusing behavior features and multi-view semi-supervised learning. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), 3130-3137. https://doi.org/10.1109/IJCNN.2016.7727598
Liang, J., Yang, J., Wu, Y., Li, C., & Zheng, L. (2016). Big data application in education: dropout prediction in edx MOOC. Proceedings of the IEEE Second International Conference on Multimedia Big Data (BigMM), 440-443, https://doi.org/10.1109/BigMM.2016.70
Martinez-Navarro, J. A. (2021). Indicadores de abandono en contextos MOOC, una aproximación pedagógica desde la literatura. Universitas Tarraconensis. Revista de Ciències de l’Educació, 3, 36-59. https://doi.org/10.17345/ute.2020.3.3031
Mi, F., & Yeung, D. Y. (2015). Temporal Models for Predicting Student Dropout in Massive Open Online Courses. 2015 IEEE International Conference on Data Mining Workshop. https://doi.org/10.1109/icdmw.2015.174
Millecamp, M., Gutiérrez, F., Charleer, S., Verbert, K., & De Laet, T. (2018). A qualitative evaluation of a learning dashboard to support advisor-student dialogues. Proceedings of the 8th International Conference on Learning Analytics and Knowledge, 56-60. https://doi.org/10.1145/3170358.3170417
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018) Foundations of machine learning. MIT press, 1 (1–3).
Ortega-Arranz, A., Bote-Lorenzo, M.L., Asensio-Pérez, J.I., Martínez-Monés, A., Gómez-Sánchez, E., & Dimitriadis, Y. (2019). To reward and beyond: Analyzing the effect of reward-based strategies in a MOOC, Computers & Education, 142, 103639, https://doi.org/10.1016/j.compedu.2019.103639.
Rolfe, V. (2015). A systematic review of the socio-ethical aspects of massive online open courses. European Journal of Open, Distance and e-Learning, 18(1), 52–72. https://bit.ly/31RC2xp
Rosé, C. P., Carlson, R., Yang, D., Wen, M., Resnick, L., Goldman, P., & Sherer, J. (2014). Social factors that contribute to attrition in MOOC. Proceedings of the first ACM conference on Learning@ scale conference, 197-198. https://doi.org/10.1145/2556325.2567879
Ruipérez-Valiente, J. A. (2020). El Proceso de Implementación de Analíticas de Aprendizaje. RIED. Revista Iberoamericana de Educación a Distancia, 23(2), 88-101 http://dx.doi.org/10.5944/ried.23.1.26283
Russo, C., Ramón, H., Alonso, N., Cicerchia, L., Esnaola, L., & Tessore, J.P. (2016). Tratamiento masivo de datos utilizando técnicas de machine learning. XVIII Workshop de Investigadores en Ciencias de la Computación WICC 2016. https://bit.ly/31Vub1S
Shah, D. (2016). Monetization over Massiveness: A Review of MOOC Stats and Trends in 2016. Class Central. https://bit.ly/2PzTxjl
Sinha, T., Jermann, P., Li, N., & Dillenbourg, P. (2014). Your click decides your fate: Inferring information processing and attrition behavior from mooc video clickstream interactions. Proceeding of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP), 3-14. https://bit.ly/3usQKqQ
Tinto, V. (1999). Taking retention seriously: Rethinking the first year of college. NACADA Journal, 19(2), 5–9. https://doi.org/10.12930/0271-9517-19.2.5
Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging deeper into learners' experiences in MOOC s: Participation in social networks outside of MOOC s, notetaking and contexts surrounding content consumption. British Journal of Educational Technology, 46(3), 570-587. https://doi.org/10.1111/bjet.12297
Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning, 17(2), 198–221. https://doi.org/10.19173/irrodl.v17i2.2448
Copyright (c) 2021 Jorge Angel Martinez Navarro, Ignacio Despujol Zabala
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Those authors who have publications with this journal accept the following terms:
a. The authors will retain their copyright and guarantee the journal the right of first publication of their work, which will be simultaneously subject to the Creative Commons License. Non-commercial attribution 4.0 International that allows to share, copy, and redistribute the material in any medium or format and adapt, remix, transform and build on the material in the following terms:
Recognition - You must give the appropriate credit, provide a link to the license, and indicate if changes have been made. You may do so in any reasonable manner, but not in a way that suggests that the licensor or its use endorses it. Non-commercial - You cannot use the material for commercial purposes. Share under it - If you remix, transform, or create on the material, your contributions must be distributed under the same license as the original.
b. Authors may adopt other non-exclusive licensing agreements for the distribution of the published work (e.g. deposit it in an institutional telematic file or publish it in a monographic volume) whenever the initial publication in this journal is indicated.
c. Authors are allowed and encouraged to distribute their work through the Internet (e.g. in institutional telematic archives or on their website) before and during the submission process, which can produce interesting exchanges and increase citations of the published work. (See The effect of open access).
d. In any case, the Editorial Team understands that the opinions expressed by the authors are their exclusive responsibility.
