Recommender systems for the learning object development for engineering education: a systematic review
Abstract
This research presents a systematic review with the objective of identifying recommender systems designed to facilitate the creation of learning objects within higher education, with a particular focus on engineering education. The investigation delves into various facets including the recommended granularity level, filtering strategies, the utilization of artificial intelligence techniques, evaluation methodologies applied, pedagogical guidelines governing recommendation generation, and the integration of competency-based training considerations. To accomplish this, we extracted 409 initial references published between 2000 and 2023 from indexed journals. After a preliminary screening based on abstracts, 8 promising references were selected. After closer examination, only 3 of them proved relevant in addressing our research inquiries. Furthermore, an extended search on Google Scholar given as result 29 supplementary works that complemented the primary studies selected. Our analysis, guided by the research questions, revealed a dearth of recommender systems for learning object development. Despite the application of diverse artificial intelligence techniques, none of the identified systems recommend design at the medium level of granularity or take into account the competency-based approach. Furthermore, these systems are not helpful for educators without experience in instructional design. This finding reveals the necessity for further research that we commit to publish in RED once we finish it.
Downloads
References
Advanced Distributed Learning. (2004). https://adlnet.gov/projects/scorm/#scorm-versions-and-resources
Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IE Sharable Content Object Reference Model (SCORM).EE transactions on knowledge and data engineering, 17(6), 734-749. https://doi.org/10.1109/TKDE.2005.99
Bennett, S., Agostinho, S., & Lockyer, L. (2015). Technology tools to support learning design: Implications derived from an investigation of university teachers’ design practices. Computers & Education, 81, 211-220. https://doi.org/10.1016/j.compedu.2014.10.016
Bourkoukou, O. & El Bachari, E. (2016). E-learning personalization based on collaborative filtering and learner’s preference. Journal of Engineering Science and Technology, 11(11), 1565-1581. https://jestec.taylors.edu.my/Vol%2011%20issue%2011%20November%202016/11_11_5.pdf
Brereton, P., Kitchenham, B., Budgen, D., Turner, M., & Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain. The Journal of Systems and Software, 80(4), 571-583. https://doi.org/10.1016/j.jss.2006.07.009
Brik, M. & Touahria, M. (2020). Contextual Information Retrieval within Recommender System: Case Study “E-learning System”. TEM Journal, 9(3), 1150-1162. https://doi.org/10.18421/TEM93-41
Cabada, R. Z., Estrada, M. L. B., & García, C. A. R. (2011). EDUCA: A web 2.0 authoring tool for developing adaptive and intelligent tutoring systems using a Kohonen network. Expert Systems with Applications, 38(8), 9522-9529. https://doi.org/10.1016/j.eswa.2011.01.145
Callejas Cuervo, M., Hernández Niño, E. J. & Pinzón Villamil, J. N. (2011). Objetos de aprendizaje, un estado del arte. Entramado, 7(1), 176-189. https://revistas.unilibre.edu.co/index.php/entramado/article/view/3393/2785
Comisión Europea. (2009). El Marco Europeo de Cualificaciones para el aprendizaje permanente (EQF-MEC). Oficina de Publicaciones Oficiales de las Comunidades Europeas. https://www.oitcinterfor.org/sites/default/files/file_publicacion/marcoeuropeo_brochure.pdf
Chiappe Laverde, A., Segovia Cifuentes, Y. & Rincón Rodríguez, H.Y. (2007). Toward an instructional design model based on learning objects. Educational Technology Research and Development, 55(6), 671-681. https://doi.org/10.1007/s11423-007-9059-0
Cuadrado-Gallego, J. J. (2005). Adaptación de las métricas de reusabilidad de la ingeniería del software a los learning objects. Revista de Educación a Distancia, 1-8. https://revistas.um.es/red/article/view/24471
Eguigure, Y., & Prieto, M. (2009). Evaluación de la Calidad Pedagógica de OAs Utilizando Etiquetas Lingüísticas Difusas. En M. E. Prieto Méndez, S. Sánchez-Alonso, X. Ochoa & S. J. Pech Campos (Eds.), Recursos Digitales para el Aprendizaje (pp. 328-337). Universidad Autónoma de Yucatán.
Erdt, M., Fernández, A., & Rensing, C. (2015). Evaluating Recommender Systems for Technology Enhanced Learning: A Quantitative Survey. IEEE Transactions on learning technologies,8(4), 326-344. https://doi.org/ 10.1109/TLT.2015.2438867
Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in engineering education. Engineering Education, 78, 674-681. http://117.202.29.23:8080/jspui/bitstream/1/297/1/LEARNING%20AND%20TEACHING%20STYLES%20in%20Enginnering%20Education.pdf
Ferreira-Satler, M., Menéndez, V. H., Romero, F. P., Zapata, A., & Prieto, M. E. (3-5 de febrero de 2010a). Ontologías borrosas para representar perfiles de usuario en una herramienta de gestión de objetos de aprendizaje. XV Congreso Español sobre Tecnologías y Lógica Fuzzy (pp. 537-542). Huelva, España. https://doi.org/10.13140/2.1.2999.9360
Ferreira-Satler, M., Romero, F. P., Menéndez, V. H., Zapata, A., & Prieto, M. E. (18-23 de julio de 2010b). A fuzzy ontology approach to represent user profiles in e-learning environments. International Conference on Fuzzy Systems (pp. 1-8). Barcelona, España. https://doi.org/10.1109/FUZZY.2010.5584789
Ferreira-Satler, M., Romero, F. P., Menendez-Dominguez, V. H., Zapata, A., & Prieto, M. E. (2012). Fuzzy ontologies-based user profiles applied to enhance e-learning activities. Soft Computing, 16, 1129-1141. https://doi.org/10.1007/s00500-011-0788-y
Fleming, N., & Mills, C. (1992). Not Another Inventory, Rather a Catalyst for Reflection. To Improve the Academy, 11, 137-149. https://digitalcommons.unl.edu/podimproveacad/246/
Gagné, R. M. (1985). The Conditions of Learning and Theory of Instruction. Holt, Rinehart and Winston.
Gértrudix, M., Álvarez, S., Galisteo, A., Gálvez, M. d. C., & Gértrudix, F. (2007). Actions in the design and development of digital educational objects: institutional programmes. Universities and Knowledge Society Journal, 4(1), 14-25. https://dx.doi.org/10.7238/rusc.v4i1.296
Herlocker, J., Konstan, J. A., & Riedl, J. (2002). An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Information retrieval, 5, 287-310. https://doi.org/10.1023/A:1020443909834
Honey, P., & Mumford, A. (1982). The Manual of Learning Styles. Peter Honey.
Institute of Electrical and Electronics Engineers. (2002). Standard for Learning Object Metadata (1484.12.1-2002). https://doi.org/ 10.1109/IEEESTD.2002.94128
Iglesias, A., Martínez, P., Aler, R., & Fernández, F. (2009). Reinforcement learning of pedagogical policies in adaptive and intelligent educational systems. Knowledge-Based Systems, 22(4), 266-270. https://doi.org/10.1016/j.knosys.2009.01.007
Iglesias, A., Martínez, P., Aler, R., & Fernández, F. (2008). Learning teaching strategies in an Adaptive and Intelligent Educational System through Reinforcement Learning. Applied Intelligence, 31(1), 89-106. https://doi.org/10.1007/s10489-008-0115-1
Iglesias, A., Martínez, P., Aler, R., & Fernández, F. (8 de agosto de 2006). Learning pedagogical policies from few training data. 17th European Conference on Artificial Intelligence (pp. 1-6). Riva del Garda, Italia. http://hdl.handle.net/10016/17532
Iglesias, A., Martínez, P., Aler, R., & Fernández, F. (2-5 de octubre de 2004). Learning content sequencing in an educational environment according to student needs. 15th International Conference on Algorithmic Learning Theory (pp. 454-463). Padova, Italia. https://doi.org/10.1007/978-3-540-30215-5_34
Iglesias, A., Martínez, P., & Fernández, F. (3-7 de noviembre de 2003a). Navigating through the RLATES Interface: A Web-Based Adaptive and Intelligent Educational System. Workshops On The Move to Meaningful Internet Systems (pp. 175-184). Catania, Sicilia, Italia. https://doi.org/10.1007/978-3-540-39962-9_29
Iglesias Maqueda, A. M., Martínez Fernández, P., Aler, R., & Fernández Rebollo, F. (3-6 de diciembre de 2003b). Analysing the Advantages of Using Exploration and Exploitation Strategies in an Adaptive and Intelligent Educational System. 2nd Intemational Conference on Multimedia and Infonnation & Communication Technologies in Education (pp. 489-493). Badajoz, España. http://hdl.handle.net/10016/17547
Iglesias, A., Martínez, P., & Fernández, F. (2003c). An experience applying reinforcement learning in a web-based adaptive and intelligent educational system. Informatics in Education, 2(2), 223-240. https://doi.org/10.15388/infedu.2003.17
Iglesias, A. M., Martínez, P., Cuadra, M. D., Castro, E., & Fernández, F. (3-6 de abril de 2002). Learning to teach database design by trial and error. Fourth International Conference on Enterprise Information Systems (pp. 500-505). Ciudad Real, España. https://e-archivo.uc3m.es/bitstream/handle/10016/19832/learning_ICEIS_2002.pdf?sequence=1&isAllowed=y
Instructional Management System Global Learning Consortium. (2003). IMS Learning Design Version 1 Final Specification. https://www.imsglobal.org/learningdesign/index.html
Joy, J., & Pilai, R. (2022). Review and classification of content recommenders in E-learning environment. Journal of King Saud University - Computer and Information Sciences, 34(9), 7670-7685. https://doi.org/10.1016/j.jksuci.2021.06.009
Karga, S., & Satratzemi, M. (7-10 de julio de 2014). Mentor: A hybrid recommender system in order to support teachers in learning design authoring process. 14th International Conference on Advanced Learning Technologies (pp. 522-523). Atenas, Grecia. https://doi.org/10.1109/ICALT.2014.154
Karga, S., & Satratzemi, M. (2018). A hybrid recommender system integrated into LAMS for learning designers. Education and Information Technologies, 23(3), 1297-1329. https://doi.org/10.1007/s10639-017-9668-0
Karga, S., & Satratzemi, M. (2019a). Using explanations for recommender systems in learning design settings to enhance teachers’ acceptance and perceived experience. Education and Information Technologies, 24, 2953-2974. https://doi.org/10.1007/s10639-019-09909-z
Karga, S., & Satratzemi, M. (16-19 de setiembre de 2019b). Evaluating Teachers’ Perceptions of Learning Design Recommender Systems. 14th European Conference on Technology Enhanced Learning, Transforming Learning with Meaningful Technologies (pp. 98-111). Delft, Países Bajos. https://doi.org/10.1007/978-3-030-29736-7_8
Kitchenham, B., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. [Informe Técnico]. https://www.researchgate.net/profile/Barbara-Kitchenham/publication/302924724_Guidelines_for_performing_Systematic_Literature_Reviews_in_Software_Engineering/links/61712932766c4a211c03a6f7/Guidelines-for-performing-Systematic-Literature-Reviews-in-Software-Engineering.pdf
Kitchenham, B., Pretorius, R. D., Brereton, P., Turner, M., Niazi, M., & Linkman, S. (2010). Systematic literature reviews in software engineering – A tertiary study. Information and Software Technology, 52(8), 792-805. https://doi.org/10.1016/j.infsof.2010.03.006
Kolb, D.A. (1984). Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall.
Maphosa, V., & Maphosa, M. (2023). Fifteen Years of Recommender Systems Research in Higher Education: Current Trends and Future Direction. Applied Artificial Intelligence, 37(1), 406-425. https://doi.org/10.1080/08839514.2023.2175106
Menéndez, V., Castellanos M. E., Zapata, A., & Prieto, M. (2011). Generación de objetos de aprendizaje empleando un enfoque asistido. Pixel-Bit. Revista de Medios y Educación, (38), 141-153. https://idus.us.es/bitstream/handle/11441/22641/file_1.pdf?sequence=1&isAllowed=y
Menéndez, V. H., Prieto, M. E., & Zapata, A. (2010). Sistemas de Gestión Integral de Objetos de Aprendizaje. Revista Iberoamericana de Tecnologías del Aprendizaje, 5(2), 56-62.
Menéndez, V. H., & Prieto, M. E. (2009). Herramientas de Gestión para Objetos de Aprendizaje: Plataforma AGORA. En M. E. Prieto Méndez, S. Sánchez-Alonso, X. Ochoa & S. J. Pech Campos (Eds.), Recursos Digitales para el Aprendizaje (pp. 148-157). Universidad Autónoma de Yucatán.
Myers, I. B., & Myers, P. B. (1995). Gifts Differing: Understanding Personality Type. Davies-Black Publishing.
O’Neal, A. F. (2008). The current status of instructional design theories in relation to today’s authoring systems. British Journal of Educational Technology, 39(2), 251-267. https://doi.org/10.1111/j.1467-8535.2008.00815.x
Prieto, M. E., Zapata, A., & Menéndez, V. H. (2010). Data Mining Learning Objects. En C. Romero, S. Ventura, M. Pechenizky & R. Baker (Eds.), Handbook of Educational Data Mining (1a ed., pp. 481-492). https://doi.org/10.1201/b10274
Prieto, M. E., Menéndez, V. H., Segura, A. A., & Vidal, C. L. (24-26 de setiembre de 2008). A Recommender System Architecture for Instructional Engineering. World Summit on Knowledge Society: Emerging Technologies and Information Systems for the Knowledge Society (pp. 314-321). Atenas, Grecia. https://doi.org/10.1007/978-3-540-87781-3_35
Raj, N. S., & Renumol, V. G. (2022). A systematic literature review on adaptive content recommenders in personalized learning environments from 2015 to 2020. Journal of Computers in Education, 9(1), 113-148. https://doi.org/10.1007/s40692-021-00199-4
Recker, M. M., & Wiley, D. A. (2001). A Non-authoritative Educational Metadata Ontology for Filtering and Recommending Learning Objects. Interactive Learning Environments, 9(3), 255-271. https://doi.org/10.1076/ilee.9.3.255.3568
Reigeluth, C. M. (1999). What is Instructional Design Theory and How is it Changing? Lawrence Erlbaum Associates.
Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56-58. https://doi.org/10.1145/245108.245121
Romero, F. P., Ferreira-Satler, M., Olivas, J. A., Prieto-Mendez, M. E., & Menéndez-Dominguez, V. H. (22-24 de noviembre de 2011). A fuzzy-based recommender approach for learning objects management systems. 11th International Conference on Intelligent Systems Design and Applications (pp. 984-989). https://doi.org/10.1109/ISDA.2011.6121786
Sanz, C. V. (2015). Los objetos de aprendizaje, un debate abierto y necesario. Bit & Byte, 1(1), 33-35. http://sedici.unlp.edu.ar/bitstream/handle/10915/46729/Revista_completa.pdf?sequence=1&isAllowed=y
Segura, A., Vidal, C., Menéndez, V., Zapata, A., & Prieto, M. (1-2 de octubre de 2009). Exploring characterizations of learning object repositories using data mining techniques. Third International Conference on Metadata and Semantic Research (pp. 215-225). Milan, Italia. https://doi.org/10.1007/978-3-642-04590-5_20
Sicilia, M. Á. (2016). Reusabilidad y reutilización de objetos didácticos: mitos, realidades y posibilidades. Revista de Educación a Distancia, (50). https://doi.org/10.6018/red/50/6
Sicilia, M. A. (2006). Semantic learning designs: recording assumptions and guidelines. British Journal of Educational Technology, 37(3), 331-350. https://doi.org/10.1111/j.1467-8535.2006.00609.x
Suárez, J., Pérez, M., Chiappe, A., Cantillo, B., Molina, C., Salas Alvarez, D. J., Zapata, D., Múnera, E., Tibaná Herrera, G., Valencia, J. A., Navarro, M., Rozo, M., Duque, N., Medina, A., Enriquez, L., Duarte, A., Aleman Romero, A., Bedoya-Ortiz, D., Méndez Medina & J., Barrera, W. (2012). Recursos Educativos Digitales Abiertos - Colombia. Ministerio de Educación Nacional de Colombia. https://www.academia.edu/2068565/Recursos_Educativos_Digitales_Abiertos_Colombia
Tobón, S. (2008). Gestión curricular y ciclos propedéuticos. ECOE.
Vidal Castro, C., & Ferreira Satler, M. (2010). Modelo ontológico para la secuenciación de objetos de aprendizaje. Revista Iberoamericana de Tecnologías del Aprendizaje, 5(2), 63-67.
Vidal, C. L., Segura, A. A., Menéndez, V. H., & Prieto, M. E. (2009). Knowledge-based architecture for instructional engineering. International Journal of Knowledge and Learning, 5(3-4), 371-388. https://doi.org/10.1504/ijkl.2009.031230
Vidal, C., & Prieto, M. (2009). Una Ontología de apoyo a actividades de Diseño instruccional. En M. E. Prieto Méndez, S. Sánchez-Alonso, X. Ochoa & S. J. Pech Campos (Eds.), Recursos Digitales para el Aprendizaje (pp. 99-108). Universidad Autónoma de Yucatán.
Wiley, D. A. (2002). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. En D. A. Wiley (Ed.), The instructional use of learning objects (1a ed., pp. 3-23). AIT/AECT. https://members.aect.org/publications/InstructionalUseofLearningObjects.pdf
Alfredo Zapata, Víctor H. Menéndez, Manuel E. Prieto, Cristóbal Romero. (2015). Evaluation and Selection of Group Recommendation Strategies for Collaborative Searching of Learning Objects. International Journal of Human-Computer Studies, 76, 22-39. https://doi.org/10.1016/j.ijhcs.2014.12.002
Zapata, A., Menéndez, V.H., Prieto, M.E., & Romero, C. (2013). A framework for recommendation in learning object repositories: An example of application in civil engineering. Advances in Engineering Software, 56, 1-14. https://doi.org/10.1016/j.advengsoft.2012.10.005
Zapata, A., Menéndez, V. H., Prieto, M. E., & Romero, C. (13-15 de junio de 2012). Búsqueda colaborativa para el apoyo a la recomendación de Objetos de Aprendizaje. Ninth multidisciplinary symposium on the design and evaluation of digital content for education (pp. 57-67). Alicante, España. https://rua.ua.es/dspace/bitstream/10045/35602/1/spdece2012_06.pdf
Zapata, A., Menéndez, V. H., Prieto, M. E., & Romero, C. (2011). A Hybrid Recommender Method for Learning Objects. IJCA Proceedings on Design and Evaluation of Digital Content for Education (DEDCE), (1), 1-7. https://www.ijcaonline.org/dedce/number1/dece001.pdf
Zapata, A., Menéndez, V. H., & Prieto, M. E. (2009a). Aplicación de Minería de Datos para la extracción de reglas en Objetos de Aprendizaje. En M. E. Prieto Méndez, S. Sánchez-Alonso, X. Ochoa & S. J. Pech Campos (Eds.), Recursos Digitales para el Aprendizaje (pp. 107-117). Universidad Autónoma de Yucatán.
Zapata, A., Menéndez, V. H., & Prieto, M. E. (30 de noviembre – 2 de diciembre de 2009b). Discovering Learning Objects Usability Characteristics. Ninth International Conference on Intelligent Systems Design and Applications (pp. 1126-1130). Pisa, Italia. https://doi.org/10.1109/ISDA.2009.184
Zapata, A., Menéndez, V., Eguigure, Y., & Prieto, M. (16-18 de noviembre de 2009c). Quality evaluation model for learning objects from pedagogical perspective. A case of study. 2nd International Conference of Education, Research and Innovation (pp. 2228-2238). Madrid, España. https://library.iated.org/view/ZAPATA2009QUA
Zapata, M. (2005). Secuenciación de contenidos y objetos de aprendizaje. Revista de Educación a Distancia, Monográfico II. https://revistas.um.es/red/article/view/25221/24521
Copyright (c) 2023 Distance Education Journal
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Las obras que se publican en esta revista están sujetas a los siguientes términos:
1. El Servicio de Publicaciones de la Universidad de Murcia (la editorial) conserva los derechos patrimoniales (copyright) de las obras publicadas, y favorece y permite la reutilización de las mismas bajo la licencia de uso indicada en el punto 2.
2. Las obras se publican en la edición electrónica de la revista bajo una licencia Creative Commons Reconocimiento-NoComercial-SinObraDerivada 3.0 España (texto legal). Se pueden copiar, usar, difundir, transmitir y exponer públicamente, siempre que: i) se cite la autoría y la fuente original de su publicación (revista, editorial y URL de la obra); ii) no se usen para fines comerciales; iii) se mencione la existencia y especificaciones de esta licencia de uso.
3. Condiciones de auto-archivo. Se permite y se anima a los autores a difundir electrónicamente las versiones pre-print (versión antes de ser evaluada) y/o post-print (versión evaluada y aceptada para su publicación) de sus obras antes de su publicación, ya que favorece su circulación y difusión más temprana y con ello un posible aumento en su citación y alcance entre la comunidad académica. Color RoMEO: verde.
