Validity and reliability of an instrument to evaluate the digital competence of teachers in relation to online tutorials in the stages of Early Childhood Education and Primary Education

Francisco D. Guillen-Gamez; Maria J. Mayorga-Fernández; Jose A. Contreras-Rosado

doi:10.6018/red.474981

Authors

Francisco D. Guillen-Gamez Universidad de Cordoba (UCO) https://orcid.org/0000-0001-6470-526X
Maria J. Mayorga-Fernández Universidad de Málaga (UMA) https://orcid.org/0000-0003-3749-1264
Jose A. Contreras-Rosado Universidad Antonio de Nebrija https://orcid.org/0000-0003-4625-7541

Keywords: digital competence, tutorial action, teachers, psychometrics, reliability, validity, factorial invariance, research methods

Abstract

This article demonstrates the validity and reliability of an instrument to evaluate the level of digital competence of Early Childhood Education and Primary Education teachers to carry out online tutorials using ICT resources as part of the educational process during the COVID-19 pandemic. The instrument consists of 35 items classified into five dimensions (A – functions of the tutor with the students; B – functions of the tutor with the teaching staff; C – functions of the tutor with the family; D – training in ICT and transfer; E – use of ICT resources). The instrument was applied to a sample of 1,098 active teachers from 14 autonomous communities located in Spain. Reliability was measured using Cronbach’s Alpha, the Spearman-Brown Coefficient, Guttman Two Halves, Omega McDonald, and for composite reliability. To check the validity of the instrument, we analysed the validity of understanding, and exploration of dimensionality using Factorial-Exploratory Analysis (EFA), and the instrument was adjusted for the different models through Confirmatory Factor Analysis (CFA). In addition, factorial invariance was evaluated based on the variables sex (male/female), type of centre (public/charter school), and type of teaching staff (Early Childhood Education/Primary Education), as well as external validity. The results of the reliability analyses were highly satisfactory and, in relation to the construct validity, the results found a good fit of the model, both in the internal validity and in the factorial invariance. The final version of the instrument consists of 25 items.

Downloads

Download data is not yet available.

References

Ait Hammou, Y., & Elfatihi, M. (2019). Moroccan teachers’ level of ICT integration in secondary EFL classrooms. International Journal of Language and Literary Studies, 1(3), 1-19.

Akpabio, E., & Ogiriki, I. B. (2017). Teachers use of information and communication technology (ICT) in teaching english language in senior secondary schools in Akwa Ibom state. Equatorial Journal of Education and Curriculum Studies, 2(2), 28-3.

Alexandrovich, Z. S., Pavlovna, M. N., Mikhailovich, S. K., & Viktorovich, L. V. (2020). Professional communication strategies in the design process of an open education tutor. PalArch's Journal of Archaeology of Egypt/Egyptology, 17(7), 70-81.

Barahona, J. D., García, J. M., & Pañego, M. M. (2019). Estudio de las actitudes y el interés de los docentes de primaria de educación física por las TIC en la Comunidad Valenciana. Retos: nuevas tendencias en educación física, deporte y recreación, (35), 267-272.

Bentler, P. M. (1989). EQS structural equations program manual. Los Angeles: BMDP Statistical Software.

Broadbent, J. (2017). Comparing online and blended learner's self-regulated learning strategies and academic performance. The Internet and Higher Education, 33, 24-32. https://doi.org/10.1016/j.iheduc.2017.01.004

Byrne, B. M. (2010). Structural equation modeling with AMOS: basic concepts, applications, and programming (multivariate applications series). New York: Taylor & Francis Group, 396, 7384.

Cabero-Almenara, J., & Palacios-Rodríguez, A. (2020). Marco Europeo de Competencia Digital Docente «DigCompEdu». Traducción y adaptación del cuestionario «DigCompEdu Check-In». Edmetic, 9(1), 213-234. https://doi.org/10.21071/edmetic.v9i1.12462

Cabero, J. & Barroso, J. (2016) ICT teacher training: a view of the TPACK model, Cultura y Educación, 28:3, 633-663, https://doi.org/10.1080/11356405.2016.1203526

Ceballos Lopez, N. (2017). The tutoring as democratic space. A view from student teacher's voice. Revista Espanola De Orientacion Y Psicopedagogia, 28(1), 86-103.

Chappell, S., Nunnery, J., Pribesh, S., & Hager, J. (2011). A meta-analysis of supplemental educational services (SES) provider effects on student achievement. Journal of Education for Students Placed at Risk, 16, 1–23.

Cheung, G.W. & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. https://doi.org/10.1207/S15328007SEM0902_5

Cudeck, R. & Browne, M.W. (1983). Cross-validation of covariance structures. Multivariate Behavioral Research, 18, 147–167. https://doi.org/10.1207/s15327906mbr1802_2

De Soroa, J. C. (2020). La orientación educativa al descubierto. Colectivo Orienta.

De Winter, J. C. F. & Dodou, D. (2012). Factor recovery by principal axis factoring and maximum likelihood factor analysis as a function of factor pattern and sample size. Journal of Applied Statistics, 39(4), 695-710. https://doi.org/10.1080/02664763.2011.610445

Elhadary, T., Elhaty, I. A., Mohamed, A. A., & Alawna, M. (2020). Evaluation of Academic Performance of Science and Social Science students in Turkish Universities during COVID-19 Crisis. Journal of Critical Reviews, 7(11), 1740-1751.

Erdogan, A., & Sahin, I. (2010). Relationship between math teacher candidates’ technological pedagogical and content knowledge (TPACK) and achievement levels. Procedia-Social and Behavioral Sciences, 2, 2707–2711. https://doi.org/10.1016/j.sbspro.2010.03.400

Espinoza Freire, E. E., & Ricaldi Echevarría, M. L. (2018). El tutor en los entornos virtuales de aprendizaje. Revista Universidad y Sociedad, 10(3), 201-210.

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C. & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299. https://psycnet.apa.org/doi/10.1037/1082-989X.4.3.272

Freiberg, H. A., Stover, J. B., De la Iglesia, G., & Fernández, M. (2013). Polychoric And Tetrachoric Correlations In Exploratory And Confirmatory Factorial Studies. Prensa Médica Latinoamericana-ISSN, 1688-4094.

García-martín, S. & Cantón-Mayo, I. (2019). Teachers 3.0: Patterns of Use of Five Digital Tools, Digital Education Review, 25 202-215. https://doi.org/10.1344/der.2019.35.202-215

Gordon, D. (2003). Better Teaching and Leaning in Digital Classroom. Cambridge: Harvard Education Press.

Guillén-Gámez, F. D., Mayorga-Fernández, M. J., & Ramos, M. (2020). Examining the Use Self-perceived by University Teachers about ICT Resources: Measurement and Comparative Analysis in a One-way ANOVA Design. Contemporary Educational Technology, 13(1), 1-13. https://doi.org/10.30935/cedtech/8707

Guillén-Gámez, F. D., Mayorga-Fernández, M. J., Bravo-Agapito, J., & Escribano-Ortiz, D. (2020). Analysis of teachers’ pedagogical digital competence: Identification of factors predicting their acquisition. Technology, Knowledge and Learning, 1-18. https://doi.org/10.1007/s10758-019-09432-7

Guillén-Gámez, F. D., Ruiz-Palmero, J., Sánchez-Rivas, E., & Colomo-Magaña, E. (2020). ICT resources for research: an ANOVA analysis on the digital research skills of higher education teachers comparing the areas of knowledge within each gender. Education and information technologies, 25(5), 4575-4589. https://doi.org/10.1007/s10639-020-10176-6

Guy, R. S., & Lownes-Jackson, M. (2012). Assessing effectiveness of web-based tutorials using pre-test and post-test measurements. Interdisciplinary Journal E-Learning & Learning Objects, 8, 15-38.

Hair, J., Black, W., Babin, B., & Anderson, R. (2010). Multivariate data analysis. Prentice-Hall: USA.

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the academy of marketing science, 43(1), 115-135. https://doi.org/10.1007/s11747-014-0403-8

Henson, R. K., y Roberts, J. K. (2006). Use of exploratory factor analysis in published research. Educational and Psychological Measurement, 66, 393-416. https://doi.org/10.1177%2F0013164405282485

Hidson, E. Pedagogía en colaboración: competencia digital de los profesores con recursos didácticos compartidos. Píxel-Bit. Revista de Medios y Educación, 61, 197-229. https://doi.org/10.12795/pixelbit.88108

Higgins, S., Xiao, Z., Katsipataki, M. (2012). The Impact of Digital Technology on Learning: A Summary for the Education Endowment Foundation (full report). UK: Durham University.

Hill, N. E., Witherspoon, D. P., & Bartz, D. (2018). Parental involvement in education during middle school: Perspectives of ethnically diverse parents, teachers, and students. The Journal of Educational Research, 111(1), 12-27. https://doi.org/10.1080/00220671.2016.1190910

Hinkin, T. R., Tracey, J. B., & Enz, C. A. (1997). Scale construction: Developing reliable and valid measurement instruments. Journal of Hospitality & Tourism Research, 21(1), 100-120. https://doi.org/10.1177%2F109634809702100108

Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure 17 analysis: Conventional criteria versus new alternatives. Structural Equation 18 Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

Huang, H. T. (2017). Private English Tutoring and Adolescents' Motivation to Learn English as a Foreign Language: A Self System Perspective. Taiwan Journal of TESOL, 14(1), 1-36.

Jang, S., & Tsai, M. (2013). Exploring the TPACK of Taiwanese secondary school science teachers using a new contextualized TPACK model. Australasian Journal of Educational Technology, 29, 566–580. https://doi.org/10.14742/ajet.282

Judyk, M.C. & Margalot, A.B. (2018). Trabajo colaborativo e interdisciplinario utilizando TIC. Contribuciones Científicas G/EA, 30, 95-103.

Koerwer, S. (2007). ThinkingStorm: The ABCs of online tutoring. Information Today, 24 (7), 2-2.

Laabidi, H. (2017). Exploring the Impact of the Factor of Gender on ICT Integration in Teaching. International Journal of English Language Teaching and Linguistics (IJELTAL), 1(2), 105-120.

Lamata, A. (2010). Plan de acción familiar. Familia y colegio, necesariamente unidos. Padres y Maestros/Journal of Parents and Teachers, (336), 27-31.

MacCallum, R.C., Roznowski, M., & Necowitz, L.B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111, 490–504.

Marsh, H. W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First-and higher order factor models and their invariance across groups. Psychological bulletin, 97(3), 562-582. https://psycnet.apa.org/doi/10.1037/0033-2909.97.3.562

Martin, F., Budhrani, K., & Wang, C. (2019). Examining Faculty Perception of Their Readiness to Teach Online. Online Learning, 23(3), 97-119. http://dx.doi.org/10.24059/olj.v23i3.1555

Meroño, L., Calderón Luquin, A., Arias Estero, J. L., & Méndez Giménez, A. (2018). Diseño y validación del cuestionario de percepción del profesorado de Educación Primaria sobre el aprendizaje del alumnado basado en competencias (# ICOMpri2). Revista Complutense de Educación, 29 (1), 215-235. http://dx.doi.org/10.5209/RCED.52200

Miretzky, D. (2004). The communication requirements of democratic schools: Parent-teacher perspectives on their relationships. Teachers College Record, 106(4), 814-851.

Moreno Olmedilla, J. M., Bolivar Botia, A., Clavijo Ruiz, M., Cortés de las Heras, J., Gómez Alfonso, J. A., Hernández Ortega, J., ... & Pérez Pueyo, A. (2020). Informe de investigación COVID19: Voces de docentes y familias. Proyecto Atlántida

Neely, A. R., Cotton, J., & Neely, A. D. (2017). E-mentoring: A model and review of the literature. AIS Transactions on Human-Computer Interaction, 9(3), 220-242.

O’Malley, P., Jenkins, S., Brooke, M., Donehower, C., Rabuck, D. & Lewis, M. (2013). Effectiveness of Using iPads to Build Math Fluency. Online Submission, Paper presented at the Council for Exceptional Children Annual Meeting. San Antonio, Texas, Apr 3-6.

Orellana, E. R., Hernández, I. C., Baz, B. O., & Martín-Domínguez, J. (2021). Evaluar competencias digitales en Educación Infantil desde las prácticas de aula. Píxel-Bit. Revista de Medios y Educación, 61, 37-69. https://doi.org/10.12795/pixelbit.85580

Oro, R. S. (2010). El profesor como tutor: un reto a consolidar en el ejercicio profesional de la orientación. REOP-Revista Española de Orientación y Psicopedagogía, 21(2), 346-357.

Ortega, I. (2007). El tutor virtual: aportaciones a los nuevos entornos de aprendizaje. Revista Electrónica de Teoría de la Educación, Vol. Extraordinario, 100-115.

Othman, R. (2020). Managing student and faculty expectations and the unexpected during the COVID-19 lockdown: role transformation. Accounting Research Journal, 1-12. https://doi.org/10.1108/ARJ-09-2020-0283

Padula, J. E. (2002). Contigo en la distancia. El Rol del tutor en la Educación No Presencial [http://www. uned. es/catedraunesco-ad/publicued/pbc08/rol_bened. htm].

Pardo, A. M. S., & Peñalvo, F. J. G. (2008). online tutoring and Mentoring. In Encyclopedia of networked and virtual organizations (pp. 1120-1127). IGI Global. Http://www.doi.org/10.4018/978-1-59904-885-7.ch147

Paudel, P. (2020). Online education: Benefits, challenges and strategies during and after COVID-19 in higher education. International Journal on Studies in Education, 3(2), 70-85. https://doi.org/10.46328/ijonse.32

Pérez, E. R., & Medrano, L. A. (2010). Análisis factorial exploratorio: bases conceptuales y metodológicas. Revista Argentina de Ciencias del Comportamiento (RACC), 2(1), 58-66.

Pettersson, F. (2018). On the issues of digital competence in educational contexts–a review of literature. Education and information technologies, 23(3), 1005-1021. https://doi.org/10.1007/s10639-017-9649-3

Portillo, J., Garay, U., Tejada, E. & Bilbao, N. (2020). Self-perception of Digital Competence of Educators during the COVID-19 pandemic: A cross-analysis of different educational stages. Sustainability, 12, 10128. 10.20944/preprints202011.0426.v1

Rahiem, M. D. (2020). Technological Barriers and Challenges in the Use of ICT during the COVID-19 Emergency Remote Learning. Universal Journal of Educational Research, 8(11B), 6124-6133. http://www.doi.org/10.13189/ujer.2020.082248

Ramírez-Montoya, M. S., Mena, J., & Rodríguez-Arroyo, J. A. (2017). In-service teachers’ self-perceptions of digital competence and OER use as determined by a xMOOC training course. Computers in Human Behavior, 77, 356-364.

Rodríguez López, M., & Llorent Vaquero, M. (2015). Equipos directivos y políticas TIC tras la supresión del Plan Escuela TIC 2.0. Opción. Revista de Ciencias Humanas y Sociales, 31 (1), 1157-1174.

Romero Martínez, S. J., Ordóñez Camacho, X. G., Guillén-Gamez, F. D., & Bravo Agapito, J. (2020). Attitudes toward Technology among Distance Education Students: Validation of an Explanatory Model. Online Learning, 24(2), 59-75. http://dx.doi.org/10.24059/olj.v24i2.2028

Roschelle, J., Feng, M. & Murphy, C. (2016). Online Mathematics Homework Increases Student Achievement. AERA Open, 2 (4) 1-12. https://doi.org/10.1177%2F2332858416673968

Rufete, L., Cascales, A. y Gomariz, M.A. (2020). Competencias digitales

de los docentes de Educación Infantil y Primaria desde la perspectiva de

la acción tutorial. http://hdl.handle.net/10201/91240

Shaffer, B. T., Cohen, M. S., Bigelow, D. C., & Ruckenstein, M. J. (2010). Validation of a disease‐specific quality‐of‐life instrument for acoustic neuroma: the Penn Acoustic Neuroma Quality‐of‐Life scale. The Laryngoscope, 120(8), 1646-1654. https://doi.org/10.1002/lary.20988

Shenderovich, Y., Thurston, A., & Miller, S. (2016). Cross-age tutoring in kindergarten and Elementary school settings: A systematic review and meta-analysis. International Journal of Educational Research, 76, 190-210.

Tanis, H., & Barker, I. (2017). E-Mentoring at a Distance: An Approach to Support Professional Development in Workplaces. Turkish Online Journal of Distance Education, 18(3), 135-155.

Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington: American Psychological Association

Usart Rodríguez, M.; Lázaro Cantabrana, J.L. & Gisbert Cervera, M. (2021). Validation of a tool for self-evaluating teacher digital competence. Educación XX1, 24(1), 353-373. http://doi.org/10.5944/educXX1.27080

Vallejo, A. P., Aguayo, B. B., & Ruiz, M. J. Y. C. (2020, April). ICT Impact in Orientation and University Tutoring According to Students Opinion. In World Conference on Information Systems and Technologies (pp. 245-252). Springer, Cham. https://doi.org/10.1007/978-3-030-45697-9_24

Vasquez, E., III, & Slocum, T. (2012). Evaluation of synchronous online tutoring for students at risk of reading failure. Exceptional Children, 78(2), 221–235.

Viberg, O., & Grönlund, Å. (2017). Understanding students’ learning practices: challenges for design and integration of mobile technology into distance education. Learning, Media and Technology, 42(3), 357-377. https://doi.org/10.1080/17439884.2016.1088869

Vuorikari, R., Velicu, A., Chaudron, S., Cachia, R., & Di Gioia, R. (2020). How families handled emergency remote schooling during the Covid-19 lockdown in spring 2020-Summary of key findings from families with children in 11 European countries (No. JRC122303). Joint Research Centre (Seville site).

Wolfe, J., Smith, J., Neumann, S., Miller, S., Schafer, E. C., Birath, A. L., ... & Spangler, C. (2020). Optimizing communication in schools and other settings during COVID-19. The Hearing Journal, 73(9), 40-42.

Yeboah, A. K., & Smith, P. (2016). Relationships between Minority Students Online Learning Experiences and Academic Performance. Online Learning, 20(4), 1-26. http://dx.doi.org/10.24059/olj.v20i4.577

Zia, A., Naz, I., & Qureshi, U. (2017). Role of Information and Communication Technologies in Knowledge Gap: A Comparative Study of Public and Private Schools in Lahore, Pakistan. Journal of Research & Reflections in Education (JRRE), 11(2), 124-140.