The construction, reliability, and evidence of validity of a scale to measure new technologies by Spanish psychologist
Abstract
In this study we present the construction and analysis of a questionnaire for analyze the use of new technologies (NNTT) by psychology professionals, based on a sample of 1759 psychologists.). The final version composed by 36 items was generated from a bibliographic review on the new technologies applied to psychology. Exploratory factor analysis of selected items based on a randomized sample (n = 853) indicated the presence of five factors related to 1.- Online storage; 2.- Use of basic office and internet tools; 3.- Communication routes with the patient; 4.- Basic and advanced knowledge of programming and evaluation languages; 5.- Psychological interventions through NNTT. Validity of the 5-factor structure was supported by confirmatory factor analysis carried out with the remaining sample (n = 906). The results of reliability (internal consistency) and validity were adequate. Significant differences are found by sex, men obtain higher scores in both factor 1 and 3. The developed questionnaire is a valid and reliable instrument to measure the proposed construct.
Downloads
References
Ak, S., Koruklu, N., & Yilmaz, Y. (2013). A study on Turkish adolescent´s internet use: possible predictors of internet addiction. Cyberpsychology, Behavior, and Social Networking, 16, 205-209.
American Psychological Association (APA) (2013). Guidelines for the practice of telepsychology. American Psychologist, 68(9), 791–800. https://doi.org/10.1037/a0035001
Andersson, G., Carlbring, P., & Lindefors, N. (2016). History and Current Status of ICBT. Guided Internet-Based Treatments in Psychiatry, 1–16. https://doi.org/10.1007/978-3-319-06083-5_1
Banbury, A., Nancarrow, S., Dart, J., Gray, L., & Parkinson, L. (2018). Telehealth Interventions Delivering Home-based Support Group Videoconferencing: Systematic Review. Journal of Medical Internet Research, 20(2), e25. https://doi.org/10.2196/jmir.8090
Bandalos, D. L. , & Finney, S. J. (2018). Factor Analysis. The Reviewer’s Guide to Quantitative Methods in the Social Sciences, 98–122. https://doi.org/10.4324/9781315755649-8
Barak, A. (2007). Emotional support and suicide prevention through the Internet: A field project report. Computers in Human Behavior, 23(2), 971–984. https://doi.org/10.1016/j.chb.2005.08.001
Barak, A., Hen, L., Boniel-Nissim, M., & Shapira, N. (2008). A Comprehensive Review and a Meta-Analysis of the Effectiveness of Internet-Based Psychotherapeutic Interventions. Journal of Technology in Human Services, 26(2-4), 109–160. https://doi.org/10.1080/15228830802094429
Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Psychology, 3, 77-85.
Beidel, D. C., Frueh, B. C., Neer, S. M., & Lejuez, C. W. (2017). The efficacy of Trauma Management Therapy: A controlled pilot investigation of a three-week intensive outpatient program for combat-related PTSD. Journal of Anxiety Disorders, 50, 23–32. https://doi.org/10.1016/j.janxdis.2017.05.001
Bentler, P. M. (1977). Factor simplicity index and transformations. Psychometrika, 59, 567-579. https://doi.org/10.1007/bf02294054
Botella, C., Fernández-Álvarez, J., Guillén, V., García-Palacios, A., & Baños, R. (2017). Avances recientes en la terapia de exposición a la realidad virtual para las fobias: una revisión sistemática. Current Psychiatry Reports. 19:42. https://doi.org/10.1007/s11920-017-0788-4
Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, É., Forget, H., Loranger, C., & Roucaut, F. X. (2017). Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A three-arm randomized controlled trial. British Journal of Psychiatry, 210(4), 276–283. https://doi.org/10.1192/bjp.bp.116.184234
Brown, S. A. (2005). Measuring perceived benefits and perceived barriers for physical activity. American Journal of Health Behavior, 29(2), 107–116. https://doi.org/10.5993/AJHB.29.2.2
Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Routledge.
Calvo, M. E. (2019). Científicas e inventoras a través de los cuentos. IQUAL, 2, 147-170. https://doi.org/10.6018/iqual.340701
Chen, F. (2007). Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling, 14, 464-504. https://doi.org/10.1080/10705510701301834.
Cohen, J. (1988). Statistical Power Analysis for the Behavioural Science (2nd Edition). In Statistical Power Anaylsis for the Behavioral Sciences. Laurence Erlbaum Associates.
Dueber, D. M. (2017). Bifactor Indices Calculator: A Microsoft Excel-based tool to calculate various indices relevant to bifactor CFA models. https://doi.org/10.13023/edp.tool.01 [Also available at http://sites.education.uky.edu/apslab/resources/]
Dunn, T. J., Baguley, T., & Brunsden, V. (2013). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 295-441. https://doi.org/10.1111/bjop.12046
Encinas, F. J. L., & González, S. M. V. (2010). Menores y nuevas tecnologías: conductas indicadoras de posible problema de adicción. Psicothema, 22(2), 180-188
Espinoza, M., Baños, R. M., García-Palacios, A., & Botella, C. (2013). La realidad virtual en las intervenciones psicológicas con pacientes oncológicos. Psicooncología, 10(2-3). https://doi.org/10.5209/rev_psic.2013.v10.n2-3.43447
Fernández, C. S., & Alcalde, J. B. F. (2015). No, sin mi móvil. Diferencias de género y uso de las nuevas tecnologías. ICONO 14, Revista de comunicación y tecnologías emergentes, 13(1), 208-246.
Fernández-Luque, L., & Bau, T. (2015). Health and Social Media: Perfect Storm of Information. Healthcare Informatics Research, 21(2), 67. https://doi.org/10.4258/hir.2015.21.2.67
Ferrando, P. J. & Lorenzo-Seva, U. (2014). El análisis factorial exploratorio de los ítems: algunas consideraciones adicionales. Anales de Psicología, 30(3), 1170-1175. https://doi.org/10.6018/analesps.30.3.199991
Ferrando, P. J. & Lorenzo-Seva, U. (2017). Program FACTOR at 10: Origins, development and future directions. Psicothema, 29(2), 236-240. https://doi.org/10.7334/psicothema2016.304
Finch, W. H. (2020). Using Fit Statistic Differences to Determine the Optimal Number of Factors to Retain in an Exploratory Factor Analysis. Educational and Psychological Measurement, 80(2), 217–241. https://doi.org/10.1177/0013164419865769
Fornell, C. & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of marketing research, 39-50.
Gackenbach, J. (2007). Introduction. Psychology and the Internet, 1–10. https://doi.org/10.1016/b978-012369425-6/50019-9
Gartner Symposium ITxpo (2013). https://www.gartner.com/en
Gassova, Z., & Werner-Wilson, R. J. (2018). Characteristics of e-therapy websites involving marriage and family therapists. American Journal of Family Therapy, 46(1), 27–43. https://doi.org/10.1080/01926187.2018.1428127
Godoe, P., & Johansen, T. S. (2012). Understanding adoption of new technologies: Technology readiness and technology acceptance as an integrated concept. PsycEXTRA Dataset. https://doi.org/10.1037/e537242013-005
Gorsuch, R. L. (2003). Factor analysis. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Research methods in psychology, Vol. 2 (pp. 143–164). John Wiley & Sons Inc.
Green, D. (2006). Ground rules in online psychotherapy (Doctoral dissertation). University London.
Hancock, G. R., & Mueller, R. O. (2001). Rethinking Construct Reliability within Latent Variable Systems. In R. Cudeck, S. du Toit, & D. S?rbom (Eds.), Structural Equation Modeling: Present und Future—A Festschrift in Honor of Karl Joreskog (pp. 195-216). Scientific Software International.
Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic journal of business research methods, 6(1), 53-60. https://doi.org/10.21427/d7cf7r
Howard, M. C. (2015). A Review of Exploratory Factor Analysis Decisions and Overview of Current Practices: What We Are Doing and How Can We Improve? International Journal of Human-Computer Interaction, 32(1), 51–62. https://doi.org/10.1080/10447318.2015.1087664
IBM Corp. (2017). IBM SPSS Statistics for Windows. IBM Corp
Instituto Nacional de Estadística (INE) (2019). Encuesta sobre equipamiento y uso de tecnologías de información y comunicación en los hogares. Condiciones de vida. Nivel y condiciones de vida (IPC). INE. URL: https://www.ine.es/dyngs/INEbase/es/operacion.htm?c=Estadistica_C&cid=1254736176741&menu=ultiDatos&idp=1254735976608
Kaiser, H. F. (1970). A second generation little jiffy. Psychometrika, 35(4), 401-415.
Kazdin, A. E. (2015). Technology-based interventions and reducing the burdens of mental illness: perspectives and comments on the special series. Cognitive and Behavioral Practice, 22(3), 359-366. https://doi.org/10.1016/j.cbpra.2015.04.004
King, R., Bambling, M., Reid, W., & Thomas, I. (2006). Telephone and online counselling for young people: A naturalistic comparison of session outcome, session impact and therapeutic alliance. Counselling and Psychotherapy Research, 6(3), 175–181. https://doi.org/10.1080/14733140600874084
Kramer, G. M., & Luxton, D. D. (2016). Telemental Health for Children and Adolescents: An Overview of Legal, Regulatory, and Risk Management Issues. Journal of Child and Adolescent Psychopharmacology, 26(3), 198–203. https://doi.org/10.1089/cap.2015.0018
Kysely, A., Bishop, B., Kane, R., Cheng, M., De Palma, M., & Rooney, R. (2020). Expectations and Experiences of Couples Receiving Therapy Through Videoconferencing: A Qualitative Study. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.02992
Lai, M. H., Maniam, T., Chan, L. F., & Ravindran, A. V. (2014). Caught in the Web: A Review of Web-Based Suicide Prevention. Journal of Medical Internet Research, 16(1), e30. https://doi.org/10.2196/jmir.2973
Lavorgna, L., Russo, A., De Stefano, M., Lanzillo, R., Esposito, S., Moshtari, F., Rullani, F., Piscopo, K., Buonanno, D., Brescia Morra, V., Gallo, A., Tedeschi, G., & Bonavita, S. (2017). Health-Related Coping and Social Interaction in People with Multiple Sclerosis Supported by a Social Network: Pilot Study With a New Methodological Approach. Interactive Journal of Medical Research, 6(2), e10. https://doi.org/10.2196/ijmr.7402
Lenhard, W. & Lenhard, A. (2016). Calculation of Effect Sizes. Retrieved from: https://www.psychometrica.de/effect_size.html. Dettelbach (Germany): Psychometrica. https://doi.org/10.13140/RG.2.2.17823.92329
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 381–5.
Lorenzo-Seva, U. (2003). A factor simplicity index. Psychometrika, 68, 49-60. https://doi.org/10.1007/bf02296652
Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: A computer program to fi t the exploratory factor analysis model. Behavior Research Methods, 38(1), 88-91.
Lorenzo-Seva, U., & Ferrando, P. J. (2019). Robust Promin: A method for diagonally weighted factor rotation. Liberabit: Revista Peruana de Psicología, 25(1), 99–106. https://doi.org/10.24265/liberabit.2019.v25n1.0
Mallen, M. J., Day, S. X., & Green, M. A. (2003). Online versus face-to-face conversation: An examination of relational and discourse variables. Psychotherapy: Theory, Research, Practice, Training, 40(1-2), 155–163. https://doi.org/10.1037/0033-3204.40.1-2.155
Mallen, M. J., Vogel, D. L., & Rochlen, A. B. (2005). The Practical Aspects of Online Counseling. The Counseling Psychologist, 33(6), 776–818. https://doi.org/10.1177/0011000005278625
Mardia, K. V. (1970). Measures of Multivariate Skewness and Kurtosis with Applications. Biometrika, 57, 519-530. https://doi.org/10.1093/biomet/57.3.519
McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates.
Muthén, L. K., & Muthén, B. O. (2017). Mplus User's Guide. Sixth Edition. Muthén & Muthén. Statmodel. https://www.statmodel.com/download/usersguide/MplusUserGuideVer_8.pdf
Osborne, J. W., Costello, A. B., & Kellow, J. T. (2014). Best practices in exploratory factor analysis (pp. 86-99). Louisville, KY: CreateSpace Independent Publishing Platform. https://doi.org/10.4135/9781412995627.d8
Palos-Sanchez, P., Reyes-Menendez, A., & Saura, J. R. (2019). Modelos de Adopción de Tecnologías de la Información y Cloud Computing en las Organizaciones. Información Tecnológica, 30(3), 3–12. https://doi.org/10.4067/s0718-07642019000300003
Perle, J. G., Langsam, L. C., & Nierenberg, B. (2011). Controversy clarified: An updated review of clinical psychology and tele-health. Clinical Psychology Review, 31(8), 1247–1258. https://doi.org/10.1016/j.cpr.2011.08.003
Peter, L., Reindl, R., Zauter, S., Hillemacher, T., & Richter, K. (2019). Effectiveness of an Online CBT-I Intervention and a Face-to-Face Treatment for Shift Work Sleep Disorder: A Comparison of Sleep Diary Data. International Journal of Environmental Research and Public Health, 16(17), 3081. https://doi.org/10.3390/ijerph16173081
Pujadas, M. M., & Machín, A. L. (2006). El correo electrónico en la relación médico-paciente: uso y recomendaciones generales. Atención primaria, 37(7), 413-417.
Putnick, D. L., & Bornstein, M. H. (2016). Measurement Invariance Conventions and Reporting: The State of the Art and Future Directions for Psychological Research. Developmental review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315
Richards, D., & Richardson, T. (2012). Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clinical psychology review, 32(4), 329-342. https://doi.org/10.1016/j.cpr.2012.02.004
Richards, D., & Timulak, L. (2013). Satisfaction with therapist-delivered vs. self-administered online cognitive behavioural treatments for depression symptoms in college students. British Journal of Guidance y Counselling, 41(2), 193–207. https://doi.org/10.1080/03069885.2012.726347
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: calculating and interpreting statistical indices. Psychological Methods 21, 137–150. https://doi.org/10.1037/met0000045
Shore, J. H., Hilty, D. M., & Yellowlees, P. (2007). Emergency management guidelines for telepsychiatry. General Hospital Psychiatry, 29(3), 199–206. https://doi.org/10.1016/j.genhosppsych.2007.01.013
Sohn, S. Y., Rees, P., Wildridge, B., Kalk, N. J., & Carter, B. (2019). Prevalence of problematic smartphone usage and associated mental health outcomes amongst children and young people: a systematic review, meta-analysis and GRADE of the evidence. BMC psychiatry, 19(1), 1-10.
Soto-Pérez, F., Franco, M., Monardes, C. y Jiménez, F. (2010). Internet y psicología clínica : revisión de las ciber-terapias. Revista de Psicopatología y Psicología Clínica, 15(1). https://doi.org/10.5944/rppc.vol.15.num.1.2010.4082
Spijkerman, M. P. J., Pots, W. T. M., & Bohlmeijer, E. T. (2016). Effectiveness of online mindfulness-based interventions in improving mental health: A review and meta-analysis of randomised controlled trials. Clinical Psychology Review, 45, 102–114. https://doi.org/10.1016/j.cpr.2016.03.009
Stoll, J., Müller, J. A., & Trachsel, M. (2020). Ethical Issues in Online Psychotherapy: A Narrative Review. Frontiers in Psychiatry, 10. https://doi.org/10.3389/fpsyt.2019.00993
Svetina, D., Rutkowski, L., & Rutkowski, D. (2019): Multiple-Group Invariance with Categorical Outcomes Using Updated Guidelines: An Illustration Using Mplus and the lavaan/semTools Packages. Structural Equation Modeling: A Multidisciplinary Journal, 0, 1-20. https://doi.org/10.1080/10705511.2019.1602776
Tiainem, T., & Berki, E. (2019). The re-Production process of gender bias: A case of ICT professors through recruitment in a gender-neutral country. Studies in Higher Education, 44(1), 170-184. https://doi.org/10.1080/03075079.2017.1351428
Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209–220. https://doi.org/10.1037/a0023353
Vaquerizo, E. J. (2019). Eficacia de las intervenciones online basadas en redes sociales para promover actividades físicas. Lecturas: Educación Física y Deportes, 24(251), 124-136.)
Viladrich, V., Angulo-Brunet, A., & Eduardo, D. (2017). A journey around alpha and omega to estimate internal consistency reliability. Anales de Psicología, 33(3), 755-782. https://doi.org/10.6018/analesps.33.3.268401
Villadangos, S. M., & Labrador, F. J. (2009). Menores y nuevas tecnologías (NT): uso o abuso. Anuario de Psicología Clínica y de la Salud, 5(1), 75-83.
Wallin, E. E. K., Mattsson, S., & Olsson, E. M. G. (2016). The preference for internet-based psychological interventions by individuals without past or current use of mental health treatment delivered online: a survey study with mixed-methods analysis. JMIR mental health, 3(2), e25. https://doi.org/10.2196/mental.5324
Wechsler, T. F., Kümpers, F., & Mühlberger, A. (2019). Inferiority or Even Superiority of Virtual Reality Exposure Therapy in Phobias? — A Systematic Review and Quantitative Meta-Analysis on Randomized Controlled Trials Specifically Comparing the Efficacy of Virtual Reality Exposure to Gold Standard in vivo Exposure in Agoraphobia, Specific Phobia, and Social Phobia. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.01758
Witt, K., Spittal, M. J., Carter, G., Pirkis, J., Hetrick, S., Currier, D., … Milner, A. (2017). Effectiveness of online and mobile telephone applications (“apps”) for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysis. BMC Psychiatry, 17(1). https://doi.org/10.1186/s12888-017-1458-0
Wittkopf, P. G., Lloyd, D. M., & Johnson, M. I. (2018). Managing limb pain using virtual reality: a systematic review of clinical and experimental studies. Disability and Rehabilitation, 41(26), 3103–3117. https://doi.org/10.1080/09638288.2018.1485183
Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2), 49–54. https://doi.org/10.21315/eimj2019.11.2.6
Zaccaria, D., Casanova, G., & Guaita, A. (2019). Impact of ICT and social networks sites utilization on older people’s well-being and loneliness: a scoping review. Innovation in Aging, 3(Supplement_1), S530–S530. https://doi.org/10.1093/geroni/igz038.1949
Copyright (c) 2023 Servicio de Publicaciones, University of Murcia (Spain)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The works published in this journal are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) retains the property rights (copyright) of published works, and encourages and enables the reuse of the same under the license specified in paragraph 2.
© Servicio de Publicaciones, Universidad de Murcia, 2022
2. The works are published in the online edition of the journal under a Creative Commons Reconocimiento-CompartirIgual 4.0 (legal text). You can copy, use, distribute, transmit and publicly display, provided that: i) you cite the author and the original source of publication (journal, editorial and URL of the work), ii) are not used for commercial purposes, iii ) mentions the existence and specifications of this license.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
3. Conditions of self-archiving. Is allowed and encouraged the authors to disseminate electronically pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version reviewed and accepted for publication) of their works before publication, as it encourages its earliest circulation and diffusion and thus a possible increase in its citation and scope between the academic community. RoMEO Color: Green.
