Rating mean of expert judges and asymmetric confidence intervals in content validity: An SPSS syntax
Abstract
The estimation of content validity, obtained by rational analysis of expert judges, is usually done with coefficients that standardize between 0.0 and 1.0 the judges' judgment. However, this estimate can also be expressed in the metric of the judges' responses, in the form of the response mean, and with asymmetric confidence intervals around this mean. The aim of the present manuscript is to implement a procedure for these estimates (response mean and asymmetric confidence intervals) in a program written in SPSS syntax. The rationale of the procedure is explained, and an applied example of the calculation is developed. The program is freely distributed upon request to the authors.
Downloads
References
Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40(4), 955-959. https://doi.org/10.1177/001316448004000419
Bonett, D. G., & Price, R. M. (2020). Confidence intervals for ratios of means and medians. Journal of Educational and Behavioral Statistics, 20(10), 1–21. https://doi.org/10.3102/1076998620934125
Elosua, P. (2009). ¿Existe vida más allá del SPSS? Descubre R. Psicothema, 21(4), 652-655.
Escrig, J., Miralles, J. M., Martinez, D., & Rivadulla, I. (2007). Intervalos de confianza: por qué usarlos. Cirugía Española, 81(3), 121-125. https://doi.org/10.1016/S0009-739X(07)71281-2
Duricki, D. A., Soleman, S., & Moon, L. D. (2016). Analysis of longitudinal data from animals with missing values using SPSS. Nature protocols, 11(6), 1112–1129. https://doi.org/10.1038/nprot.2016.048
Gogoi, P. (2020). Application of SPSS programme in the field of social science research. International Journal of Recent Technology and Engineering, 8(5), 2424-2426. https://doi.org/10.35940/ijrte.D9260.018520
Haine, D. (2019). Popularity of statistical softwares in epidemiology. Available in: https://www.denishaine.ca/blog/popepi-rmd/
Hernandez-Nieto, R. A. (2002), Contributions to Statistical Analysis. Merida, Venezuela: Universidad de Los Andes.
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382-385. https://doi.org/10.1097/00006199-198611000-00017
Masuadi, E., Mohamud, M., Almutairi, M., Alsunaidi, A., Alswayed, A. K., & Aldhafeeri, O. F. (2021). Trends in the usage of statistical software and their associated study designs in health sciences research: A bibliometric analysis. Cureus, 13(1), e12639. https://doi.org/10.7759/cureus.12639
Merino-Soto, C. (2018). Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. Anales de Psicología, 34(3), 587-590. https://dx.doi.org/10.6018/analesps.34.3.283481
Merino-Soto, C., & Livia-Segovia, J. (2009). Intervalos de confianza asimétricos para el índice la validez de contenido: Un programa Visual Basic para la V de Aiken [Asymmetric confidence intervals for index content validity: A Visual Basic program for Aiken's V]. Anales de Psicología, 25(1),169-171.
Miller, J. M., & Penfield, R. D. (2005). Using the score method to construct asymmetric confidence intervals: An SAS program for content validation in scale development. Behavior Research Methods, 37, 450-452. https://doi.org/10.3758/BF03192713
Moscoso, M. S., & Merino-Soto, C. (2017). Construcción y validez de contenido del Inventario de Mindfulness y Ecuanimidad: una perspectiva iberoamericana [Construction and content validity of the Mindfulness and Equanimity Inventory: an Ibero-American perspective]. Mindfulness & Compassion, 2(1), 9-16. https://doi.org/10.1016/j.mincom.2017.01.001
Muenchen, R. A. (2017). The popularity of data science software. Acceso: 22/04/2021. Available in: http://r4stats.com/articles/popularity/
Osterlind, S. J. (1992). Constructing test items: multiple-choice, constructed-response, performance, and other formats. Boston: Kluwer Academic Publishers.
Pedrosa, I., Suárez-Álvarez, J., & García-Cueto, E. (2013). Evidencias sobre la validez de contenido: avances teóricos y métodos para su estimación. Acción Psicológica, 10(2), 3-18.
Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals for the mean of a rating scale item. Psychological methods, 8(2), 149–163. https://doi.org/10.1037/1082-989x.8.2.149
Penfield, R. D., & Miller, J. M. (2004). Improving content validation studies using an asymmetric confidence interval for the mean of expert ratings. Applied Measurement in Education, 17, 359-370. https://doi.org/10.1207/s15324818ame1704_2
Penfield, R. D., & Giacobbi, P. R. (2004). Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8, 213-225. https://doi.org/10.1207/s15327841mpee0804_
Rivadeneira Pacheco, J. L., Barrera Argüello, M. V., & De La Hoz Suárez, A. I. (2020). Análisis general del SPSS y su utilidad en la estadística [General analysis of SPSS and its usefulness in statistics]. Journal of Business Sciences, 2(4), 17-25. https://revista.estudioidea.org/ojs/index.php/eidea/article/view/19
Rovinelli, R. J. & Hambleton, R. K. (1977). On the use of content specialists in the assessment of criterion-referenced test item validity. Dutch Journal of Educational Research, 2, 49-60.
Shaikh, M. A. (2016). Use of statistical tests and statistical software choice in 2014: tale from three Medline indexed Pakistani journals. The Journal of the Pakistan Medical Association, 66(4), 464-466.
Shaikh, M. A. (2017). Study designs, use of statistical tests, and statistical analysis software choice in 2015: Results from two Pakistani monthly Medline indexed journals. The Journal of the Pakistan Medical Association, 67(9), 1428-1431.
Tellez, A., Garcia, C., & Corral-Verdugo, V. (2015). Effect size, confidence intervals and statistical power in psychological research. Psychology in Russia: State of the Art, 8(3), 27-46. https://doi.org/10.11621/pir.2015.0303
Valeri, L., & Vanderweele, T. J. (2013). Mediation analysis allowing for exposure-mediator interactions and causal interpretation: theoretical assumptions and implementation with SAS and SPSS macros. Psychological methods, 18(2), 137–150. https://doi.org/10.1037/a0031034
Vanus, J., Kubicek, J., Gorjani, O. M., & Koziorek, J. (2019). Using the IBM SPSS SW Tool with wavelet transformation for CO₂ prediction within iot in smart home care. Sensors, 19(6), 1407. https://doi.org/10.3390/s19061407
Willink, R. (2005). A confidence interval and test for the mean of an asymmetric distribution. Communications in Statistics - Theory and Methods, 34(4), 753-766. https://doi.org/10.1081/STA-200054419
Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.1080/01621459.1927.10502953
Copyright (c) 2022 Servicio de Publicaciones, University of Murcia (Spain)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The works published in this journal are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) retains the property rights (copyright) of published works, and encourages and enables the reuse of the same under the license specified in paragraph 2.
© Servicio de Publicaciones, Universidad de Murcia, 2022
2. The works are published in the online edition of the journal under a Creative Commons Reconocimiento-CompartirIgual 4.0 (legal text). You can copy, use, distribute, transmit and publicly display, provided that: i) you cite the author and the original source of publication (journal, editorial and URL of the work), ii) are not used for commercial purposes, iii ) mentions the existence and specifications of this license.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
3. Conditions of self-archiving. Is allowed and encouraged the authors to disseminate electronically pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version reviewed and accepted for publication) of their works before publication, as it encourages its earliest circulation and diffusion and thus a possible increase in its citation and scope between the academic community. RoMEO Color: Green.
