Authenticity Scale: Validity and Reliability Evidence in a Sample from Brazil and Portugal
Abstract
This study introduces distinct types of validity and reliability evidence of the Authenticity Scale (AS) in a sample from Brazil and Portugal. It consists of an online survey with 1,077 Brazilian citizens and 622 Portuguese citizens. The study tested the model with three correlated factors (self-alienation, authentic living, and accepting external influence), the unidimensional model, and the bi-factor model. The model with three correlated factors was retained, with the three subscales demonstrating moderate to good reliability. Multigroup confirmatory factor analysis suggested scalar invariance across culture, gender, age, education, occupation, and Covid-related concern and impact. The items were assessed by graded response model (GRM), which suggested that the three subscales are not able to distinguish respondents with high authenticity traits. GRM and descriptive statistics indicated that the rating-scale is inappropriate, particularly for authentic living subscale, which is affected by ceiling effect. Associations with presence of meaning showed additional validity evidence. Despite the limitations, the AS is an effective measure to assess authenticity across different groups. Potential modifications for the improvement of the AS are discussed.
Downloads
References
Akin, A., & Taş, İ. (2015). Yaşam anlami ölçeği: Geçerlik ve güvenirlik çalişmasi [Meaning in life questionnaire: A study of validity and reliability]. Turkish Studies, 10(3), 27-36. https://doi.org/10.7827/TurkishStudies.7860
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. American Educational Research Association.
Baker, F. B., & Kim, S. H. (2017). The basics of item response theory using R. Springer International Publishing. https://doi.org/10.1007/978-3-319-54205-8
Balbino, I. F., Galinha, I. C., Morais, C. C., & Calado, S. S. (2018). Contributo para a validação da versão portuguesa da Escala de Autenticidade [Contribution to the validation of the Portuguese version of the Authenticity Scale]. Psicologia, Saúde & Doenças, 19(3), 564–577. https://doi.org/10.15309/18psd190308
Barrett-Lennard, G. T. (1998). Carl Rogers’ helping system: Journey and substance. Sage.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. The Guilford Press.
Chalmers, R. P. (2012). mirt: A Multidimensional Item Response Theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. https://doi.org/10.18637/jss.v048.i06
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5
Cook, K. F., Kallen, M. A., & Amtmann, D. (2009). Having a fit: Impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption. Quality of Life Research, 18(4), 447-460. https://doi.org/10.1007/s11136-009-9464-4
Damásio, B. F., & Koller, S. H. (2015). Meaning in Life Questionnaire: Adaptation process and psychometric properties of the Brazilian version. Revista Latinoamericana de Psicología, 47(3), 185-195. https://doi.org/10.1016/j.rlp.2015.06.004
Dancey, C. P., & Reidy, J. (2007). Statistics without maths for psychology. Pearson Education.
Di Fabio, A. (2014). Authenticity Scale: Un primo contributo alla validazione della versione italiana [Authenticity Scale: A first contribution to validation of the Italian version]. Counseling: Giornale Italiano di Ricerca e Applicazioni, 7(2), 231-238.
Dunn, K. J., & McCray, G. (2020). The place of the bifactor model in confirmatory factor analysis investigations into construct dimensionality in language testing. Frontiers in Psychology, 11, 1357. https://doi.org/10.3389/fpsyg.2020.01357
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.2307/3151312
Gable, S. L., & Haidt, J. (2005). What (and why) is positive psychology? Review of General Psychology, 9, 103-110. https://doi.org/10.1037/1089-2680.9.2.103
Gliem, J. A., & Gliem, R. R. (2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. Proceedings of the Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education, USA.
Grégoire, S., Baron, L., Ménard, J., & Lachance, L. (2014). The Authenticity Scale: Psychometric properties of a French translation and exploration of its relationships with personality and well-being. Canadian Journal of Behavioural Science, 46(3), 346-355. https://doi.org/10.1037/a0030962
Grijak, Đ. (2017). Psychometric evaluation of the authenticity scale on the sample of students in Serbia. Psihologija, 50(1), 85-99. https://doi.org/10.2298/PSI160504001G
İlhan, T., & Özdemir, Y. (2013). Otantiklik Ölçeğinin Türkçe’ye uyarlanması: Geçerlik ve güvenirlik çalışması [Adaptation of Authenticity Scale to turkish: A validity and reliability study]. Turkish Psychological Counseling and Guidance Journal, 4(40), 142-153.
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2021). semTools: Useful tools for structural equation modeling. https://CRAN.R-project.org/package=semTools
Kernis, M. H., & Goldman, B. M. (2005). From thought and experience to behavior and interpersonal relationships: A multicomponent conceptualization of authenticity. In A. Tesser, J. V. Wood, & D. Stapel (Eds.), On building, defending, and regulating the self: A psychological perspective (pp. 31-52). Psychology Press.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). The Guilford Press.
Li, C. H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavioral Research Methods, 48(3), 936-949. https://doi.org/10.3758/s13428-015-0619-7
Linacre, J. M. (1999). Investigating rating scale category utility. Journal of Outcome Measurement, 3(2), 103-122.
Mokken, R. J. (1971). Theory and procedure of scale analysis: With applications in political research. The Hague Mouton.
Nartova-Bochaver, S., Reznichenko, S., & Maltby, J. (2021). The Authenticity Scale: validation in Russian culture. Frontiers in Psychology, 11, 609617. https://doi.org/10.3389/fpsyg.2020.609617
Nunes, C. H. S. D. S., & Primi, R. (2005). Impact of the sample size in the item and subject's parameters estimates under item response theory. Avaliação Psicológica, 4(2), 141-153.
Paek, I., & Cole, K. (2019). Using R for item response theory model applications. Routledge.
Peterson, R. A., Kim, Y., & Choi, B. (2020). A meta-analysis of construct reliability indices and measurement model fit metrics. Methodology, 16(3), 208-223. https://doi.org/10.5964/meth.2797
Portugal, M. V. (2017). Versão portuguesa do Questionário do Sentido da Vida: Primeiros estudos psicométricos [Portuguese version of the Meaning in Life Questionnaire]. Non-published master thesis, Universidade de Lisboa. https://repositorio.ul.pt/bitstream/10451/33211/1/ulfpie052851_tm.pdf
R Core Team. (2023). R: A language and environment for statistical computing (version 4.1.3) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354-373. https://doi.org/10.1037/a0029315
Rogers, C. R. (1961). On becoming a person: A therapist’s view of psychotherapy. Constable.
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(Suppl 1), 1-97. https://doi.org/10.1007/BF03372160
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-338. https://doi.org/10.3200/JOER.99.6.323-338
Shamsi, A., Ghamarani, A., Samadi, M., & Ahmadzadeh, M. (2012). The study of the validity and reliability of the Authentic Personality Scale. Psychological Methods and Models, 2(8), 89-100.
Sherman, R. A. (2015). Multicon: An R Package for the Analysis of Multivariate Contructs, R package Version 1.6. https://cran.r-project.org/web/packages/multicon/multicon.pdf
Sijtsma, K., & Molenaar, I. W. (1987). Reliability of test scores in nonparametric item response theory. Psychometrika, 52(1), 79–97. https://doi.org/10.1007/BF02293957
Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. SAGE Publications.
Soper, D.S. (2023). A-priori sample size calculator for structural equation models [software]. Available from https://www.danielsoper.com/statcalc
Steger, M. F., Frazier, P., Oishi, S., & Kaler, M. (2006). The meaning in life questionnaire: assessing the presence of and search for meaning in life. Journal of Counseling Psychology, 53(1), 80-93. https://doi.org/10.1037/0022-0167.53.1.80
Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., ... & de Vet, H. C. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34-42. https://doi.org/10.1016/j.jclinepi.2006.03.012
Vainio, M. M., & Daukantaitė, D. (2016). Grit and different aspects of well-being: Direct and indirect relationships via sense of coherence and authenticity. Journal of Happiness Studies, 17(5), 2119-2147. https://doi.org/10.1007/s10902-015-9688-7
Valentini, F., & Damásio, B. F. (2016). Variância Média Extraída e Confiabilidade Composta: Indicadores de precisão [Average Variance Extracted and Composite Reliability: Reliability coefficients]. Psicologia: Teoria e Pesquisa, 32(2), 1-7. https://doi.org/10.1590/0102-3772e322225
Winnicott, D. W. (1965). The maturational processes and the facilitating environment. International Universities Press.
Wood, A. M., Linley, P. A., Maltby, J., Baliousis, M., & Joseph, S. (2008). The authentic personality: A theoretical and empirical conceptualization and the development of the Authenticity Scale. Journal of Counseling Psychology, 55(3), 385-399. https://doi.org/10.1037/0022-0167.55.3.385
Xia, M., Lv, H., & Xu, X. (2022). Validating the Chinese version authenticity scale: Psychometrics in college and community samples. Current Psychology, 41, 7301–7313. https://doi.org/10.1007/s12144-020-01326-7
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213. https://doi.org/10.1111/j.1745-3984.1993.tb00423.x
Zlyvkov, V. L., Lukomska, S. O., Kotukh, О. V., Dykhovichnyi, O. O., & Kruglova, N. V. (2019). Authenticity of the english language teacher’s: the validation of authenticity questionnaire using item response theory. Science progress in European countries: new concepts and modern solutions, 335.
Zoysa, P., Kumar, S., Amarasuriya, S. D., & Mendis, N. S. (2021). Being yourself: An assessment of authenticity in undergraduates of a University in Sri Lanka. Asia Pacific Journal of Counselling and Psychotherapy, 12(2), 138-153. https://doi.org/10.1080/21507686.2021.1924810
Copyright (c) 2024 Servicio de Publicaciones, University of Murcia (Spain)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The works published in this journal are subject to the following terms:
1. The Publications Service of the University of Murcia (the publisher) retains the property rights (copyright) of published works, and encourages and enables the reuse of the same under the license specified in paragraph 2.
© Servicio de Publicaciones, Universidad de Murcia, 2022
2. The works are published in the online edition of the journal under a Creative Commons Reconocimiento-CompartirIgual 4.0 (legal text). You can copy, use, distribute, transmit and publicly display, provided that: i) you cite the author and the original source of publication (journal, editorial and URL of the work), ii) are not used for commercial purposes, iii ) mentions the existence and specifications of this license.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
3. Conditions of self-archiving. Is allowed and encouraged the authors to disseminate electronically pre-print versions (version before being evaluated and sent to the journal) and / or post-print (version reviewed and accepted for publication) of their works before publication, as it encourages its earliest circulation and diffusion and thus a possible increase in its citation and scope between the academic community. RoMEO Color: Green.
