The meta-analytical random effects model with g tends to underestimate parameters: an alternative model

Mario Calabria Sen; Juan Botella; Juan I. Durán

doi:10.6018/analesps.654891

Authors

Mario Calabria Sen Universidad Autónoma de Madrid https://orcid.org/0009-0005-1968-7695
Juan Botella Universidad Autónoma de Madrid https://orcid.org/0000-0001-8633-4981
Juan I. Durán https://orcid.org/0000-0001-8562-0919

DOI: https://doi.org/10.6018/analesps.654891

Keywords: Mixture Model, Random Effects Model, Meta-analysis, Standardized Mean Difference

Abstract

The Classic Random Effects Model (CREM) has some limitations when using the standardized mean difference (SMD) as effect size index. Suero et al. (2025) have reformulated CREM as a Mixture Model (MM) and have developed unbiased estimators of the main parameters µ_δand τ². They compared their performance with two classic methods widely used: Restricted Maximum Likelihood (REML; Viechtbauer, 2005) and DerSimonian & Laird estimator (DL; 1986), finding small but systematic underestimations yielded by the classic procedures. The aim of this study is to check if Suero et al.’s (2025) results can be found beyond simulation contexts. For this purpose, we created three databases with real meta-analyses (MA) from clinical, experimental and educations fields of psychology. The results found are consistent with those found by Suero et al. (2025) being the mean estimates of MM higher than those of REML and DL. We also discuss about outliers found in real MAs such as bizarre effect sizes, disproportionated sample sizes or MAs with small numbers of primary studies.

Downloads

Download data is not yet available.

Metrics

Views/Downloads

Abstract
19
pdf
2

References

Bakbergenuly, I., Hoaglin, D. C., & Kulinskaya, E. (2020). Estimation in meta-analyses of mean difference and standardized mean difference. Statistics in Medicine, 39, 171–191. https://doi.org/10.1002/sim.8422

Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4), 1088–1101.

Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2023). Heterogeneity estimation in meta-analysis of standardized mean differences when the distribution of random effects departs from normal: A Monte Carlo simulation study. BMC medical research methodology, 23(1), 19. https://doi.org/10.1186/s12874-022-01809-0

Böhning, D., Malzahn, U., Dietz, E., Schlattmann, P., Viwatwongkasem, C., & Biggeri, A. (2002). Some general points in estimating heterogeneity variance with the DerSimonian–Laird estimator. Biostatistics, 3(4), 445-457.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177–188. https://doi.org/10.1016/0197-2456(86)90046-2

Durán, J. I., Suero, M., & Botella, J. (2024, November 4). MixtureREM. Retrieved from osf.io/rwt2q

Friedman, L. (2000). Estimators of random effects variance components in meta-analysis. Journal of Educational and Behavioral Statistics, 25(1), 1-12.

Guolo, A., & Varin, C. (2017). Random-effects meta-analysis: the number of studies matters. Statistical methods in medical research, 26(3), 1500–1518. https://doi.org/10.1177/0962280215583568

Hamman, E. A., Pappalardo, P., Bence, J. R., Peacor, S. D., & Osenberg, C. W. (2018). Bias in meta‐analyses using Hedges’d. Ecosphere, 9(9), e02419.

Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.2307/1164588

Hedges, L. V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93, 388-395.

Hedges, L. V. & Olkin, I. (1985). Statistical Methods for Metaanalysis. San Diego, CA: Academic Press.

Jackson, D., & Turner, R. (2017). Power analysis for random-effects meta-analysis. Research synthesis methods, 8(3), 290–302. https://doi.org/10.1002/jrsm.1240

Kühberger, A., Fritz, A. & Scherndl, T. (2014). Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size. PLoS ONE 9(9): e105825. https://doi.org/10.1371/journal.pone.0105825

Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019). A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Research synthesis methods, 10(1), 83–98. https://doi.org/10.1002/jrsm.1316

Lin, L., & Aloe, A. M. (2021). Evaluation of various estimators for standardized mean difference in meta-analysis. Statistics in Medicine, 40, 403–426. https://doi.org/10.1002/sim.8781

López-Nicolás, R. (2023). Research Synthesis: Challenges, Reproducibility and Future Directions. [Unpublished doctoral dissertation] Universidad de Murcia.

Marks-Anglin, A. & Chen, Y. (2020). Small-study effects: current practice and challenges for future research. Statistics and Its Interface, 13(4), 457-484.

Sterne, J. A., Gavaghan, D., & Egger, M. (2000). Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. Journal of clinical epidemiology, 53(11), 1119-1129.

Suero, M. Botella, J. Durán, J. I. & Blázquez-Rincón, D. (2025). Reformulating the Meta-analytical Random Effects Model as a Mixture Model. Behavior Research Methods, 57, 7. https://doi.org/10.3758/s13428-024-02554-6

Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016). Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research synthesis methods, 7(1), 55–79. https://doi.org/10.1002/jrsm.1164

Viechtbauer W (2010). “Conducting meta-analyses in R with the metafor package.” Journal of Statistical Software, 36(3), 1–48. doi:10.18637/jss.v036.i03.

Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30(3), 261- 293.