Data from: Correction for bias in meta-analysis of little-replicated studies

(2018) Data from: Correction for bias in meta-analysis of little-replicated studies. DRYAD doi:10.5061/dryad.5f4g6 [Dataset]

Abstract

Data S1R script for simulations. Simulations of fixed- and random-effects meta-analysis using alternative estimators: one-sample mean, two-sample Hedges' g, and two-sample lnR, for comparison of performance by inverse-variance weighting and inverse-adjusted-variance weighting.Doncaster&Spake_Data_S1.txtData S2R script for calculating mean-adjusted error variance. Finds the mean-adjusted study variance for all the primary studies contributing to a meta-analysis, for a one-sample mean, or two-sample log response ratio, or or two-sample Hedges' g.Doncaster&Spake_Data_S2.txt,1. Meta-analyses conventionally weight study estimates on the inverse of their error variance, in order to maximize precision. Unbiased variability in the estimates of these study-level error variances increases with the inverse of study-level replication. Here we demonstrate how this variability accumulates asymmetrically across studies in precision-weighted meta-analysis, to cause undervaluation of the meta-level effect size or its error variance (the meta-effect and meta-variance). 2. Small samples, typical of the ecological literature, induce big sampling errors in variance estimation, which substantially bias precision-weighted meta-analysis. Simulations revealed that biases differed little between random- and fixed-effects tests. Meta-estimation of a one-sample mean from 20 studies, with sample sizes of 3 to 20 observations, undervalued the meta-variance by ~20%. Meta-analysis of two-sample designs from 20 studies, with sample sizes of 3 to 10 observations, undervalued the meta-variance by 15-20% for the log response ratio (lnR); it undervalued the meta-effect by ~10% for the standardised mean difference (SMD). 3. For all estimators, biases were eliminated or reduced by a simple adjustment to the weighting on study precision. The study-specific component of error variance prone to sampling error and not parametrically attributable to study-specific replication was replaced by its cross-study mean, on the assumption of random sampling from the same population variance for all studies, and sufficient studies for averaging. Weighting each study by the inverse of this mean-adjusted error variance universally improved accuracy in estimation of both the meta-effect and its significance, regardless of number of studies. For comparison, weighting only on sample size gave the same improvement in accuracy, but could not sensibly estimate significance. 4. For the one-sample mean and two-sample lnR, adjusted weighting also improved estimation of between-study variance by DerSimonian-Laird and REML methods. For random-effects meta-analysis of SMD from little-replicated studies, the most accurate meta-estimates obtained from adjusted weights following conventionally-weighted estimation of between-study variance. 5. We recommend adoption of weighting by inverse adjusted-variance for meta-analyses of well- and little-replicated studies, because it improves accuracy and significance of meta-estimates, and it can extend the scope of the meta-analysis to include some studies without variance estimates.

This record has no associated files available for download.

Contributors

Contributor: C. Patrick Doncaster

Download statistics