Ignoring non-English-language studies may bias ecological meta-analyses

Abstract

Meta‐analysis plays a crucial role in syntheses of quantitative evidence in ecology and biodiversity conservation. The reliability of estimates in meta‐analyses strongly depends on unbiased sampling of primary studies. Although earlier studies have explored potential biases in ecological meta‐analyses, biases in reported statistical results and associated study characteristics published in different languages have never been tested in environmental sciences. We address this knowledge gap by systematically searching published meta‐analyses and comparing effect‐size estimates between English‐ and Japanese‐language studies included in existing meta‐analyses. Of the 40 published ecological meta‐analysis articles authored by those affiliated to Japanese institutions, we find that three meta‐analysis articles searched for studies in the two languages and involved sufficient numbers of English‐ and Japanese‐language studies, resulting in four eligible meta‐analyses (i.e., four meta‐analyses conducted in the three meta‐analysis articles). In two of the four, effect sizes differ significantly between the English‐ and Japanese‐language studies included in the meta‐analyses, causing considerable changes in overall mean effect sizes and even their direction when Japanese‐language studies are excluded. The observed differences in effect sizes are likely attributable to systematic differences in reported statistical results and associated study characteristics, particularly taxa and ecosystems, between English‐ and Japanese‐language studies. Despite being based on a small sample size, our findings suggest that ignoring non‐English‐language studies may bias outcomes of ecological meta‐analyses, due to systematic differences in study characteristics and effect‐size estimates between English‐ and non‐English languages. We provide a list of actions that meta‐analysts could take in the future to reduce the risk of language bias.

More information

Identifiers

Catalogue record

Export record

Altmetrics