The University of Southampton
University of Southampton Institutional Repository

Examining the context sensitivity of research findings from archival data

Examining the context sensitivity of research findings from archival data
Examining the context sensitivity of research findings from archival data
This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies. 45% of the reproductions returned results matching the original reports, together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability — for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Research reliability, archival data, context sensitivity, generalizability, reproducibility
0027-8424
Delios, Andrew
418be60e-1fb4-4509-8c7a-7fa6087e90c0
Clemente, Elena Giulia
9616a5e0-1c28-493a-9fef-885abe857789
Wu, Tao
c30f1ec4-92ee-424b-8883-5881487f2f4b
Tan, Hongbin
d7708e3a-4550-4df3-bc2c-a1d810622858
Wang, Yong
1210cde2-c9aa-47bc-b04f-63f4506133b4
Gordon, Michael
a50bd3f4-22c0-4fbf-9d81-bd2b2a6c0a75
Viganola, Domenico
7797c7b4-8fd4-4935-a218-13fe90692cce
Chen, Zhaowei
5a4bd2e8-0f54-46cb-87a2-40e5b0355a48
Dreber, Anna
98c5d999-5ba7-44f1-a3da-b3903ea89c48
Johannesson, Magnus
4a1049cb-7269-4614-bc08-82e73cb6c013
Pfeiffer, Thomas
607995fd-2389-489e-af00-cabc1d103315
Uhlmann, Eric Luis
7caa5ebf-05f8-49bc-b793-b1429c3e2d47
Dawson, Ian
dff1b440-6c83-4354-92b6-04809460b01a
Generalizability Tests Forecasting Collaboration
Delios, Andrew
418be60e-1fb4-4509-8c7a-7fa6087e90c0
Clemente, Elena Giulia
9616a5e0-1c28-493a-9fef-885abe857789
Wu, Tao
c30f1ec4-92ee-424b-8883-5881487f2f4b
Tan, Hongbin
d7708e3a-4550-4df3-bc2c-a1d810622858
Wang, Yong
1210cde2-c9aa-47bc-b04f-63f4506133b4
Gordon, Michael
a50bd3f4-22c0-4fbf-9d81-bd2b2a6c0a75
Viganola, Domenico
7797c7b4-8fd4-4935-a218-13fe90692cce
Chen, Zhaowei
5a4bd2e8-0f54-46cb-87a2-40e5b0355a48
Dreber, Anna
98c5d999-5ba7-44f1-a3da-b3903ea89c48
Johannesson, Magnus
4a1049cb-7269-4614-bc08-82e73cb6c013
Pfeiffer, Thomas
607995fd-2389-489e-af00-cabc1d103315
Uhlmann, Eric Luis
7caa5ebf-05f8-49bc-b793-b1429c3e2d47
Dawson, Ian
dff1b440-6c83-4354-92b6-04809460b01a

Delios, Andrew, Clemente, Elena Giulia and Wu, Tao , Generalizability Tests Forecasting Collaboration (2022) Examining the context sensitivity of research findings from archival data. Proceedings of the National Academy of Sciences, 119 (30), [e2120377119]. (doi:10.1073/pnas.2120377119).

Record type: Article

Abstract

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies. 45% of the reproductions returned results matching the original reports, together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability — for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.

Text
Examining the Generalizability of Research Findings from Archival Data - Manuscript and Supp Materials - Accepted Manuscript
Download (2MB)

More information

e-pub ahead of print date: 19 July 2022
Keywords: Research reliability, archival data, context sensitivity, generalizability, reproducibility

Identifiers

Local EPrints ID: 468536
URI: http://eprints.soton.ac.uk/id/eprint/468536
ISSN: 0027-8424
PURE UUID: 406b635b-7e28-447a-b19c-3391e40729a5
ORCID for Ian Dawson: ORCID iD orcid.org/0000-0003-0555-9682

Catalogue record

Date deposited: 17 Aug 2022 17:03
Last modified: 19 Jan 2023 05:01

Export record

Altmetrics

Contributors

Author: Andrew Delios
Author: Elena Giulia Clemente
Author: Tao Wu
Author: Hongbin Tan
Author: Yong Wang
Author: Michael Gordon
Author: Domenico Viganola
Author: Zhaowei Chen
Author: Anna Dreber
Author: Magnus Johannesson
Author: Thomas Pfeiffer
Author: Eric Luis Uhlmann
Author: Ian Dawson ORCID iD
Corporate Author: Generalizability Tests Forecasting Collaboration

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×