Investigating the analytical robustness of the social and behavioural sciences

StatusVoR
Dublin Core metadata
Alternative title
Authors
Aczel, Balazs
Szaszi, Barnabas
Clelland, Harry T.
Kovacs, Marton
Holzmeister, Felix
van Ravenzwaaij, Don
Schulz-Kümpel, Hannah
Hoffmann, Sabine
Nilsonne, Gustav
Kosa, Livia
Monograph
Monograph (alternative title)
Date
2026-04-01
Publisher
Journal title
Nature
Issue
Volume
652
Pages
Pages
135–142
DOI
ISSN
0028-0836
ISSN of series
Weblink
https://www.nature.com/articles/s41586-025-09844-9
Access date
Abstract PL
Abstract EN
The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1,2,3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, in which, for one claim per study, at least five reanalysts independently reanalysed the original data. The statistical appropriateness of the reanalyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent reanalyses yielded the same result (within a tolerance region of ±0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator increased to 57%. Of the reanalyses conducted, 74% reached the same conclusion as the original investigation, 24% yielded no effects or inconclusive results and 2% reported the opposite effect. This exploratory study indicates that the common single-path analyses in social and behavioural research should not be simply assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.
Abstract other
Keywords PL
Keywords EN
Keywords other
Sustainable Development Goals
SDG 4. Quality Education
Exhibition title
Place of exhibition (institution)
Exhibition curator
URI
https://share.swps.edu.pl/handle/swps/2296
Type
License type
closedaccess
There is no access to this item
Funder
Time range from
Time range to
Contact person name
Related publication
Related publication
Grant/project name
Views
Views0
Acquisition Date20.10.2022
Downloads
Downloads0
Acquisition Date20.10.2022
Altmetrics©
Dimensions
Google Scholar
Google Scholar