当前位置: X-MOL 学术Psychological Methods › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Questionable research practices and cumulative science: The consequences of selective reporting on effect size bias and heterogeneity.
Psychological Methods ( IF 10.929 ) Pub Date : 2023-03-23 , DOI: 10.1037/met0000572
Samantha F Anderson 1 , Xinran Liu 1
Affiliation  

Despite increased attention to open science and transparency, questionable research practices (QRPs) remain common, and studies published using QRPs will remain a part of the published record for some time. A particularly common type of QRP involves multiple testing, and in some forms of this, researchers report only a selection of the tests conducted. Methodological investigations of multiple testing and QRPs have often focused on implications for a single study, as well as how these practices can increase the likelihood of false positive results. However, it is illuminating to consider the role of these QRPs from a broader, literature-wide perspective, focusing on consequences that affect the interpretability of results across the literature. In this article, we use a Monte Carlo simulation study to explore the consequences of two QRPs involving multiple testing, cherry picking and question trolling, on effect size bias and heterogeneity among effect sizes. Importantly, we explicitly consider the role of real-world conditions, including sample size, effect size, and publication bias, that amend the influence of these QRPs. Results demonstrated that QRPs can substantially affect both bias and heterogeneity, although there were many nuances, particularly relating to the influence of publication bias, among other factors. The present study adds a new perspective to how QRPs may influence researchers' ability to evaluate a literature accurately and cumulatively, and points toward yet another reason to continue to advocate for initiatives that reduce QRPs. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

中文翻译:

有问题的研究实践和累积科学:选择性报告对效应量偏差和异质性的影响。

尽管对开放科学和透明度的关注越来越多,但有问题的研究实践 (QRP) 仍然很普遍,使用 QRP 发表的研究在一段时间内仍将是发表记录的一部分。一种特别常见的 QRP 类型涉及多项测试,并且在某些形式中,研究人员仅报告了所进行测试的一部分。多项测试和 QRP 的方法学调查通常侧重于对单个研究的影响,以及这些做法如何增加假阳性结果的可能性。然而,从更广泛的、文献范围的角度考虑这些 QRP 的作用,关注影响整个文献结果可解释性的后果,是很有启发性的。在本文中,我们使用蒙特卡洛模拟研究来探索涉及多重测试、樱桃挑选和问题拖钓的两个 QRP 对效应量偏差和效应量之间的异质性的影响。重要的是,我们明确考虑了现实世界条件的作用,包括样本量、效应量和发表偏倚,它们修正了这些 QRP 的影响。结果表明,QRP 可以显着影响偏倚和异质性,尽管存在许多细微差别,特别是与发表偏倚的影响以及其他因素有关。本研究为 QRP 如何影响研究人员准确和累积地评估文献的能力增加了一个新的视角,并指出了继续倡导减少 QRP 的举措的另一个原因。(PsycInfo 数据库记录 (c) 2023 APA,
更新日期:2023-03-23
down
wechat
bug