Category: reproducibility

Contrary to common belief, randomised controlled trials inevitably produce biased results

Much of the social and medical sciences depend on randomised control trials. But while this may be considered the foundational experimental method, a certain degree of bias inevitably arises in any trial; whether this is sample bias, selection bias, or measurement bias. This is important as the level of validity of a trial’s causal claims can be a matter of […]

To save the research literature, let’s make literature reviews reproducible

Last week the Impact Blog featured a post from Richard P. Phelps, in which he proposed that journals get rid of their requirement for a literature review. Arnaud Vaganay agrees with much of what Phelps said, literature reviews are erratic and self-serving, but suggests doing away with them altogether is likely to make science less efficient and less credible. Instead, […]

Journal data sharing policies are moving the scientific community towards greater openness but clearly more work remains

Data sharing is a key part of the drive towards greater openness in scientific research, allowing readers to reproduce and confirm an article’s findings, or even reuse its data as part of a new study. Many journals have policies requiring researchers to share their data in full, with PLOS being a forerunner in this area. But how effective has the […]

Software updates: the “unknown unknown” of the replication crisis

The replication crisis is largely concerned with known problems, such as the lack of replication standards, non-availability of data, or p-hacking. One hitherto unknown problem is the potential for software companies’ changes to the algorithms used for calculations to cause discrepancies between two sets of reported results. Anastasia Ershova and Gerald Schneider encountered this very problem in the course of […]

Six principles for assessing scientists for hiring, promotion, and tenure

The negative consequences of relying too heavily on metrics to assess research quality are well known, potentially fostering practices harmful to scientific research such as p-hacking, salami science, or selective reporting. The “flourish or perish” culture defined by these metrics in turn drives the system of career advancement in academia, a system that empirical evidence has shown to be problematic […]

We need more carrots: give academic researchers the support and incentives to share data

Making data available for other researchers to find, use, reuse, and reproduce is fundamental to open science, and ultimately makes research more efficient and effective. Yet despite funder policies that encourage and require data sharing, researchers in both the UK and the US report lower percentages of data sharing than the global average. In addition to progressive policies, Grace Baynes […]

What factors do scientists perceive as promoting or hindering scientific data reuse?

Increased calls for data sharing have formed part of many governments’ agendas to boost innovation and scientific development. Data openness for reuse also resonates with the recognised need for more transparent, reproducible science. But what are scientists’ perceptions about data reuse? Renata Gonçalves Curty, Kevin Crowston, Alison Specht, Bruce W. Grant and Elizabeth D. Dalton make use of existing survey […]

Collaboration and concerted action are key to making open data a reality

The case for open data is increasingly inarguable. Improved data practice can help to address concerns about reproducibility and research integrity, reducing fraud and improving patient outcomes, for example. Research also shows good data practice can lead to improved productivity and increased citations. However, as Grace Baynes reports, recent survey data shows that while the research community recognises the value […]