Some scholars have claimed that replication – the independent repetition of an earlier study, answering the same study question, using the same or similar methods under the same or similar circumstances – is not possible in the humanities. The reasoning is that the humanities search for cultural meaning can yield multiple valid answers, and that research objects are people and […]
Category: replication
Contrary to common belief, randomised controlled trials inevitably produce biased results
Much of the social and medical sciences depend on randomised control trials. But while this may be considered the foundational experimental method, a certain degree of bias inevitably arises in any trial; whether this is sample bias, selection bias, or measurement bias. This is important as the level of validity of a trial’s causal claims can be a matter of […]
Journal data sharing policies are moving the scientific community towards greater openness but clearly more work remains
Data sharing is a key part of the drive towards greater openness in scientific research, allowing readers to reproduce and confirm an article’s findings, or even reuse its data as part of a new study. Many journals have policies requiring researchers to share their data in full, with PLOS being a forerunner in this area. But how effective has the […]
Software updates: the “unknown unknown” of the replication crisis
The replication crisis is largely concerned with known problems, such as the lack of replication standards, non-availability of data, or p-hacking. One hitherto unknown problem is the potential for software companies’ changes to the algorithms used for calculations to cause discrepancies between two sets of reported results. Anastasia Ershova and Gerald Schneider encountered this very problem in the course of […]
Collaboration and concerted action are key to making open data a reality
The case for open data is increasingly inarguable. Improved data practice can help to address concerns about reproducibility and research integrity, reducing fraud and improving patient outcomes, for example. Research also shows good data practice can lead to improved productivity and increased citations. However, as Grace Baynes reports, recent survey data shows that while the research community recognises the value […]
To understand the replication crisis, imagine a world in which everything was published.
Countering the claim that failed replications merely reflect the underlying truth of the scientific process and are not a matter of concern, Andrew Gelman argues that actually, the process is largely broken and we are in danger of dismissing the value of replication efforts. Current practice, centered on publication thresholds, is not filtering out poorly designed and executed studies. John Snow points me […]
Incentives for open science: New prizes to encourage research integrity and transparency in social science.
The high-profile political science study on same-sex marriage views, now determined to be fraudulent, is the latest case exposing the failure of incentive structures in the academy. The academic community must strengthen research evidence and do more to promote transparency. Temina Madon shares the launch of prizes run by the Berkeley Initiative for Transparency in the Social Sciences (BITSS) that look to provide recognition, visibility and […]
Why perpetuate a 300-year-old anachronism? Reincarnating the research article into a ‘living document’.
Online publication provides us with new freedom to update, amend and extend the research article as we know it. Daniel Shanahan presents a vision of the evolution of the article beyond the limits of the printed page. Creating a living document for a single research project, updated in real time, would lead to it being evaluated based on the question […]
Replication is not about making or breaking careers: it is about providing an opportunity to move science forward.
Replication and closer scrutiny of published findings are generally welcome in the scientific community, but questions have been raised over how replication attempts are being reported. Whilst there are certainly arguments for more friendly and cooperative tones to scientific debate, Dorothy Bishop welcomes this next chapter in rigorous debate. Reputation and career prospects will, at the end of the day, come […]
A replicated study on nuclear proliferation shows the critical necessity of reviewing accepted scientific results.
In replicating a 2009 study on the role of asymmetric nuclear weapons possession, Mark Bell and Nicholas Miller found that a computational error led to the overestimation of the deterrent effect of nuclear weapons by a factor of several million. It is … Continue reading →