Category: replication crisis

The overall incidence of published replication studies in economics is minuscule – greater incentives are required

Replicability is considered a hallmark of good scientific practice, an important post-publication quality check. But how many studies are chosen for replication? Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff, Gert G. Wagner have examined the economics literature and find that only one in one thousand publications are replication studies. The introduction of mandatory data disclosure policies may help to increase the […]

The growing, high-stakes audit culture within the academy has brought about a different kind of publishing crisis

The spate of high-profile cases of fraudulent publications has revealed a widening replication, or outright deception, crisis in the social sciences. To Marc Spooner, researchers “cooking up” findings and the deliberate faking of science is a result of extreme pressures to publish, brought about by an increasingly pervasive audit culture within the academy. By now most readers will have heard […]

It is advisor attitudes that are likely to shape students’ attitudes towards questionable research practices

In debates on the validity of academic research findings, focus has been drawn to so-called questionable research practices, commonly understood to encompass a laundry list of behaviours that can increase the likelihood of statistically significant (and so more publishable) results. Anand Krishna and Sebastian M. Peter report on research examining attitudes to questionable research practices among students who have recently […]

Replication is both possible and desirable in the humanities, just as it is in the sciences

Some scholars have claimed that replication – the independent repetition of an earlier study, answering the same study question, using the same or similar methods under the same or similar circumstances – is not possible in the humanities. The reasoning is that the humanities search for cultural meaning can yield multiple valid answers, and that research objects are people and […]

Contrary to common belief, randomised controlled trials inevitably produce biased results

Much of the social and medical sciences depend on randomised control trials. But while this may be considered the foundational experimental method, a certain degree of bias inevitably arises in any trial; whether this is sample bias, selection bias, or measurement bias. This is important as the level of validity of a trial’s causal claims can be a matter of […]

Journal data sharing policies are moving the scientific community towards greater openness but clearly more work remains

Data sharing is a key part of the drive towards greater openness in scientific research, allowing readers to reproduce and confirm an article’s findings, or even reuse its data as part of a new study. Many journals have policies requiring researchers to share their data in full, with PLOS being a forerunner in this area. But how effective has the […]

Software updates: the “unknown unknown” of the replication crisis

The replication crisis is largely concerned with known problems, such as the lack of replication standards, non-availability of data, or p-hacking. One hitherto unknown problem is the potential for software companies’ changes to the algorithms used for calculations to cause discrepancies between two sets of reported results. Anastasia Ershova and Gerald Schneider encountered this very problem in the course of […]