Category: research evaluation

Research assessments based on journal rankings systematically marginalise knowledge from certain regions and subjects

Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego Chavarro, Ismael Ràfols and colleagues shows how such systems underestimate and prove detrimental to the production of research relevant to important social, economic, and environmental issues. These systems also reflect the biases […]

Government policies favouring research for economic returns can overlook existing strengths in arts and humanities

There is an argument that the best way for governments to allocate resources for research is to prioritise those areas most likely to deliver economic returns. Andrew Gibson and Ellen Hazelkorn explain how, shortly after its Great Recession, Ireland prioritised research fields aligned with industrial sectors rather than disciplinary excellence or societal challenges. By starting with an orientation toward the […]

Clickbait and impact: how academia has been hacked

It has become increasingly clear that prevailing academic incentive structures have a potentially damaging and distorting effect on the nature of academic debates. Portia Roelofs and Max Gallien use the example of a controversial recent journal publication to illustrate how deliberately provocative articles have the capacity to hack academia, to privilege clicks and attention over rigour in research. This is […]

Peer review processes risk stifling creativity and limiting opportunities for game-changing scientific discoveries

Today, academics must prepare written proposals describing the research they wish to conduct and submit them to funding agencies for evaluation – a process known as peer review. According to Don Braben and Rod Dowler, the current peer review process actually serves as a blocker to more radical research, stifling creativity and limiting opportunities for game-changing discoveries. Obviously peer review should not be abandoned entirely, but it […]

Open peer review: bringing transparency, accountability, and inclusivity to the peer review process

Open peer review is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation. Tony Ross-Hellauer provides an overview of work conducted as part of an OpenAIRE2020 project to offer clarity on OPR, and issues an open call to publishers and researchers interested in OPR to come together to share […]

We have the technology to save peer review – now it is up to our communities to implement it

Today marks the beginning of Peer Review Week 2017. Here on the Impact Blog, we’ll be featuring posts covering a variety of perspectives on and issues relating to peer review, and which also consider this year’s theme of “Transparency”. To kick things off, Jon Tennant, Daniel Graziotin and Sarah Kearns consider what can be done to address the various shortcomings […]

Collaborative researcher behaviour has not (yet) increased in response to incentive regimes’ performance measures

A somewhat cynical view of researcher motivations suggests that, when faced with new quantitative performance measures as part of their local incentive regimes, researchers will quickly modify their behaviours in an effort to optimise their own performance. Charlotte Wien, Bertil F. Dorch and Asger Væring Larsen set about testing this notion, looking more closely at how their own Danish system […]

Rather than promoting economic value, evaluation can be reclaimed by universities to combat its misuse and negative impacts

To critics across higher education, evaluation frameworks such as the REF and TEF represent mechanisms of control, the generation of a “target and terror” culture. Deirdre Duffy explains how the REF and TEF resonate most closely with impact evaluation, a form of evaluation that can prove useful for a simple cost-benefit analysis but can also be problematic as it encourages […]

Book Review: Research and Evaluation for Busy Students and Practitioners: A Time-Saving Guide (2nd Ed.) by Helen Kara

In this new second edition of Research and Evaluation for Busy Students and Practitioners: A Time-Saving Guide, Helen Kara offers a book for students, researchers and practitioners looking to manage their time effectively and maintain a good work-life balance whilst undertaking methodologically and ethically robust social research and evaluation projects. This is a well-written and clear guide that will trigger self-reflection and boost confidence, […]