Category: research evaluation

Open peer review: bringing transparency, accountability, and inclusivity to the peer review process

Open peer review is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation. Tony Ross-Hellauer provides an overview of work conducted as part of an OpenAIRE2020 project to offer clarity on OPR, and issues an open call to publishers and researchers interested in OPR to come together to share […]

We have the technology to save peer review – now it is up to our communities to implement it

Today marks the beginning of Peer Review Week 2017. Here on the Impact Blog, we’ll be featuring posts covering a variety of perspectives on and issues relating to peer review, and which also consider this year’s theme of “Transparency”. To kick things off, Jon Tennant, Daniel Graziotin and Sarah Kearns consider what can be done to address the various shortcomings […]

Collaborative researcher behaviour has not (yet) increased in response to incentive regimes’ performance measures

A somewhat cynical view of researcher motivations suggests that, when faced with new quantitative performance measures as part of their local incentive regimes, researchers will quickly modify their behaviours in an effort to optimise their own performance. Charlotte Wien, Bertil F. Dorch and Asger Væring Larsen set about testing this notion, looking more closely at how their own Danish system […]

Rather than promoting economic value, evaluation can be reclaimed by universities to combat its misuse and negative impacts

To critics across higher education, evaluation frameworks such as the REF and TEF represent mechanisms of control, the generation of a “target and terror” culture. Deirdre Duffy explains how the REF and TEF resonate most closely with impact evaluation, a form of evaluation that can prove useful for a simple cost-benefit analysis but can also be problematic as it encourages […]

Book Review: Research and Evaluation for Busy Students and Practitioners: A Time-Saving Guide (2nd Ed.) by Helen Kara

In this new second edition of Research and Evaluation for Busy Students and Practitioners: A Time-Saving Guide, Helen Kara offers a book for students, researchers and practitioners looking to manage their time effectively and maintain a good work-life balance whilst undertaking methodologically and ethically robust social research and evaluation projects. This is a well-written and clear guide that will trigger self-reflection and boost confidence, […]

Increasing REF’s impact weighting could offer incentive for institutions to address societal, economic and global challenges

Challenges posed by events such as Brexit highlight the importance of excellent research programmes. Moreover, they represent a broader context in which the next Research Excellence Framework must consider ‘impact’. But do current REF proposals go far enough towards doing this? Matthew Guest argues that there is not enough of an incentive for institutions to address heightened societal, economic and […]

The importance of being REF-able: academic writing under pressure from a culture of counting

Writing is crucial to an academic’s role of producing, shaping and distributing knowledge. However, academic writing itself is increasingly being shaped by the contemporary university’s managerial practices and evaluation frameworks. Sharon McCulloch describes how her research on academics’ writing practices has revealed tensions around the ways in which managerial practices interact with academics’ individual career goals, disciplinary values and sense […]

Embedding open science practices within evaluation systems can promote research that meets societal needs in developing countries

Researchers’ choices are inevitably affected by assessment systems. This often means pursuing publication in a high-impact journal and topics that appeal to the international scientific community. For researchers from developing countries, this often also means focusing on other countries or choosing one aspect of their own country that has such international appeal. Consequently, researchers’ activities can become dislocated from the […]

Mendeley reader counts offer early evidence of the scholarly impact of academic articles

Although the use of citation counts as indicators of scholarly impact has well-documented limitations, it does offer insight into what articles are read and valued. However, one major disadvantage of citation counts is that they are slow to accumulate. Mike Thelwall has examined reader counts from Mendeley, the academic reference manager, and found them to be a useful source of early […]