As Goodhart’s law states: ‘when a measure becomes a target, it ceases to be a good measure’. Using bibliometrics to measure and assess researchers has become increasingly common, but does implementing these policies therefore devalue the metrics they are based on? In this post Alberto Baccini, Giuseppe De Nicolao and Eugenio Petrovich, present evidence from a study of Italian researchers […]
Category: research assessment
What’s in a name? How false author affiliations are damaging academic research
When reading a research paper, can you be certain that the institution the author claims to be affiliated with is actually the institution that was responsible for supporting the research? In this post Vivienne C. Bachelet presents findings from a recent study suggesting that a significant proportion of author affiliations are unverifiable. Highlighting how a lack of editorial guidance in […]
The human element – why randomised control trials need mixed methods approaches
The applicability of Randomised Control Trials (RCTs) to the social sciences has been widely questioned. However, assessing the effectiveness of RCTs, in practice has proved difficult. In this post, Naila Kabeer reflects on her recent research into randomised control trials utilised as part of a programme to alleviate poverty in West Bengal and highlights how, when RCTs are used in […]
Grimpact – Time to acknowledge the dark side of the impact agenda
A critical blind spot in the impact agenda has been that impact is understood and defined solely in positive terms. In this post Gemma Derrick and Paul Benneworth introduce the concept of ‘Grimpact’, to describe instances where research negatively impacts society, and argue that the implicit optimism of research assessment has rendered researchers and science systems poorly equipped to deal […]
Assessing Impact Assessment – What can be learnt from Australia’s Engagement and Impact Assessment?
The impact agenda is an international and evolutionary phenomenon that has undergone numerous iterations. Discussing the development and recent release of the results of the Australian Engagement and Impact Assessment (EIA), Ksenia Sawczak considers the effectiveness of this latest exercise in impact assessment, finding it to provide an inadequate account of the impact of Australian research and ultimately a shaky evidence […]
Between value for money & development impact: Some reflections for the Global Challenges Research Fund
The Global Challenges Research Fund has engaged many researches with Overseas Development Aid and the auditing and assessment infrastructures associated with it. In this post Valeria Izzi and Becky Murray outline how researchers can adopt a value for money (VfM) approach that can justify North/South research projects in a way that accounts for economy, efficiency, effectiveness, as well equity. Since […]
The Impact Chain – How to craft an effective impact case study narrative
A vital part of any research assessment program is the ability to clearly demonstrate the impacts, whatever they may, of the research undertaken. In this post, Katy McEwan presents the impact chain approach for writing impact case studies. A method, which provides a framework for producing impact narratives and helps authors clearly describe and spotlight the consequences, intended and unintended, […]
Counting is not enough – How plain language statements could improve research assessment
Academic hiring and promotion committees and funding bodies often use publication lists as a shortcut to assessing the quality of applications. In this repost, Janet Hering argues that in order to avoid bias towards prestigious titles, plain language statements should become a standard feature of academic assessment. Let’s start with the obvious. Evaluation and assessment are part and parcel of the […]
REF2021: Adding Insult to Injury?
In this repost, Dr Liz Morrish responds to the recent guidelines issued for REF 2021. Highlighting potential unintended consequences and bad incentives, she argues that the ability of higher education institutions to enter staff into the REF who have been made redundant or removed from their positions, may lead to fewer opportunities and greater exploitation of already precariously employed academics. 57 days […]
REF2021: Adding Insult to Injury?
In this repost, Dr Liz Morrish responds to the recent guidelines issued for REF 2021. Highlighting potential unintended consequences and bad incentives, she argues that the ability of higher education institutions to enter staff into the REF who have been made redundant or removed from their positions, may lead to fewer opportunities and greater exploitation of already precariously employed academics. 57 days […]