Category: metrics

Against metrics: how measuring performance by numbers backfires

A proliferation of companies, government agencies, and higher education institutions are in the grip of what Jerry Z. Muller has termed “metric fixation”. But by tying rewards to metrics, organisations risk incentivising gaming and encouraging behaviours that may be at odds with their larger purpose. The culture of short-termism engendered by metrics also impedes innovation and stifles the entrepreneurial element […]

The academic papers researchers regard as significant are not those that are highly cited

For many years, academia has relied on citation count as the main way to measure the impact or importance of research, informing metrics such as the Impact Factor and the h-index. But how well do these metrics actually align with researchers’ subjective evaluation of impact and significance? Rachel Borchardt and Matthew R. Hartings report on a study that compares researchers’ […]

Beyond Impact Factors: an Academy of Management report on measuring scholarly impact

What constitutes scholarly impact? And which stakeholders have importance for research? Usha Haley shares findings of a recent Academy of Management report that sought answers to these questions by surveying its 20,000 members and conducting a selection of in-depth interviews with prominent figures. A majority of respondents indicated journal rankings did not reflect scholarly impact, yet publications in top-tier journals […]

Where are we with responsible metrics? And where might we go next? Reflections from two recent events

Widespread scepticism and concern among researchers, universities, representative bodies and learned societies about the broader use of metrics in research assessment and management has led to concerted efforts to promote the “responsible use” of such metrics. But how effectively are UK higher education institutions engaging with this agenda? Lizzie Gadd reflects on two recent responsible metrics-themed events. While it is […]

From invisibility to impact: radically different measures are needed to capture the true impact of research

Academics are increasingly expected to produce directly applicable solutions to hard-to-solve “real-world” problems such as poverty, development, and environmental degradation. However, conventional assessments of science have not yet been adequately adapted to capture the diverse effects of this type of problem-centred research. Examining a prominent recent example of multidisciplinary research on consumption, environment and sustainability in Ireland, Henrike Rau, Gary […]

Analysing Altmetric data on research citations in policy literature – the case of the University of Sheffield

One of the sources of attention Altmetric.com tracks is the number of times research outputs have been cited in policy literature. Andy Tattersall and Chris Carroll explored the case of the University of Sheffield and what the data says about the impact of its research on national and international policy. The percentage of outputs with at least one policy mention […]

Quantity does matter as citation impact increases with productivity

Many scholars are encouraged to focus on the quality not the quantity of their publications, the rationale being that becoming too focused on productivity risks reducing the quality of one’s work. But is this, in fact, the case? Peter van den Besselaar and Ulf Sandström have studied a large sample of researchers and found that, while results vary by field, […]

Why has no other European country adopted the Research Excellence Framework?

Most European countries have followed the UK’s lead in developing performance-based research funding systems (PRFS) for their universities. However, what these countries have not done is adopt the same system, the Research Excellence Framework being its most recent iteration. Instead, many use indicators of institutional performance for funding decisions rather than panel evaluation and peer review. Gunnar Sivertsen has examined […]

2017 in review: top posts of the year

As 2017 nears its end and before our focus is drawn to whatever the new year might have in store, now is the perfect time to look back and reflect on the last twelve months on the Impact Blog. Editor Kieran Booluck reports on another year in which our readership has grown, and also shares a selection of the most […]