For many years, academia has relied on citation count as the main way to measure the impact or importance of research, informing metrics such as the Impact Factor and the h-index. But how well do these metrics actually align with researchers’ subjective evaluation of impact and significance? Rachel Borchardt and Matthew R. Hartings report on a study that compares researchers’ […]
Category: research quality
Random audits could shift the incentive for researchers from quantity to quality
The drive to publish papers has created a hyper-competitive research environment in which researchers who take care to produce relatively few high-quality papers are out-competed by those who cut corners so their bibliometrics look good. Adrian Barnett suggests one way to push back against the pressure to “publish or perish” is to randomly audit a small proportion of researchers and […]
Looming REF deadlines lead to a rush in publication of lower quality research
The increased significance of research assessments and their implications for funding and career prospects has had a knock-on effect on academic publication patterns. Moqi Groen-Xu, Pedro A. Teixeira, Thomas Voigt and Bernhard Knapp report on research that reveals a marked increase in research productivity immediately prior to an evaluation deadline, which quickly reverses once the deadline has passed. Moreoever, the […]
Beyond Impact Factors: an Academy of Management report on measuring scholarly impact
What constitutes scholarly impact? And which stakeholders have importance for research? Usha Haley shares findings of a recent Academy of Management report that sought answers to these questions by surveying its 20,000 members and conducting a selection of in-depth interviews with prominent figures. A majority of respondents indicated journal rankings did not reflect scholarly impact, yet publications in top-tier journals […]
How funder pressures can torpedo the credibility of research – the cautionary tale of Google and New America
With policy recommendations subject to ever greater scrutiny – not only of their viability but also the credibility of their sources of expertise – many think tanks and research institutions invest considerable time and effort into building and nurturing a reputation for research quality and intellectual independence. However, this most valuable asset remains extremely precarious. Till Bruckner recounts the recent […]
Does high-impact research come at the expense of quality? An automated analysis of the REF impact landscape
As part of the 2014 Research Excellence Framework, the impact of research was assessed for the first time. But how effective was the impact category in capturing the many diverse forms of interaction between academia and society? Were certain interpretations of impact more highly rewarded than others? And does high-impact research come at the expense of quality? Martin Zaltz Austwick […]
Why the ban on P-Values? Understanding sampling error is key to improving the quality of research.
The weight placed on p-values and significance testing has come under increasing criticism, with one social psychology journal banning their use entirely. Nicole Radziwill argues that many of the issues come down to sampling errors. Inferential statistics is good because it lets us make decisions about a whole population based on one sample. But inferential statistics is bad if your sample size is too […]