Category: metrics

Getting our hands dirty: why academics should design metrics and address the lack of transparency.

Metrics in academia are often an opaque mess, filled with biases and ill-judged assumptions that are used in overly deterministic ways. By getting involved with their design, academics can productively push metrics in a more transparent direction. Chris Elsden, Sebastian Mellor and Rob Comber introduce an example of designing metrics within their own institution. Using the metric of grant income, their tool ResViz shows […]

Human intuition is essential to science: Why metrics will not improve scientific governance

Scientists not only rely on knowledge reflected in textbooks and papers, but on their intuitions and experience. Answering scientific questions requires imagining what might be the case and then exploring it. Eric Giannella argues the uncertainty of science makes intuition and judgement essential. Yet the effect of metrics is to reduce the role of judgment. Even the most sophisticated set of metrics […]

Being trained by Twitter stats: Social media and the expanding ways we are measured in everyday life.

With the wider availability of real-time analytics on social media, what do all these metrics mean for the way that we interact with each other and for how we understand and judge ourselves? David Beer argues these developments could be seen as part of the broader metricisation of society. As we respond to performance metrics in our workplaces, so too we are now […]

Web analytics in the workplace: What Amazon and web newsrooms have in common – and where they differ.

The Politics of Data series continues with an investigation of how data-driven performance measurements operate in the workplace. What can we learn from the case of clicks in online news? Drawing from her ethnographic research shadowing web journalists, Angèle Christin illuminates the ambiguous and rather vague sets of meanings associated with clicks and web metrics that have become so omnipresent in online journalism. […]

Literacy as Numbers: The efficacy, merits and validity of transnational literacy assessment programmes.

Debates about the nature of literacy and how to account for the diversity of learning are far from resolved. A new book, Literacy as Numbers, looks at how literacy itself is being reframed around globalized assessment regimes. Camilla Addey delves into how these comparable numbers, now so heavily relied on in national policy, are produced, and how they are shaping our understanding of the meanings and purposes […]

Systems of measurement have a productive power in our lives

Metrics already perform a powerful productive role in the social world; they vindicate and limit, they cajole and incentivise, they legitimate and justify. When we reflect on how metrics are frequently used to manage performance, to facilitate competition, to judge us or to compare what we do with others, it is crucial that we see metrics as being central to the […]

Choosing the Right Criteria: Universities need to identify what behaviours should be valued and reward accordingly.

If universities are to thrive, evaluation criteria must capture and recognise exemplary contributions. To discontinue the legacy of poor indicators, Athene Donald calls for the focus on quality to be in more fundamental ways. For example, the mobility of an individual to accept overseas posts and attend international conferences has been frequently used as a proxy for excellence. Moving around […]

Against REFonomics: Quantification cannot satisfy the demands of rationality, equity and tolerability.

Academics are assured by government ministers and institutional heads that research assessment is designed on their behalf. Liz Morrish looks at whether the assessment tools created have extended their reach and left academics exposed. At its best, the REF distorts research agendas and priorities. However, a graver hazard is that a new selective and competitive academic will be formed, whose research […]

‘Frontier methods’ offer a powerful but accessible approach for measuring efficiency of public sector organisations

How can the efficiency of public sector organisations best be measured? Jesse Stroobants and Geert Bouckaert write that while the efficiency of an organisation is typically measured using performance indicators, there are some notable problems with this approach, such as the tendency for different indicators to produce conflicting conclusions on organisational performance. As an alternative, they outline so called ‘frontier methods’, which use direct […]

We need API use cases!

We need your help! We are beginning work on developing the DMPTool API (application programming interface) and need use cases. If you aren’t familiar with APIs (, I’ve attached a few slides to help explain the concept. But basically, it’s how you … Continue reading