Responsible research metrics?

The ongoing search for balance in research assessment.

Lucinda May - Research Services Manager, University of Manchester Library.

There is increasing recognition that current methods of research assessment can help to reinforce systemic inequities within research. Whether seeking to assess the performance of an individual research output, or an entire University, using narrow quantitative measures without accompanying qualitative analysis to inform decision-making carries considerable risks.

Though data, analysis and metrics are useful in the right context, ‘the current description, production, and use of these metrics are experimental and open to misunderstanding,’ warns Universities UK: ‘They can lead to negative effects and behaviours as well as positive ones.’ Potential negative effects are particularly concerning from the perspective of Equity, Diversity and Inclusion (EDI), with evidence of gendered and racial biases in research citation practices representing yet another example of systemic prejudices in academia, with deeply troubling consequences for researchers’ careers and wellbeing, and the wider world.

Comments from Deputy President and Deputy Vice-Chancellor Professor Luke Georgiou on the latest Academic Ranking of World Universities were notably measured, recognising these rankings’ limitations alongside their potential value. Dr. Elizabeth Gadd has highlighted not only the limitations of these methodologies, but also the uncomfortable ‘incompatibility between universities’ alignment with socially responsible practices and continued engagement with socially irresponsible ranking practices.’ 

Initiatives and calls to action

There have been a number of high-profile initiatives and calls to action in response to these issues in recent years. In 2016, Universities UK established a UK Forum for Responsible Metrics: a group of research funders, sector bodies, and infrastructure experts working in partnership to promote the responsible use of research metrics. Since 2019, the European Commission’s SUPER MoRRI project is exploring responsible research and innovation policies and practices, and their societal and economic impact; and in July 2022, the EC finalised an Agreement on Reforming Research Assessment to which over 350 research organisations have pledged commitment. The introduction of UKRI’s Enhancing Research Culture fund and The Wellcome Institute’s Open Access Policy requirement for demonstrable implementation of responsible metrics principles make it clear that research funders are keen to see real change within academic institutions.

Guidance materials

The University of Manchester Library has developed a suite of guidance materials to support researchers and research groups in responsible use of research metrics. I invite you to consult and share these webpages and openly available learning resources with your colleagues and networks: 

• Open Research position statement   
• Responsible research metrics  
• Part 1: Introduction to responsible research metrics 
• Part 2: Why and how research metrics are used
• Part 3: Using responsible research metrics and what happens when you don’t 
• Using SciVal and Altmetric Explorer to retrieve altmetrics for publications 

Members of the Library and Research and Business Engagement Services will be attending an upcoming workshop by the European University Association (EUA) and Universities UK (UUK) aiming to further raise awareness of this coalition and its Agreement on Reforming Research Assessment. We look forward to learning more as part of our ongoing work to embed responsible metrics principles into the University’s research activities.