What should we do with research ‘excellence’?

Over the last 20 years, the notion of ‘excellence’ has permeated almost every inch of the research ecosystem – from research funding schemes, evaluation frameworks to publishing decisions. Once believed to be a way to measure the best of the best, ‘excellence’ is now more likely to be viewed as too ambiguous, the source of undesirable behaviours and a barrier to an inclusive research culture.  

To dig into this, RoRI’s EXCELLENCE project is exploring how the concept of ‘excellence’ is defined and used when it comes to research funding and evaluation. The project has two parts: the first is an extensive literature review analysing how ‘excellence’ has evolved and been understood; and the second is an empirical study looking at the use of ‘excellence’ by funders.  

What did we do?

Focusing on research funding activities, we carried out an extensive literature review covering academic papers and other relevant sources.
From this collection, we:

• briefly outline how the notion of ‘excellence’ emerged and how it has been described in the academic literature 
• analyse how excellence-driven funding programmes shape institutions, projects and the way researchers work
• discuss themes around the consequences of ‘excellence’ 
• present three categories of mitigating strategies aimed at countering the negative consequences associated with ‘excellence’  

In academic literature, there’s a noticeable lack of data about how funding organisations use ideas of ‘excellence’. To help plug this gap, the second part of the project will provide detailed case studies and analysis of how funders use ‘excellence’ and what they’re doing to mitigate negative effects resulting from their approach.  

What did we find out about the use of ‘excellence’?

‘Excellence’ remains an ill-defined concept and since its creation has gained different meanings in different situations across research evaluation, funding and practices. There are examples of its use being associated with opposite effects; for example, the same excellence funding scheme being associated with both risk taking and risk aversion.

The notion of ‘excellence’ has been greatly shaped by North-American and Western-European contexts. Additionally, research into ‘excellence’ is also mostly done through a westernised lens. Despite aspiring to be an objective and global way to find the best of the best, uses of ‘excellence’ can encourage inequities in the research ecosystem.  

While the downsides of ‘excellence’ are widely talked about in the literature, alternatives are rarely offered. Whatever the solution, fixes should aim to question the underlying assumptions found within the ‘excellence’ regime. 

After ‘excellence’, what comes next? 

Suggestions to counter the negative effects of ‘excellence’ fall into three main categories: 

1) Patching

These are changes and initiatives that address visible problems in the research ecosystem, such as DORA and the Leiden Manifesto which reduce the overreliance on bibliometrics in funding decision making. Another is funders adjusting the size of the grant to the needs of different fields, instead of using a one-model-fits-all approach which can disadvantage certain disciplines like the humanities.

2) Pluralising

Pluralising is a way to expand the notion of ‘excellence’ – overcoming narrower interpretations. For example, this could mean taking into account and measuring the openness, use and impact of research alongside traditional published outputs. There’s also been a lot of effort to embed and enforce the link between equity, diversity and inclusion (EDI) and excellent research such as through the Tri-agency Statement on EDI in Canada.    

3) Transforming 

These ideas involve radical change. For example, replacing the big focus on measuring performance with something else or even abandoning the assessment of productivity or ‘excellence’ altogether. One idea raising interest is the move away from ‘zero-sum excellence’ towards ‘threshold excellence’. This is where funding is distributed among all those who meet the minimum criteria – a fixed target – using additional diverse criteria or lottery-style mechanisms to further narrow the number of applicants down. We’re investigating whether lottery-style funding mechanisms are a good idea in our RANDOMISATION project.  

What’s next?

We’re wrapping up the project’s second part which we’ll share later this year. Through a set of case studies, it will provide a detailed look at how funders use ’excellence’ and their issues and concerns of using it as a measure of research.

Read the full working paper and our commentary in the Times Higher Education. You can also  listen to the project team debate whether ‘excellence’ is still a useful concept during a recent Metascience session.