Matthew
Narratives
GRAIL
Funder Data Platform
AGORRA
Undisciplined
Peer Review
Portfolios
MetaROR
AFIRE
Launched in September 2019, the Research on Research Institute (RoRI) is an international consortium of research funders, academics, research organizations and technologists which aims to champion transformative research on research systems, concepts, cultures, careers and decision-making. Following an intense period of consultation and co-design, we are excited to unveil our first wave of projects, which will run for the remainder of RoRI’s pilot phase (until autumn 2021).
We are also delighted to announce our partners, who will be collaborating in the design and delivery of these projects. RoRI’s consortium now includes 20 partners, drawn from 12 countries or regions:
*founding member of the consortium
James Wilsdon, Director of RoRI, said: “We are hugely grateful to all RoRI’s partners, who share a commitment to using evidence, data and shared learning to improve how we support and evaluate research. Between them, our partners invest almost US$20 billion each year in research, so by supporting and conducting real-time experimentation, data-gathering and analysis, we have real opportunities to strengthen research cultures and decision making. These agendas will be more important than ever as we chart pathways out of the Covid-19 pandemic, which has forced rapid innovations worldwide in research funding, evaluation, peer review and communication. We need to draw evidence and lessons from the crisis that can strengthen the diversity, preparedness and resilience of research systems over the longer term.“
Our first wave of projects will each involve clusters of between five and fifteen RoRI partners, working together to design and deliver a commonly-agreed programme of work. The projects are:
-CRITERIA: How do grant application criteria influence inequalities in research funding?Whether they use application forms, interviews, metrics, internal review, external review or a lottery, funders want their selection processes to efficiently select those proposals most likely to meet their objectives. Despite recognition of the importance of diversity, persistent inequalities in funding are observed across many settings. This project will use the RoRI Funder Data Platform to: a) examine evidence for funding inequalities in terms of gender and field bias; and b) test different explanations for the presence or absence of these inequalities, using a mixture of qualitative and quantitative data. The findings will help funders understand potential drivers of inequalities in funding and identify where mitigation is possible through changes to selection processes.
-FAIRware: An open source software tool to assess the FAIR status of research outputsThis project aims to build open source software tool(s) to allow researchers, institutions and funders to assess and improve the ‘FAIRness’ of the research outputs they produce. Over recent years, the FAIR principles (Findability, Accessibility, Interoperability, Reusability) have gained considerable traction as a basis for describing how research data, and potentially other research outputs, should be documented and shared to ensure that they can be discovered, accessed and used effectively, such that their value is maximised.
-PATHWAYS: a six-country, multi-partner study of data, frameworks & future directions for career pathways in researchThis project will explore career pathways in research, broadly defined, with an empirical and policy focus on six countries: Austria, Canada, Denmark, Germany, UK and USA. The project team will be drawn from RoRI partners in these countries, and a wider network of data, research and policy partners. It will follow a modular structure, conducted via desk analysis and online meetings: Phase 1 will explore the data landscape, and identify opportunities to improve quality and access to existing data; Phase 2 will identify how collaboration in definition, generation and monitoring of career data can inform research policy and management, and support more diverse and inclusive research cultures; Phase 3 will culminate in a global online conference on ‘reforming the postdoctoral experience’ accompanied by a detailed report.
-RANDOMISATION: Focal randomisation in grant funding – a multi-funder studyInterest is growing in uses of randomisation (or lottery mechanisms) in grant funding, and a handful of funders are now undertaking experiments at various scales. By supporting RoRI partners in a series of trials and facilitating alignment & learning across these, the consortium aims to undertake the largest analysis of such techniques to date, It will explore effects on bias, burden, legitimacy of process; and longer-term outcomes. This will be the first stage in a wider series of funder experiments, designed to test and evaluate different modes of allocation and evaluation, and their relationship to bias, diversity and inclusion.
-TRANSFORMING ‘EXCELLENCE’: definitions, applications & alternatives to excellence in the evaluation, management and measurement of researchInitiatives like the UK’s REF, Germany’s Exzellenzinitiative and Switzerland’s Eccellenza grants have put excellence at the centre of research policy and evaluation. However, the concept remains ambiguous, and is often operationalised through reliance on a small number of metrics. Critics argue that it creates perverse incentives in research, and some have called for the concept to be pluralised, taking account of additional evidence and indicators, such as non-academic impact or openness. Others have argued for scrapping the concept entirely, as little more than a buzzword. This project will assess ways in which the concept of excellence is currently being used by RoRI partners and other key actors in research systems, and how these definitions and uses are changing and can be improved.