Find what you're looking for

Popular Pages

Grail

Getting responsible about AI and machine learning in research funding
and evaluation

Summary

A recent UK funder study of potential uses of AI in national research evaluation sparked debate in the journal Nature.

The GRAIL project is exploring good principles and practices for using AI and machine learning in the research funding ecosystem in ways that are both ethical and effective. The project aims to create an inter-funder community of learning around opportunities, challenges, and facilitators for using AI/ML in research funding and evaluation, and to use funder insights and experiences to explore what more grounded use of AI in their settings looks like. To inform future actions and use of AI/ML, the project will characterise current approaches to and use of AI within research funding and develop specific recommendations for strategies to manage for social and organisational impact of AI usage within funding decision-making and assessment.

The opportunities presented by AI and ML technologies—and the dilemmas and uncertainties that accompany them—are the focus of intense debate in almost every sector, including research. There have been some specific calls to build these technologies into research management and evaluation, and there are now a range of small-scale pilots underway among funders and publishers internationally. However, there is a lack of evidence or guidance on organisational, team, or individual best practices for how to responsibly and effectively integrate AI and ML technologies into decision-making processes in a research funding and evaluation context.

The GRAIL project aims to support funders in the responsible design, use, and evaluation of AI tools through community and mutual learning, organisational insights, and practical guidelines. There are three main strands to the project:

  1. Working group. The foundation of the project is a cross-funder working group, with monthly sessions oriented around knowledge exchange, collaborative resource development, and group investigation of shared challenges. These sessions will be supplemented with two more targeted efforts:
  1. Understanding current practices. Conducting survey-based analysis with follow-on interviews to understand past experiences and current practices, and map the landscape of opportunities and challenges that funders are already facing with AI tools;
  1. Developing new practices. Collaborative guidelines for designing, evaluating and managing AI-enabled processes in a research funding context.

Together, these strands will improve our shared base of knowledge and evidence to enable more effective and responsible use of AI and ML tools in research funding

Project team

Denis Newman-Griffis, University of Sheffield & Research Fellow, RoRI 

Partners and steering group

The GRAIL project steering group is co-chaired by Jon Holm (Research Council of Norway); Katrin Milzow (Swiss National Science Foundation); and Gustav Petersson (Swedish Research Council).

Project partners include: 

  • Austrian Science Foundation (FWF)
  • Australian Research Council (ARC)
  • CWTS-Leiden
  • Novo Nordisk Foundation (NNF), 
  • Research Council of Norway (RCN)
  • Research England/UKRI (RE)
  • Swiss National Science Foundation (SNSF)
  • Volkswagen Foundation (VWF)
  • Wellcome Trust (WT)

Project Lead

Timeline and outputs

Credit: RoRI-RCN working paper, December 2022

The GRAIL project will run for 24-months until mid-2025. It builds on an initial workshop series co-hosted by RoRI and Research Council of Norway in January 2021 (later summarised in this RoRI/RCN working paper). 

The GRAIL project will generate the following outputs:

(1) knowledge-sharing events
A mixture of public-facing, partner-only and conference events will be used to inform ongoing GRAIL discussions and disseminate findings to relevant audiences. Proposed events include:

  • A special panel at the STI 2023 conference focused on highlighting key questions, challenges, and needed actions in the use of AI and ML technologies by research funding organisations.
  • Concluding workshop for project participants. To be hosted by Research Council of Norway (RCN), wth the aim of sharing insights and developments from the project, building a forward-looking community among partners (spring-summer 2025)

(2) publications
A mix of working papers, academic articles and commentaries, including:

  • Working paper and article on survey findings. This will give a primarily quantitative overview of the range of engagement levels/strategies with AI/ML currently exhibited by an international sample of research funding organisations across disciplines. 
  • Working paper and article on interview findings (Arm 2). This will complement the first article by giving a more in-depth qualitative discussion of the challenges and opportunities experienced by funder staff engaged with AI/ML work within their organisations, and the practices used to manage these. 

RoRI report summarising the headline findings of the project (spring 2025), including a set of guiding questions for formulating/designing AI projects in research funding settings.

Related Projects