Funder Data Platform
RoRI Atlas of Peer Review
Peer review performs a vital and expanding role across the research system: in assessing funding proposals; registered reports (and other research designs and protocols); outputs (including journal articles, proceedings, chapters, books, preprints and datasets); and in contributing to research evaluation exercises, academic appointment and promotion processes.
In recent years, there has been increasing innovation in peer review, often in response to well-documented problems, such as system overload, bias, and lack of transparency. However, analyses of peer review have tended to focus on particular domains in isolation (e.g. purely on journals or funding proposals), instead of taking a more holistic approach to the question of how peer review might evolve and be improved. Opportunities to innovate and reform peer review and evaluation across different domains may not have been fully explored.
To help fill this gap, we will produce a RoRI Atlas of Peer Review and use this to generate a series of scenarios for the future of peer review. The Atlas will involve a systematic mapping of different domains, actors, objects, criteria, technologies, workflows, functions and economic models. The resulting scenarios will be designed to facilitate discussion between different groups of their possible roles in shaping future trajectories and possibilities for peer review.
Alongside the Atlas, this project will involve a live experiment in peer review, through the creation and launch of MetaROR (MetaResearch Open Review): a new publication and peer review platform. Developed in close partnership with our friends at the Association for Interdisciplinary Meta-research and Open Science (AIMOS) the MetaROR platform will take an innovative approach to peer review of research outputs in the field of research on research, or meta-research, and will enable the project to test many of the principles and approaches we identify through our broader analysis of the peer review landscape.
Findings from our pilot phase were also well summarised in an article for the LSE Impact blog on ‘four schools of thought in reforming peer review’.
Stephen Pinfield, Senior Research Fellow, RoRI and University of Sheffield
Ludo Waltman, Senior Research Fellow, RoRI and CWTS-Leiden
André Brasil, Research Fellow, RoRI and CWTS-Leiden
The MATTHEW project was initiated by CWTS-Leiden and Novo Nordisk Foundation. Its steering group is chaired by Gert V. Balling, Senior Impact Partner at Novo Nordisk Foundation and Co-Chair, RoRI Partnership Board.
The Peer Review working group includes the following RoRI partners:
Australian Research Council
Canadian Institutes of Health Research (CIHR)
FWF–Austrian Research Fund
UK Research and Innovation (UKRI)
To deepen our engagement with peer review in the scholarly publishing context, we also hope to partner with organisations and initiatives committed to innovation in this arena. Examples include: cOAlition S; Open Research Europe; and ASAPbio, as well as AIMOS and our other partners in the MetaROR platform.
This will be a two-year project, running to mid-2025. Its outcomes will include: