Researchers optimistic about Distributed Peer Review

For the call “Open Up – New Research Spaces for the Humanities and Cultural Studies”, RoRI partner the Volkswagen Foundation is trialling a new form of peer review in which all applicants to the scheme are also reviewers of proposals.

The ambition is to investigate ways in which peer review of applications for research funding can be made fairer, faster and more robust. Volkswagen Foundation is working with Professors Tom Stafford and Stephen Pinfield from RoRI to ensure effective evaluation and reporting of the trial.

The deadline for applications was in April, and around 140 proposals were received (an increase on previous years), suggesting applicants were not deterred by the addition of the distributed peer review element.

A survey of applicants revealed positive expectations of distributed peer review:

• 85% find the distributed peer review process suitable.
• 77% believe it will identify appropriate reviewers.
• 74% trust it to be fair and to fund the best research.
• 70%  believe the process will select more adventurous proposals than the conventional panel review.

The disciplinary spread of applications was across the full range of the humanities and social sciences.

Applicants will be reviewing each others’ proposals this summer – with appropriate mechanisms in place to handle conflicts of interest and prevent gaming of the system. Awards will be announced in December. 

The Research on Research Institute team will conduct a thorough evaluation, combining quantitative analysis of the process with in-depth interviews with applicants and awardees, so the full report can provide evidence on how well the distributed peer review process has met its ambitions.

Volkswagen Foundation is generously funding projects selected through either the classic panel review process or the distributed peer review process, meaning that applicants have two opportunities to be selected for funding. This also supports direct comparison of the potential advantages of distributed peer review.

For more information on the experiment, read this interview with Dr. Hanna Denecke, team leader in the “Exploration” profile area at the Volkswagen Foundation.