October 20, 2026
New RoRI study evaluates two-year trial of Distributed Peer Review (DPR) at the Volkswagen Foundation
Study reveals high applicant satisfaction and streamlined individual reviewing, alongside limited overlap in final funding decisions between DPR and traditional panel review
A new working paper published by RoRI evaluates a two-year trial of Distributed Peer Review (DPR) conducted with the Volkswagen Foundation for their interdisciplinary funding program in the Social Sciences and Humanities. Implemented in parallel with the standard panel review process across the 2024 and 2025 calls, DPR requires applicants to a funding call to also serve as reviewers for other submissions to the same call.
Rather than relying solely on a small committee, the trial put evaluation directly into the hands of the applicants themselves. Across 301 proposals, researchers evaluated four or five competing submissions within their call. Every project received nine or ten independent reviews, with the highest and lowest scores trimmed to calculate a fair average, while a traditional panel reviewed the same pool of proposals in parallel to compare results.
The study found limited overlap in final funding decisions between the two methods, with just three overlapping selections out of 18 funded proposals in 2024, and one in 2025.
The evaluation highlighted several operational benefits of DPR, including spending less time reviewing and receiving feedback from diverse perspectives.
On average, DPR reviewers spent less than half the time per allocated proposal compared to traditional panel members.
Additionally, feedback from optional surveys showed strong willingness to participate in future DPR calls among both funded (82.8% in 2024; 94.5% in 2025) and unfunded respondents (59.6% in 2024; 71.7% in 2025).
The study also noted that disciplinary alignment between reviewers and proposals, as well as prior peer review experience, had minimal to no association with the scores awarded.
Across all stakeholders, concerns remained regarding reviewer expertise, applicant workload, and potential strategic scoring. Researchers observed moderate stability in funding decisions under DPR, estimating via simulations that ~56% of DPR-selected proposals would be re-selected if the process were repeated. DPR reviewers consistently emphasized ‘originality’ across both years, whereas panel reviewers’ emphasis on funding criteria varied substantially between phases.
This study was produced as part of RoRI’s AFIRE (Accelerator For Innovation & Research Funding Experimentation) project, in partnership with the Volkswagen Foundation. The working paper, Distributed Peer Review at the Volkswagen Foundation, is authored by Anna Butters, Melanie Benson Marshall, Stephen Pinfield, Tom Stafford, Alexander Bondarenko, Robert Nuske, Pierre Schwidlinski, and Hanna Denecke.
