Replication

Monday 12th October 3-4pm GMT

Part of the ECR Research on Research Seminar Series.

(When) Do Replications Influence the Impact of Original Claims? A Bibliometric Investigation

Josefina Weinerova

Replication is a cornerstone of cumulative science, providing essential insights into the robustness and generalisability of research findings. Yet, despite an increasing number of published replications, they often fail to affect the belief in, and use of, the original conclusions (Hardwicke et al., 2021). This raises a critical question: If replications are fundamental to scientific self-correction, why do their results so often fail to shape knowledge?

In this talk, I will present preliminary results of a bibliometric study of the the effect of failed replication on citations of original studies. We used studies from the FORRT Library of Replication Attempts (FLoRA), which includes ~800 failed replications from across the social and behavioural sciences, and control studies published in the same year and similar journals. We test whether failure to replicate leads to lower citations of the original study and also which moderators have an impact on this effect (e.g., publication status of the replication study, open-access status, journal rank, author composition, research field). Understanding what makes some replications more influential than others can inform strategies to increase their visibility and impact.

Josefina Weinerova is a postdoctoral researcher at Birkbeck, University of London, where she works on UKRI funded project “Making Replications Count” in collaboration with Dr Lukas Wallrich and Dr Lukas Röseler. She finished her PhD in Psychology at University of Nottingham in February 2026, during which she studied the effect of Covid-19 on cognitive and brain measures.

Do failed replications make their way into public knowledge?

Youyou Wu

Using FORRT’s replication database, we tracked research findings with well-established failed replications to examine whether public knowledge sources had updated their content. We examined three traditional knowledge channels: Wikipedia, university textbooks, and course syllabi. We also queried two AI systems, Google AI Mode and ChatGPT, about the original findings to assess whether their responses reflected the replication evidence. Our results suggest that AI systems represent a promising new pathway for communicating replication evidence to the public. They also highlight the value of FORRT’s curation of replication evidence in enabling the dissemination of replication findings across knowledge platforms.

Youyou Wu is an Associate Professor of Psychology at University College London, based in the Department of Psychology and Human Development at IOE. Trained as a psychologist, her research applies computational methods, including machine learning and natural language processing, to the study of individual differences such as personality. More recently, her work has expanded into metascience, focusing on how AI can improve research systems. She joined the Research on Research Institute (RoRI) in 2024 as part of the GRAIL project, developing best practices for the use of AI in research funding. She also contributes to the AFIRE project, which helps funders design AI-related experiments. In 2025, she was awarded the UKRI Metascience AI Early Career Fellowship to study the influence of generative AI on the language of science. 

Register at https://us02web.zoom.us/meeting/register/jpxu37ViR9SFPC-wRn550Q