Matthew
Narratives
GRAIL
Funder Data Platform
AGORRA
Undisciplined
Peer Review
Portfolios
MetaROR
AFIRE
ICOR public meeting focused on the growing significance of metascience for improving scientific research
This ICOR meeting focused on the growing significance of metascience for improving scientific research. As metascience becomes a recognised practice for evaluating scientific evidence and identifying potential biases, it’s important for stakeholders from different regions to collaborate on metascience projects and share expertise and resources that address common challenges. A unified approach to metascience can provide the mechanisms to help us learn from what we are doing and establish a continuous improvement cycle for implementing innovations in how science is performed. It will also provide trusted information to governments and policy makers to make more informed and evidence-based choices.
This meeting hosted four talks that provided real-life examples of the increasing influence of metascience as a systematic approach to understanding the practices, processes, and challenges across the academic research ecosystem.
The meeting sparked much conversation and what came across clearly is that metascience as a field is becoming a mainstay in understanding how to perform and evaluate research. We are now at an inflection point where metascience actors and enthusiasts are actively coalescing to ensure global participation and creating open spaces to collaborate and learn from each other.
Moumita Koley, DST- Centre for Policy Research, Indian Institute of Science [slides; streaming video 3-21 min]
Moumita described her work that focuses on generating evidence and advocating for change to research assessment in India. Her work is a part of project AGORRA, a global observatory of responsible research assessment that generates comparative data, evidence and analysis to support and accelerate change across national assessment systems. Moumita provided a detailed overview of the Indian context where quantitative metrics play the dominant role in research assessment. She then shared two case studies, the first on the misalignment between research outputs and disease burden in health research, and the second on evaluating the parameters used by the Indian National Institutional Ranking Framework (NIRF) to understand if they promoted a publish or perish culture and the consequences that has had on Indian research practices.
James Wilsdon, Research on Research Institute (RORI) [slides; streaming video 21-44 mins]
James started off by letting us know RORI had recently celebrated its 5th birthday and described how RORI’s mission to accelerate transformative research on research systems, cultures, and decision-making has evolved over time. James explained how the current set-up of the RORI team enables them to co-design and deliver projects that can respond quickly to strategic dilemmas and opportunities that partners identify, or are rising up in wider policy and scientific agendas. James then went on to highlight the newly formed UK metascience unit that aims to formalise the practice of using scientific methodology to study how research is done at the national level. James finished by showing how the global metascience community has grown and matured over time and that the next major gathering for the field will be in London at the Metascience 2025 conference (and welcomed the community’s participation).
Brian Nosek, Center for Open Science (COS) [slides; streaming video 44-65 mins]
Brian’s presentation built upon the gaining popularity of metascience and explained that metascience-engaged groups have distinct and complementary interests, which spurred on the idea to create the Metascience Alliance. The Metascience Alliance aims to be a trusted third party for metascience aligned and interested organizations and individuals with three primary objectives; community building, workforce development and matchmaking. Brian explained the initial vision is to create a light coordination to better foster collaboration, however the overall make-up and direction of the Metascience Alliance is still under scope. There is currently a process underway to assign a contractor as a Founding Program Manager for a 2 year pilot to get the ball rolling and start broader outreach and engagement with metascience stakeholders. More public information will be available soon, but if you are interested in finding out more now then feel free to reach out to Brian.
Kristen Ratan, Incentivising Collaborative and Open Research (ICOR) [slides; streaming video 65-82 mins]
Kristen’s presentation focus on ICOR’s work with Aligning Sciences Across Parkinson’s (ASAP) and how ICOR is now in a position to study the impact of ASAP’s Collaborative Research Network (CRN) and open science principles that were implemented in 2021. To do this ICOR have enlisted the University of Virginia’s metascience team, led by Alex Gates, to study data and ask questions that will help ASAP make decisions on their own policies but also provide real-world evidence for other funders and institutions looking to implement similar policies and practices. The study intends to answer important questions around the process and impact of implementing open science policies, the engagement of the CRN with open and collaborative research, and changes in researcher behaviour over time. ICOR will endeavour to share the results of this project as the work progresses.