BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Research on Research - ECPv6.15.18//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://researchonresearch.org
X-WR-CALDESC:Events for Research on Research
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:UTC
BEGIN:STANDARD
TZOFFSETFROM:+0000
TZOFFSETTO:+0000
TZNAME:UTC
DTSTART:20230101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=UTC:20241203T130000
DTEND;TZID=UTC:20241203T143000
DTSTAMP:20260407T123544
CREATED:20250806T110856Z
LAST-MODIFIED:20250806T111052Z
UID:2549-1733230800-1733236200@researchonresearch.org
SUMMARY:Utility of LLMs in identifying and assessing academic genres
DESCRIPTION:The European Network for Research Evaluation in the Social Sciences and the Humanities (ENRESSH) in partnership with the Research on Research Institute (RoRI) is proud to present the next webinar in its series on research evaluation as it is practiced across disciplines and countries. \n\n\n\nThis is the second event in a thematic line on AI in research assessment. This time we ask how large language models (LLMs) can support assessment of various academic genres. \n\n\n\n\n\n\n\n\n\nINTRODUCTIONResearch assessment as intertextual reading: Opportunities and challenges in the use of Artificial intelligence in evaluation of SSHSpecial advisor Dr Jon Holm\, Research Council of Norway \n\n\n\nSPEAKERSEvaluating social science\, arts and humanities journal article quality with ChatGPTProfessor Mike Thelwall\, University of Sheffield \n\n\n\nResearch quality evaluation of journal articles is time-consuming\, even for post-publication expert review tasks like national research evaluation exercises. It is also necessary to assess the strength of candidates’ works for recruitment and promotion. This talk assesses ChatGPT’s ability to score social science\, arts and humanities journal articles using papers and criteria from the UK’s Research Excellence Framework 2021. The systemic implications of using artificial intelligence to fully or partially replace human judgement for this core task will also be discussed. \n\n\n\nAdaptation of neural language models to different textual domains and the use of language models for comparative analysis of textDr Denis Newman-Griffis\, Senior Lecturer\, University of Sheffield\, and a Research Fellow of the Research on Research Institute \n\n\n\nSpeakers\n\n\n\nMike Thelwall (he) is a Professor of Data Science in the Information School at the University of Sheffield in the UK. He primarily investigates quantitative methods to support research evaluation\, including artificial intelligence\, citation analysis and altmetrics. He has recently shown that ChatGPT can provide useful research quality assessments for published journal articles. His books include: Quantitative Methods in Research Evaluation Citation Indicators\, Altmetrics\, and Artificial Intelligence (https://doi.org/10.48550/arXiv.2407.00135). He is an associate editor of the Journal of the Association for Information Science and Technology and sits on five other editorial boards. \n\n\n\nDenis Newman-Griffis (they/them) is a Senior Lecturer at the University of Sheffield Centre for Machine Intelligence and a British Academy Innovation Fellow. They lead the Research on Research Institute’s GRAIL project on Responsible AI and Machine Learning for research funding and evaluation\, and they are an active participant in Responsible AI policy discussions in research\, education\, and government. \n\n\n\nJon Holm (he) is a special advisor at the Research council of Norway where he works with the development of national research assessment in Norway and the use of AI in research evaluation and analysis. Jon and Denis have recently published on the potential and pitfalls of the use of AI in research financing: Holm et al. (2025). “Big Data for Big Investments: Making Responsible and Effective Use of Data Science and AI in Research Councils” in Nielsen et al (ed.) Artificial Intelligence and Evaluation\, Routledge 2025 (ch. 7)
URL:https://researchonresearch.org/event/utility-of-llms-in-identifying-and-assessing-academic-genres/
ATTACH;FMTTYPE=image/jpeg:https://researchonresearch.org/wp-content/uploads/2025/08/Chat-GPT.jpeg
END:VEVENT
END:VCALENDAR