Mmastodon TechnologyAI first seen 4 d ago, last 4 d ago, peak #3
Study Tests Measurement Error in AI-Assisted Literature Reviews
Original: "Mining Meaning: Measurement Error in AI-Assisted Literature Reviews" The paper compared three approaches: - Zero-shot:
A new paper examines how much measurement error creeps into AI-assisted literature reviews, comparing three approaches: zero-shot prompting, where the model receives instructions without labeled examples; retrieval-augmented generation (RAG), which supplies relevant examples from previously labeled papers; and supervised fine-tuning. Researchers are weighing which method yields more reliable coding of academic texts, and the comparison is drawing attention from scholars interested in using large language models for systematic review work.
Why now: Researchers are increasingly using large language models for literature reviews and want evidence on which method produces the most reliable results
AI-assisted literature reviewszero-shot learningRAGsupervised fine-tuning
Evidence
- "Mining Meaning: Measurement Error in AI-Assisted Literature Reviews" The paper compared three approaches: - Zero-shot: Give the model the paper and instructions, without labeled examples. - RAG: Give it relevant examples from previously labeled papers. - SFT (Supervised… · mottg@researchbuzz.masto.host · 2
API: https://socialmediatrends-api.osmike.com/v1/trends/2047684