Harnessing Large Language Models in Neonatal Intraventricular Hemorrhage: Exploring Retrieval Augmented Generation Methodology for Prognostic Variable Discovery.

In 30 seconds

This pilot study combined a systematic literature review with retrieval-augmented generation to assess whether GPT-4 and Claude Sonnet could identify and extract prognostic variables for intraventricular hemorrhage outcomes in preterm neonates. The models initially identified 39 studies; 28 met some or all validation criteria after hallucinated references were excluded. Across these studies, 14 predictors were identified in four outcome domains. Gestational age was mentioned 13 times (41%), birth weight 8 times (25%), and Apgar scores 11 times (34%). The authors also developed preliminary risk tiers, while emphasizing the need for human validation.

Key findings

• Of 39 studies initially identified by the LLMs, 28 met some or all validation criteria after hallucinated references were excluded.
• Fourteen prognostic predictors were extracted across mortality, progression, complications, and resolution outcomes.
• Gestational age was mentioned 13 times (41%), birth weight 8 times (25%), and Apgar scores 11 times (34%).
• The preliminary model estimated progression risk >70% and mortality >50% for neonates <28 weeks, <1,000g, and with Apgar <3.

Why it matters

The study suggests that retrieval-augmented LLMs may help organize published evidence and identify candidate prognostic variables for neonatal IVH. However, hallucinated references and incomplete synthesis show that automated extraction cannot be assumed reliable. Human review remains essential, particularly before using generated risk groupings to inform clinical decisions or developing decision-support tools.

What to keep in mind

This was a pilot study, and LLM outputs included hallucinated references and incomplete data synthesis. The authors also report no studies predicting resolution and limited investigation of complication predictors.