Predictive performance of machine learning models for bloodstream infection by validation type: A systematic review and meta-analysis.
Heejeong Kwon, Jaewoong Kim, Hyunok Yun et al.
✦ AI-curated · Sources linked
In 30 seconds
This systematic review and meta-analysis assessed the discrimination and methodological quality of machine-learning models predicting bloodstream infection from routine clinical data. Twenty-six studies were included, with 24 studies contributing 28 estimates to the pooled analysis. The pooled AUROC was 0.82 (95 % confidence interval, 0.77-0.86), but estimates varied markedly (I 2 = 99.9 %; 95 % prediction interval, 0.46-0.96). Discrimination was lower with external/temporal validation than with internal validation (0.77 vs 0.86; P = 0.005). Overall certainty of evidence was very low.
Key findings
- Across 24 studies and 28 estimates, pooled AUROC was 0.82 (95 % confidence interval, 0.77-0.86).
- Heterogeneity was extreme (I 2 = 99.9 %), and the 95 % prediction interval was 0.46-0.96.
- Discrimination declined from internal validation (0.86) to external/temporal validation (0.77; P = 0.005).
- Most studies (96 %) had high risk of bias in the PROBAST analysis domain; overall evidence certainty was very low.
Why it matters
The pooled discrimination suggests that machine-learning approaches may help identify patients at risk of bloodstream infection, but the wide prediction interval and poorer performance under more stringent validation indicate that results may not transfer reliably across settings. The very low certainty of evidence makes methodological quality and validation central to judging whether reported performance is clinically dependable.
What to keep in mind
The evidence was highly heterogeneous, most studies had high risk of bias in the PROBAST analysis domain, and overall certainty was very low.