An AI agent matched pediatric emergency doctors on diagnosis. Now it has to show its work
In new preprints, an AI agent matched pediatric emergency physicians on diagnosis in 95 percent of cases, four-model consensus lifted hypothesis matching by 23 points, and an agentic pathology tool far outperformed a trained baseline — while a knowledge-graph project tackles the evidence underneath.
- In a preprint, an AI agent called Dr CaBot and attending pediatric emergency physicians both produced high-scoring differential diagnoses in 95 percent of cases at the initial evaluation, with no statistically significant difference between them, in a retrospective cohort of 100 children, 41 percent of whom had complex chronic conditions. Self-reported m:…5112
- In a preprint, unanimous selection by four language models increased the match rate of generated hypotheses to published findings. Self-reported m:…7456
- In a preprint, an agentic pathology framework called ARIPA reached an F1 score of 0.9849 for nuclear segmentation in patient-derived organoids, against 0.4713 for a custom-trained HoVer-Net baseline, without retraining or fine-tuning any model. Self-reported m:…0752
- In a preprint, a framework called SPIDER classified 79.0 percent of potentially conflicting biomedical relations as differences of experimental context rather than genuine disagreement, and added 2.31 million literature-supported entity pairs to a knowledge graph built from 48 databases. Self-reported m:…0010
The head-to-head result
The headline result comes from a pediatric emergency department, where researchers tested an AI agent called Dr CaBot against attending pediatric emergency physicians on differential diagnosis for acutely ill and injured children. In a retrospective cohort of 100 patients, 41 percent of whom had complex chronic conditions, both the agent and the physicians produced high-scoring differential diagnoses in 95 percent of cases at the initial physician evaluation touchpoint. Across the emergency department encounter, the study found no statistically significant difference in diagnostic accuracy between the two. m:…5112
That is a parity result, not a victory, and the authors are careful about what it does and does not show. The study was retrospective, so the agent was tested on records of cases that had already been worked up, and the authors say prospective validation is required before any clinical use. But the setting matters: acutely ill children with complex chronic conditions are among the hardest cases to diagnose, and a diagnostic aid that keeps up in that setting is a different proposition from one that performs well on textbook questions. m:…5112
Consensus beats a single answer
A second preprint suggests one reason these systems are getting harder to dismiss: they do better when they have to agree with each other. Researchers tested four large language models on generating and selecting hypotheses that match published findings. When all four models unanimously selected a hypothesis, the match rate to published findings rose. First-listed hypotheses were already more likely to match than those in positions two to five, but the unanimous pipeline did better still. m:…7456
The study also tested a method called Omega Point, which generates hypotheses without a literature search, and found it performed significantly worse than standard single-request or multi-step pipelines. The lesson is unglamorous but useful: for research hypotheses, as for diagnoses, the models improve when they are forced to check their work against the literature and against each other, and they get worse when they are allowed to free-associate. m:…7456
An agent that orchestrates instead of retraining
A third preprint, ARIPA, applies the same agentic idea to pathology images. Rather than training a new model for nuclear segmentation in patient-derived organoids, it uses vision-language models to plan and coordinate existing segmentation tools, with a human expert in the loop to refine the results. It reached an image-macro F1 score of 0.9849 after expert-guided refinement, compared with 0.4713 for a custom-trained HoVer-Net baseline — a large gap, achieved without retraining or fine-tuning any network. m:…0752
The human-in-the-loop design did real work here: expert-guided refinement raised the mean paired F1 score by 0.1468 across 19 images. That matters because pathology images vary in morphology and staining in ways that break fixed models, and the authors' approach treats the expert as part of the system rather than as a fallback. It also means the result depends on expert time, which is the scarcest resource in the workflow it is meant to help. m:…0752
The evidence underneath
The fourth preprint addresses the layer underneath all three: the knowledge these systems reason over. SPIDER builds a biomedical knowledge graph grounded in specific literature evidence, integrating 48 databases to add 2.31 million literature-supported entity pairs. Its multi-agent framework classifies whether conflicting relations in databases are genuine scientific disagreement or simply different experimental contexts — and found that 79.0 percent of potentially incompatible relations were contextual differences, with only 6.1 percent genuine disagreement. m:…0010
The practical payoff, the authors report, is that the evidence-linked graph improved language model performance on biomedical reasoning benchmarks. That is the thread connecting all four papers: the field is moving from asking whether an AI can produce a plausible answer to asking whether it can produce an answer, check it against evidence, and show where the evidence came from. The pediatric study shows the first part is now in reach; SPIDER is an attempt at the second. m:…0010
What to watch
All four studies are preprints, and none of them is a clinical deployment. The pediatric result needs prospective testing before it means anything at a bedside, the hypothesis benchmark covers ageing biology only, and ARIPA's advantage depends on expert refinement that has to be paid for in time. But the direction is consistent: agreement between models, human experts in the loop, and evidence attached to every claim. The next studies worth watching are the ones that test whether those three habits survive contact with real clinical workflows, where the cases are messier than a retrospective cohort and the experts are busier than a study protocol assumes. m:…5112 m:…7456 m:…0752 m:…0010
What is still unresolved
- Whether the pediatric diagnostic agent's parity with physicians holds in a prospective study rather than a retrospective cohort.
- Whether unanimous multi-model selection generalises beyond ageing biology to other research fields.
- Whether ARIPA's human-in-the-loop gains are worth the expert time they require in a routine pathology workflow.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.