The AI radiologist reads the referral note, not the scan

The referral note wins

A recent preprint stress-tests multimodal large language models on a hard radiology distinction: glioma versus brain abscess. The models show a text-driven span of 0.79 to 1.00 — meaning the referral text, not the MRI, drives the call. Model diagnostic accuracy lands between 0.52 and 0.73, inferior to human radiologist performance of 0.73 to 0.78 on the same task. The models are not reading the scan; they are reading the note about the scan. m:…0082

That inversion is the whole problem with bolting LLMs onto imaging — reasoning: the modality with the least information (a few lines of referral text) dominates the modality with the most (the scan itself). In a task where glioma and abscess demand different treatment, defaulting to the text's suggestion is not a rounding error; it is the model answering a different question than the one it was asked. m:…0082

Prompting can't fix it

The obvious patch fails. Prompt instructions telling the model to ignore the referral text are insufficient to eliminate the bias, and some models default to a tumor diagnosis. A bias that survives an explicit instruction to ignore its source is not a prompt problem; it is an architecture problem. If the text channel is present at inference time, the model leans on it. m:…0082

The default-to-tumor detail is the tell — reasoning: a model that hedges toward the scarier diagnosis is optimizing for something other than accuracy. In deployment, that hedge has costs: unnecessary biopsies, delayed treatment for abscess patients, and a false sense of AI assistance. Guardrails are not overhead on a clinical model; they are the product. m:…0082

The architecture answer: guardrailed agents

Enter MedMCP, an open agentic framework for medical imaging workflows proposed in a second preprint. It enforces a strict orchestration-execution boundary: LLMs handle workflow planning while domain-specific tools handle the actual image analysis. Agentic interactions are distilled into deterministic, shareable workflows for reproducibility, and the framework is designed for on-premise deployment with human-in-the-loop permission gates on every side-effecting operation. The LLM plans; it does not diagnose — which is exactly the separation the referral-text failure demands. m:…5040

NeuroPilot shows the same philosophy in neural imaging. A multimodal LLM coordinates computational imaging modules for closed-loop neural data processing: it restores motion-degraded recordings, supports consistent neural signal recovery in deep-brain and freely moving settings, and enables neuronal population tracking across spinal stimulation and 24-hour mPFC recordings aligned with sleep-wake behavior. The model orchestrates tools that do the seeing — the boundary MedMCP formalizes, applied to a different microscope. m:…2944

LLMs as evidence triage

A fourth preprint shows where LLMs do earn their place in the clinic-adjacent loop: triaging evidence. PESTO classifies gene-phenotype associations into four categories — Novel, Hypothesized, Existing, Established. In a study of 940 significant associations, 7% were classified Novel and 21% Hypothesized, and Hypothesized associations proved more likely than Novel ones to acquire human genetic support over time. Used as a classifier of prior evidence rather than a reader of scans, the LLM's text bias becomes a feature: it is judging text, which is what text is for. The lesson, stated as reasoning rather than result: keep the model on the side of the boundary where its bias is the job. m:…5441