Clinical AI's honesty week: a null result, an agent, and a real trial

Lead with the null result

The most important clinical AI paper this week is the one that found nothing. In a preprint, Pozza and colleagues tested whether adding ECG, chest X-ray and clinical-note data to structured electronic health records improves prediction of 30-day COPD readmission — and across multiple fusion architectures, the answer was no. Structured EHR data alone, at an ROC-AUC of 0.63, remained the most robust predictor after the authors corrected for overconfidence and recalibrated the fusion models. m:…2178

The null result is less interesting than how they got there. A temporal-leakage audit revealed significant alignment issues in the auxiliary modalities, affecting 5–58% of patients across different modalities — information from after the prediction window quietly leaking into the inputs. The authors' conclusion, in effect: some past multimodal wins may have been leakage wins. For anyone building clinical models, the paper sets a bar — audit temporal alignment before celebrating fusion gains — and for anyone buying them, it is a due-diligence question to ask out loud. m:…2178

An agent that predicts trials

In a preprint, Chen and colleagues introduced the inClinico Agent, a machine-learning system that predicts clinical trial efficacy and safety endpoints using only features available at trial registration — target, mechanism of action, modality, trial design and literature-derived embeddings. The efficacy model reached an AUROC of 0.828 on a test set of 2,347 trials initiated in 2021 or later, and the safety/toxicity model reached 0.809 on 1,485 trials. The design choice that matters is the frozen literature-derived embeddings and registration-time features: the system is built to avoid lookahead bias, the classic failure mode where a model quietly learns from information that would not exist at decision time. m:…5751

The authors also introduced ClinBench, a prospective benchmark of 2,040 ongoing trials to track predictive accuracy as trials reach completion — a leakage-free foundation for evaluation rather than a retrospective victory lap. If the numbers hold prospectively, the tool is aimed squarely at pharma operators and investors assessing trial risk before the money is spent. The honest framing is the prospective tracking: the authors are asking to be judged on trials that have not finished yet. m:…5751

A dosing system that ran a real trial

In a preprint, Cai and colleagues took the hardest path: a prospective randomized trial. Their LSTM-based system for basal and bolus insulin titration in type 2 diabetes, trained on 1,738 treatment courses, was evaluated in a prospective randomized trial (NCT07066891) of 97 patients — and achieved a median time in range of 84.5% (3.9–10.0 mmol/L) with minimal hypoglycemic risk. Prospective evaluation inside a randomized trial is the evidentiary gold standard that most clinical AI papers never reach. m:…4592

The paper's most useful finding may be the oversight signal: model performance is strongly associated with early glucose response, giving clinicians a practical flag for when the model needs human oversight and which patients require additional attention. That is the honesty the field needs — not just a performance number, but a built-in indicator of when the number should not be trusted. m:…4592

Methods over models

The pattern across the three preprints is that methodology is doing more work than model architecture. The fusion paper's contribution is a leakage audit; the trial agent's contribution is a prospective benchmark and lookahead safeguards; the dosing system's contribution is a randomized trial and an oversight flag. All three are preprints, and none should guide clinical or investment decisions yet — but together they sketch what credible clinical AI evidence looks like: audited inputs, prospective evaluation, and explicit statements of when the model should defer to a human. m:…2178 m:…5751 m:…4592