A psychosis MRI model fails its road test while Alzheimer's diagnostics map their boundaries

A psychosis model that doesn't travel

A structural MRI model for predicting psychosis risk looks fine in its home cohort and falls apart everywhere else. In a preprint, the model was tested across four independent clinical cohorts and returned an AUC of 0.56, which is weak discrimination, barely better than chance. Calibration did no better: a slope of 0.17 means the model's risk estimates failed to transfer between cohorts, so a prediction that meant one thing in one clinic meant something else in another. m:…3322

The paper's own analysis explains the average: model discrimination varied significantly with negative-symptom severity. Put another way, the headline AUC was never a property of the test alone. It was a property of the patients it happened to be tested on. For diagnostics developers this is the cautionary exhibit of the week: a single-cohort accuracy figure is a promise, not a result, and it tends to break on contact with real clinical heterogeneity. m:…3322

The deeper read is that negative-symptom severity is a confound no amount of training data fixes if every cohort shares it. A model trained and validated inside similar clinics will keep looking accurate while being silently wrong for the patients who differ most. My take for developers: the fix is boring and expensive, which is to validate across the full spread of symptom severity before claiming generalizability, not after. m:…3322

Blood and spectra draw their boundaries

Plasma p-tau217 got the clearest performance map of the week. In a preprint, Xu, Michopoulou, Costen and colleagues ran QC-first internal validation in ADNI with separate axis-specific assessments: AUROC of 0.9425 for amyloid positivity and 0.9495 for tau positivity. Then they drew the boundary. Within amyloid-positive individuals, tau-positivity discrimination fell to 0.8852, and at 90% sensitivity, specificity for tau within amyloid-positive participants was only 60.7 percent. In Bio-Hermes, adding amyloid-beta 40 and 42 to p-tau217 nudged AUROC from 0.894 to 0.916, a small gain that suggests plasma-only panels are approaching their ceiling. m:…6253

A second preprint, from Dell'Orco and colleagues, turned to neurochemistry instead of antibodies. 7T 1H-MRS fingerprinting along the Alzheimer's continuum classified AD versus healthy controls with an AUC of 0.88, with myo-inositol, ascorbate, and choline-containing compounds as the most significant features. Strong numbers, but the lesson of the other papers applies here too: the interesting question is not the headline AUC. It is where that number holds up. m:…3198

Drawing the clock as a network

The most operationally interesting result came from digital clock drawings. In a preprint, Pinheiro and colleagues modeled the drawings as networks rather than handcrafted summary features and found the network-based models achieved predictive performance comparable to established summary-feature baselines for cognitive impairment detection. Nothing was lost by letting the geometry speak for itself. m:…3105

The impaired group showed distinct spatial organization in the upper-right clock region, the digits 1 and 2, the hands, and the face, compared with cognitively intact participants, and the gap between command and copy network representations was larger among the impaired. This is the kind of fine-grained signal a scoring checklist cannot hand you, and it arrived without requiring new hardware. m:…3105

The broader point is about deployability. A clock drawing needs a pen and a phone camera, not a scanner or a blood draw. If network representations can extract as much signal as the handcrafted baselines, the humble clock test gets a second life as a scalable screen, with the interesting failure modes documented up front rather than discovered in the field. m:…3105