Environmental ML Faces Its Validation Reckoning

The bleaching models were grading their own homework

The week's sharpest audit targets the machine-learning models used to predict global coral bleaching severity. In a preprint, the authors show that random-split validation — the default way these models were evaluated — overstates predictive skill, because the test sets were not really independent. More than a third of the records in the Global Coral Bleaching Database (BCO-DMO dataset 773466) are exact duplicates: 34.5% of the database is the same observation counted twice, and nearby records are spatially dependent, so a random split quietly leaks training information into the test set. m:…4013

The damage shows up when the validation gets honest. Under spatial validation — where models are tested on genuinely separate locations — the environmental predictors lose their edge, and indicators of which data source a record came from explain 0.35 to 0.37 of the variance in bleaching severity. This is my read as a writer, not reported fact: the models were not learning bleaching physics so much as learning the quirks of who collected the data. For a field that feeds predictions to reef managers and policymakers, that is a credibility problem, not a technicality. m:…4013

Negative R-squared in the desert

The same pattern — good in-sample, broken out-of-sample — turns up underground. In a preprint studying aquifer hydraulic-conductivity prediction in Cedar Valley, Utah, the authors document a substantial performance gap between in-sample fit and geographic generalization. When the models (random forests, support-vector regression, and XGBoost) were evaluated with spatial cross-validation — trained on some areas, tested on others — every one of them produced negative R-squared values, meaning they performed worse than simply predicting the mean. m:…9789

Screen depth and the topographic wetness index were the largest contributors to the model-based predictions in the study area, My read as a writer, not reported fact: the features that dominate a model fitted to one basin do not necessarily travel to the next, so anyone using ML groundwater maps should be asking which validation scheme produced the skill score on the brochure. m:…9789

Even the warming trend gets audited

The reckoning is not only for machine learning. A separate preprint offers a reproducible reanalysis of statistical trend models for the Berkeley Earth global temperature record — no neural networks, just the classical question of how fast the planet is warming and whether it is speeding up. The headline number is steady: a 1970-2024 warming rate of 0.206 ± 0.022 °C per decade. But the evidence for a recent acceleration, modeled as a break in 2013, is statistically marginal at p = 0.064. This is my read as a writer, not reported fact: trend claims depend on where you let the model bend. m:…7662

The reanalysis also demonstrates how fragile extrapolation is: fitting polynomial trend models of degrees 1 through 5 to the same record produces 2100 projections ranging from 1.16 to 16.43 °C. My read as a writer, not reported fact: the linear rate is robust, but the choice of model family dominates the far-future number, so honest reporting should say which model produced it. m:…7662

One way out: put the physics back in

A fourth preprint, introducing the Ecological Knowledge-Guided Machine Learning (Eco-KGML) framework, sketches the constructive answer: integrate mechanistic knowledge and monitoring data with machine learning, rather than letting models memorize datasets. The authors position it as a flexible, scalable methodological framework for limnology. This is my read as a writer, not reported fact: the through-line of this week's papers is that environmental ML's next leap will come from harder validation and stronger priors, not bigger models — the scoreboard needs to be honest before the science can be. m:…8661