The lightning models may be overstated — and the 76–140% projection rides on them

The big number

In a preprint, an XGBoost model projects a 76–140% increase in global cloud-to-ground lightning frequency under the SSP3-7.0 high-emission scenario. The same preprint reports that lightning frequency is sensitive to global warming at roughly 30% per degree Celsius, and finds the XGBoost model captures spatial and temporal lightning patterns better than multiple linear regression and conventional convective parameterizations. That is the number now circulating for infrastructure and wildfire risk planning: at least three-quarters more lightning, possibly more than double. m:…0584

The audit

A second preprint runs the audit that the first paper's number needs. A simple climatology benchmark — predicting each grid cell's training-data mean lightning density for each day of year — reproduces 92% of the variance in daily grid-cell LIS/OTD satellite observations. And previously reported model skill at reproducing inter-annual variability turns out to be an artifact of inconsistent observational coverage: it disappears when models are evaluated under a common spatiotemporal domain. m:…8683

This suggests an uncomfortable reading of the 76–140% figure, and it should be read as reasoning, not as the audit paper's verdict: if a calendar-and-geography average explains nearly all the variance the satellites see, and reported year-to-year skill evaporates under consistent evaluation, then the projection's precision may be resting on the easy part of the problem. The audit paper does not single out the XGBoost paper as failing, and the two may use different data and domains — but the burden it places on any lightning projection is clear: show your skill outside the training distribution, under one consistent observational domain. m:…8683 m:…0584

The audit's demand is specific, and it is the demand planners should now make of every lightning projection: demonstrate skill outside the training distribution, under one consistent observational domain. Until a model clears that bar, its headline number is a hypothesis with error bars, not a measurement. m:…8683 m:…0584

The instruments carry artifacts too

The measurement side has its own version of this problem. A third preprint finds that hard-tail photon spectra in terrestrial gamma-ray flashes recorded by AGILE's MCAL detector are consistent with detector pile-up effects rather than intrinsic characteristics of the source — what looked like physics in the data was the instrument tripping over itself. When the same study compared neutron-production simulations for a standard TGF, MCNP6.2 and Geant4 disagreed by 37% (4.13E+14 versus 2.60E+14 neutrons per 1E+18 photons above 1 MeV), a gap the paper attributes to different photonuclear cross-section datasets and neutron energy-distribution modeling. m:…6538

What to do with the number

For a planner, the honest use of the 76–140% number is as a scenario, not a measurement. The 30%-per-degree sensitivity is a model result in a preprint, and the audit preprint shows lightning models' reported skill has been overstated before. The practical move is to ask any lightning-risk model for its skill outside its training distribution and under a common observational domain — the two conditions the audit paper flags as missing — and to keep the plain climatology baseline next to the projection, since it already explains 92% of what the satellites observe. m:…8683 m:…0584

None of this says lightning will not increase as the planet warms; the 30%-per-degree sensitivity is a real model result. What the audit says is that the precision of the projection — the difference between 76% and 140%, the regional patterns the XGBoost model claims to capture better than its rivals — is exactly where the overstatement lives. A planner who budgets the midpoint and ignores the error bars is buying the model's confidence, not its skill. m:…0584 m:…8683