The genomic data your models train on is missing its own context

The context is missing at the source

Genomic assemblies in public archives often arrive without the biological context that makes them usable — ploidy, cultivar, subgenome structure — and the gap is not a small one. A new preprint argues the information must be captured at submission, in controlled vocabulary, because recovering it later from the literature does not work: the literature-based recovery the authors tested failed to identify ploidy for nearly one-third of plant assemblies. The metadata is missing by design, in the sense that nobody is required to supply it — and by the time a model trains on the data, the context is gone. m:…4904

The fix the authors propose is procedural, not technical: mandatory, controlled-vocabulary checklists at the point of submission. That is a deliberately boring solution to a problem that expensive modelling cannot solve after the fact. If the ploidy of a plant assembly cannot be recovered from the literature in nearly a third of cases, then every downstream analysis that assumes it — comparative genomics, training data for genomic models, cross-species comparisons — is building on a guess. A checklist at submission is cheap; a wrong assumption baked into a training set is not. m:…4904

The measurement depends on the method

Even when the data exists, what it says depends on how it was made. A study of long-read RNA sequencing found that the library preparation protocol changes the isoforms researchers observe: PCR amplification biases against transcripts with high m6A methylation, reverse transcription and PCR increase the frequency of observed exitrons — inserted introns — compared with direct RNA sequencing, and differential transcript usage shows up in hundreds of genes across cell lines when protocols are compared. The same sample, processed two ways, can give two different answers about which transcripts are present. m:…9638

The reports still get it wrong

A third preprint takes the problem downstream, to clinical genome sequencing pipelines, and builds a way to test the whole chain from simulated reads to the final report. On real reads the framework reached 99.3% SNV recall, and report-level sensitivity for pathogenic and likely pathogenic variants was 98.3% — 115 of 117. But report-level specificity was 84.1%, 106 of 126, with failures coming mostly from upgrading variants of uncertain significance and from incorrect recessive carrier reporting. One concrete fix fell out of the testing: calling chromosome X as haploid in males eliminates false reports caused by diploid calling, with no side effects. m:…4712

The fourth paper turns the same auditing instinct on the foundation models themselves. AutoSPECTRA is an agent-driven protocol for testing how scientific foundation models generalize, and its first runs trace the limits precisely: in ESMFold2, performance limits track protein sequence composition and how closely a sequence resembles the pretraining data; in AlphaMissense, they track residue-level genomic, evolutionary and structural context. The point is not that the models fail, but that where they fail is predictable — if you audit for it. m:…5045

The two audits point in the same direction. The validation framework finds the errors at the reporting step, where a variant of uncertain significance gets upgraded or a carrier gets misreported; AutoSPECTRA finds the limits at the model step, where performance tracks how close the input sits to the training data. Neither finding is a reason to stop using these tools. Both are reasons to stop using them without a test harness — a truth-set for the pipeline, an audit for the model — and to treat a good score on an unaudited benchmark as a rumour rather than a result. m:…4712 m:…5045

A supply-chain problem, not a modelling problem

Read together, the four papers describe a supply-chain problem rather than a modelling problem. Metadata goes missing at submission, measurements shift with the protocol, reports err at the last step, and the models inherit all of it — their limits traceable, in the AutoSPECTRA audit, to exactly the data they were trained on. For teams building on genomic data, the cheapest improvement may not be a bigger model but a stricter intake: demand the metadata, record the protocol, and run the pipeline through a truth-set before trusting its reports. m:…4904 m:…9638 m:…4712 m:…5045