---
title: The drug-screening pipeline that loses to random
description: A preprint audit finds a standard machine-learning screening pipeline produces candidates no better than random, while a simple positional statistic, a generative design workflow and a viral forecasting tool all produce results that survive contact with experiment.
canonical: https://hangingcontext.com/stories/2026-11-18/drug-screening-reality-check/
last-updated: 2026-11-18
url: https://hangingcontext.com/stories/2026-11-18/drug-screening-reality-check/
published: 2026-11-18T12:42:26Z
edition: 2026-11-18T12
newsroom: research-science
beat: None
trust: confirmed
source_count: 4
cited_manifest_ids: ['1794997235134944730', '1794997235134944731', '1794991977357914224', '1794962745347772286']
---

# The drug-screening pipeline that loses to random

*A preprint audit finds a standard machine-learning screening pipeline produces candidates no better than random, while a simple positional statistic, a generative design workflow and a viral forecasting tool all produce results that survive contact with experiment.*

Confirmed · 4 Manifests · Research & Science · Published 2026-11-18 12:42 UTC

## Key facts

- In a preprint, a standard machine-learning virtual screening pipeline produced COX-2 candidates showing no docking enrichment compared with random compounds. (self-reported) [cites: 1794997235134944730]
- In the same preprint audit, 93% of the candidates generated by the Random Forest pipeline fell outside the model's applicability domain. (self-reported) [cites: 1794997235134944730]
- In a preprint, a parameter-free positional lookup table outperformed biophysical models at predicting FGFR drug-resistance phenotypes, and the signal transferred across FGFR inhibitors from pemigatinib to futibatinib. (self-reported) [cites: 1794997235134944731]
- In a preprint, REINVENT4-based generative design combined with similarity-guided screening identified 11 NCS-1 binders out of 12 tested compounds. (self-reported) [cites: 1794991977357914224]
- The Arc Institute reports that its redesigned antibody X51 neutralized both the XFG variant and a forecasted variant, XFG-Next. (self-reported) [cites: 1794962745347772286]

## The audit that failed

A standard machine-learning pipeline for finding new drugs produced candidates no better than random, according to a preprint posted on ChemRxiv. The authors tested the pipeline on COX-2 as a case study and found that its chosen compounds showed no docking enrichment compared with random compounds drawn from the ZINC250k library. The result is a warning for a field that leans heavily on retrospective benchmarks: the pipeline looked strong on paper and failed in the prospective test. [cites: 1794997235134944730]

The gap between the benchmark and the outcome was stark. The model posted a Pearson correlation of 0.756 on retrospective tests, yet its predictions showed almost no relationship with docking scores in the prospective test, with a Spearman correlation of minus 0.086. Worse, 93 percent of the candidates the pipeline generated fell outside the model's applicability domain, with a mean maximum similarity to the training data of 0.243. The authors checked their yardstick first: the docking tool they used, AutoDock Vina, successfully distinguished genuine known COX-2 actives from random compounds, so the failure sits with the pipeline rather than with the test. [cites: 1794997235134944730]

## A simple statistic beat the elaborate models

A second preprint, also on ChemRxiv, points the other way. Working on FGFR kinases, the authors found that a parameter-free positional lookup table outperformed established biophysical models, including AlphaMissense, FoldX, ThermoMPNN-D and AutoDock Vina, at predicting which mutations cause drug resistance. The signal was not tied to one drug: it transferred across FGFR inhibitors, from pemigatinib to futibatinib. It held across FGFR1, FGFR2 and FGFR3, though its predictive power weakened markedly for FGFR4. [cites: 1794997235134944731]

> **From Synorb** — Ask what Synorb covers. The MCP tool synorb-stream-search answers questions like "do you have this publisher?" without using any Manifest allowance. [Connect and ask](https://synorb.com/connect/start)


The contrast with the COX-2 audit is the point. One pipeline dressed up a familiar method and failed; a deliberately simple method, built on where in the protein a mutation sits, beat far more elaborate machinery. For teams choosing where to spend their modelling effort, the preprint suggests that the position of a mutation carries more signal than the physics-based scores layered on top of it. [cites: 1794997235134944731, 1794997235134944730]

## Design that survived contact with the lab

A third preprint offers a case where machine-led design did survive experiment. Using a generative design tool, REINVENT4, combined with similarity-guided screening, researchers identified 11 binders of the protein NCS-1 out of 12 compounds tested. The work produced the first small-molecule enhancers of the interaction between NCS-1 and the cannabinoid receptor CB1R, and high-resolution crystal structures confirmed that the new modulators engage the hydrophobic crevice of NCS-1. The difference from the COX-2 audit is that the claim rests on compounds that were made and measured, not on a benchmark score. [cites: 1794991977357914224]

## Forecasting the virus instead of screening the library

The Arc Institute, meanwhile, is trying a different route altogether: forecasting viral evolution rather than screening compounds. Its tool antiGen is a machine-learning method trained on pandemic-scale phylogenetic trees, and it is a small model, with roughly half a million parameters. Trained on data through 2023, it predicted future mutations more accurately overall than the other methods it was compared with, and it also performed well on influenza, RSV and dengue, which have less sequencing data than COVID-19. [cites: 1794962745347772286]

Its companion tool, Germinal Multi, designs a single antibody against several variants of a target at once, and can redesign existing antibodies to keep them active against new variants. The researchers redesigned an antibody, X51, which neutralized both the XFG variant and a forecasted variant, XFG-Next. The same tool was used to design antibodies against EGFR mutations in colorectal cancer. The institute is open about the limit: antiGen cannot yet determine which circulating variants will be fitter than others. [cites: 1794962745347772286]

Taken together, the four reports describe a field sorting out what machine learning is actually good for. Generic screening pipelines can flatter themselves on benchmarks while producing nothing better than chance; simple, position-aware statistics and tools built on evolutionary history are producing results that survive contact with experiment. The test that mattered, in each case, was not the score on a held-out benchmark but what happened when the prediction met a real molecule, a real virus or a real crystal. [cites: 1794997235134944730, 1794997235134944731, 1794991977357914224, 1794962745347772286]

## What is still unresolved

- Whether the COX-2 audit generalises beyond one target and one pipeline, or whether other standard pipelines fail the same prospective test.
- Whether the FGFR positional statistic holds up on resistance data collected after the preprint, and why its power weakens for FGFR4.
- Whether Germinal Multi-designed antibodies hold their activity against variants that emerge after the design date.

## Decision card

- **Useful for:** Drug-discovery teams choosing between screening pipelines and forecasting or position-aware methods; Investors diligence-checking AI drug-discovery claims
- **Not for:** Deciding whether a specific drug candidate will work in patients; Treating benchmark scores as evidence of prospective performance
- **Compare against:** Prospective experimental validation of any screening pipeline; Independent replication of the FGFR and antiGen results

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1794997235134944730` | A Standard Machine-Learning Virtual Screening Pipeline Produces Candidates Indistinguishable From Random Drug-Like Compounds: A Quantitative Audit Using COX-2 As Case Study | [chemrxiv-drug-discovery-medicinal-chemistry](https://chemrxiv.org/doi/full/10.26434/chemrxiv.15005946/v2) | 1794997489054946297, 1794997489070946299, 1794997489062946298, 1794997489078946300 | 11-18 00:00 |
| `1794997235134944731` | Drug-resistance phenotypes in FGFR deep mutational scanning are carried by a single-residue positional statistic that transfers across FGFR1-3 and weakens in FGFR4 | [chemrxiv-drug-discovery-medicinal-chemistry](https://chemrxiv.org/doi/full/10.26434/chemrxiv.15012418/v1) | 1794997489442946356, 1794997489444946357, 1794997489446946358 | 11-18 00:00 |
| `1794991977357914224` | From De Novo Molecular Design to Experimentally Validated Protein-Protein Interaction Modulators | [chemrxiv-biomolecular-chemical-biology](https://chemrxiv.org/doi/full/10.26434/chemrxiv.15012413/v1) | 1794992268576919590, 1794992268579919591, 1794992268580919592 | 11-18 00:00 |
| `1794962745347772286` | antiGen and Germinal Multi: Forecasting viral evolution and designing antibodies for future variants | [arc-institute-news](https://arcinstitute.org/news/antigen-and-germinal-multi) | 1794963012483773626, 1794963012465773623, 1794963012471773624, 1794963012488773627, 1794963012494773628, 1794963012477773625, 1794963012505773629, 1794963012534773632, 1794963012511773630 | 11-17 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. Existing 72 Hour Agent Demo keys keep their issued limits. New keys require email OTP verification. An existing demo key lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1794997235134944730`
- `1794997235134944731`
- `1794991977357914224`
- `1794962745347772286`

```sh
for id in 1794997235134944730 1794997235134944731 1794991977357914224 1794962745347772286; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Run the cited Manifests yourself. Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. Existing 72 Hour Agent Demo keys keep their issued limits. New keys require email OTP verification. An existing demo key lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons. [Get a key](https://synorb.com/connect/start)

## Related stories

- [The molecule models work — until the change is large](https://hangingcontext.com/stories/2026-11-21/molecular-prediction-hits-its-limits/)
- [Chemistry's new papers are pointed at the world outside the lab](https://hangingcontext.com/stories/2026-11-21/chemistry-aimed-outside-the-lab/)
- [Battery fade is not always damage — sometimes the chemistry just needs a stir](https://hangingcontext.com/stories/2026-11-20/battery-fade-is-not-always-damage/)
- [Before you trust the model, check the measurement](https://hangingcontext.com/stories/2026-11-20/chemistry-checks-its-own-tools/)

## Links

- HTML: https://hangingcontext.com/stories/2026-11-18/drug-screening-reality-check/
- JSON: https://hangingcontext.com/stories/2026-11-18/drug-screening-reality-check/index.json
- Markdown: https://hangingcontext.com/stories/2026-11-18/drug-screening-reality-check/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-11-18/drug-screening-reality-check/. Terms: https://hangingcontext.com/terms/reuse/
