{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "825",
  "slug": "biology-foundation-models-face-a-reality-check",
  "edition": "2026-10-30T00",
  "url": "https://hangingcontext.com/stories/2026-10-30/biology-foundation-models-face-a-reality-check/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Biology foundation models expand into single-cell and clinical prediction, while new work tests their features.",
  "headline": "Biology Foundation Models Are Growing Up, and Their Features Are Being Checked",
  "dek": "A single-cell fine-tune of AlphaGenome, a clinical foundation model tested on thousands of outcomes, and a ground-truth test of interpretability features show both the reach and the limits of the field.",
  "published_at": "2026-10-30T00:56:50Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 3 sources",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 3,
  "publishers": [
    "biorxiv-computational-biology-ai",
    "biorxiv-research-methods-reproducibility",
    "medrxiv-health-ai-informatics"
  ],
  "key_facts": [
    {
      "text": "Scooby-2 reached a Pearson correlation of 0.56 on NeurIPS bone marrow gene expression differences, up from 0.53 for Scooby, and improved OneK1K variant effect concordance from 0.37 to 0.47.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793287622882117762",
          "claim_ids": [
            "1793289993985992062",
            "1793289993987664556"
          ]
        }
      ]
    },
    {
      "text": "PhenOM achieved an AUROC of at least 0.70 in 80 percent of 1,770 validated predictive tasks without additional training.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793314469572102056",
          "claim_ids": [
            "1793314750508906531",
            "1793314750511168330"
          ]
        }
      ]
    },
    {
      "text": "Consensus sparse autoencoders with five fits achieved 78 to 98 percent reproducibility, compared with 27 to 46 percent for single fits.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793297885844675666",
          "claim_ids": [
            "1793298142480551386"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "A genome model tuned for single cells",
      "paragraphs": [
        {
          "markdown": "In a preprint, Scooby-2 adapts the AlphaGenome foundation model for single-cell genomic track prediction, improving accuracy in gene expression and variant effect modelling at the single-cell level. The point of the fine-tune is precision about non-coding variants: knowing a variant exists is less useful than knowing which cell types it affects and how strongly.",
          "cites": [
            {
              "manifest_id": "1793287622882117762",
              "claim_ids": [
                "1793289993985992062",
                "1793289993987664556"
              ]
            }
          ]
        },
        {
          "markdown": "On the NeurIPS bone marrow dataset, Scooby-2 achieves a Pearson correlation of 0.56 when predicting cell-type-specific gene expression differences, compared with 0.53 for Scooby. Using the distilled AlphaGenome backbone, variant effect prediction concordance of fine-mapped expression quantitative trait loci from the OneK1K cohort improved from 0.37 to 0.47 Spearman rank correlation.",
          "cites": [
            {
              "manifest_id": "1793287622882117762",
              "claim_ids": [
                "1793289993985992062",
                "1793289993987664556"
              ]
            }
          ]
        },
        {
          "markdown": "Those are modest but meaningful gains on difficult prediction tasks, and they come from adapting an existing foundation model rather than training one from scratch for a single cell type. The preprint does not explicitly mention a public code repository URL in the abstract, which is a practical limit for teams wanting to reproduce the fine-tune.",
          "cites": [
            {
              "manifest_id": "1793287622882117762",
              "claim_ids": [
                "1793289993985992062"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "One clinical model, thousands of predictions",
      "paragraphs": [
        {
          "markdown": "In a preprint, a generative foundation model called PhenOM completes predictive tasks across clinical domains without additional training. It achieves an area under the receiver operating characteristic curve of at least 0.70 in 80 percent of 1,770 validated predictive tasks, according to the paper.",
          "cites": [
            {
              "manifest_id": "1793314469572102056",
              "claim_ids": [
                "1793314750508906531",
                "1793314750511168330"
              ]
            }
          ]
        },
        {
          "markdown": "A model that can generate thousands of predicted outcomes without task-specific retraining changes the workflow for health teams: instead of building a separate model for each outcome, they query one model across many outcomes. That breadth is also the risk, because a single score across 1,770 tasks hides which tasks perform well and which do not, and the preprint summary does not break out the weakest tasks.",
          "cites": [
            {
              "manifest_id": "1793314469572102056",
              "claim_ids": [
                "1793314750508906531",
                "1793314750511168330"
              ]
            }
          ]
        },
        {
          "markdown": "The right question for a team considering such a model is therefore not whether it predicts in general, but whether it predicts the specific outcome, population and time horizon they care about, at a level they can verify locally. Breadth without that local check is a starting point, not a deployment decision.",
          "cites": [
            {
              "manifest_id": "1793314469572102056",
              "claim_ids": [
                "1793314750508906531"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Can you trust what the model learned",
      "paragraphs": [
        {
          "markdown": "In a preprint, researchers provide a ground-truth evaluation of sparse autoencoder features on spatial, single-cell and language-model embeddings. Consensus sparse autoencoders using five fits achieve 78 to 98 percent reproducibility, significantly higher than single sparse autoencoder fits at 27 to 46 percent, which means a feature found once may not be found again unless the fit is repeated and combined.",
          "cites": [
            {
              "manifest_id": "1793297885844675666",
              "claim_ids": [
                "1793298142480551386"
              ]
            }
          ]
        },
        {
          "markdown": "Rotation-based methods including principal component analysis, Varimax and independent component analysis are highly reproducible but capture fewer biological concepts compared with sparse autoencoders, according to the paper. That creates a trade-off: a method can be stable and still miss the biology, or it can capture more biology and still be unstable from one fit to the next.",
          "cites": [
            {
              "manifest_id": "1793297885844675666",
              "claim_ids": [
                "1793298142482318138"
              ]
            }
          ]
        },
        {
          "markdown": "For teams building on biology foundation models, the implication is direct. Before treating an internal feature as a discovered biological concept, check whether it survives repeated fitting and whether it maps to known biology. Scooby-2 and PhenOM show what these models can predict; the sparse autoencoder evaluation shows how carefully their explanations should be handled.",
          "cites": [
            {
              "manifest_id": "1793297885844675666",
              "claim_ids": [
                "1793298142480551386"
              ]
            },
            {
              "manifest_id": "1793287622882117762",
              "claim_ids": [
                "1793289993985992062"
              ]
            },
            {
              "manifest_id": "1793314469572102056",
              "claim_ids": [
                "1793314750508906531"
              ]
            }
          ]
        },
        {
          "markdown": "That caution applies in both directions. A reproducible feature is not automatically a biological discovery, and a feature that appears in only one fit is not automatically meaningless, but neither should be reported without the repeatability check. The ground-truth test gives teams a way to run that check before they build a story, a target or a clinical hypothesis on top of a model explanation.",
          "cites": [
            {
              "manifest_id": "1793297885844675666",
              "claim_ids": [
                "1793298142480551386",
                "1793298142482318138"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "Whether Scooby-2 gains hold on single-cell datasets beyond NeurIPS bone marrow and OneK1K, and whether its fine-tune can be reproduced without a public code link in the abstract.",
    "Which of PhenOM 1,770 tasks perform poorly, and whether its predictions validate on a team's own patient population.",
    "Whether sparse autoencoder features that are reproducible in this ground-truth test carry clinical meaning, since no clinical validation of features is reported and the methods still struggle to recover weak planted states compared with rotations."
  ],
  "decision_card": {
    "useful_for": [
      "Biology and health AI teams fine-tuning foundation models or interpreting their internal features.",
      "Reviewers checking whether a claimed biological feature survives repeated fitting."
    ],
    "not_for": [
      "Clinical decisions based on a foundation model prediction or an interpretability feature alone.",
      "Treating a single sparse autoencoder fit as evidence for a biological concept."
    ],
    "compare_against": [
      "Task-specific models trained for the same single-cell or clinical outcome.",
      "Rotation-based interpretability methods on the same embeddings."
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-10-30T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793287622882117762",
    "1793314469572102056",
    "1793297885844675666"
  ],
  "manifests": [
    {
      "manifest_id": "1793287622882117762",
      "title": "Scooby-2: fine-tuning AlphaGenome for single-cell genomic track prediction",
      "source_name": "biorxiv-research-methods-reproducibility",
      "source_url": "https://biorxiv.org/content/10.64898/2026.10.28.762238v1.full",
      "published_date": "2026-10-29"
    },
    {
      "manifest_id": "1793314469572102056",
      "title": "Performance of a foundation model to generate thousands of predicted outcomes across multiple use cases",
      "source_name": "medrxiv-health-ai-informatics",
      "source_url": "https://medrxiv.org/content/10.64898/2026.10.28.26366723v1.full",
      "published_date": "2026-10-29"
    },
    {
      "manifest_id": "1793297885844675666",
      "title": "Ground-truth evaluation of sparse autoencoder features on spatial, single-cell and language-model embeddings",
      "source_name": "biorxiv-computational-biology-ai",
      "source_url": "https://biorxiv.org/content/10.64898/2026.10.27.762296v1.full",
      "published_date": "2026-10-29"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793287622882117762",
      "1793314469572102056",
      "1793297885844675666"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793287622882117762",
      "all": "for id in 1793287622882117762 1793314469572102056 1793297885844675666; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-10-30/biology-foundation-models-face-a-reality-check/",
    "json": "https://hangingcontext.com/stories/2026-10-30/biology-foundation-models-face-a-reality-check/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-10-30/biology-foundation-models-face-a-reality-check/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-10-30T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-10-30/agents-go-self-serve-for-developers/",
      "headline": "Agent Platforms Stop Building the Agent and Hand Developers the Keys",
      "slug": "agents-go-self-serve-for-developers"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-30/agent-developer-stack-around-the-model/",
      "headline": "The Agent Stack Around the Model Gets Its Own Launches",
      "slug": "agent-developer-stack-around-the-model"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-30/specialized-models-pick-one-job/",
      "headline": "Three New Models Skip the General Race and Pick One Job",
      "slug": "specialized-models-pick-one-job"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-30/science-agents-checked-against-evidence/",
      "headline": "Science Agents Are Being Built to Show Their Work, Not Just Their Answers",
      "slug": "science-agents-checked-against-evidence"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Save what you follow as a Beacon",
    "text": "A Beacon is a saved query that your agent pulls with an explicit lookback window. Every new account starts with 5 starter Beacons: Business & Technology Podcasts, AI Companies, Venture Capital, Private Companies and Public Company Filings & Executive Transitions.",
    "cta": {
      "label": "See Beacons",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-10-30/biology-foundation-models-face-a-reality-check/"
  }
}