{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "538",
  "slug": "clinical-ai-must-prove-itself",
  "edition": "2026-10-24T12",
  "url": "https://hangingcontext.com/stories/2026-10-24/clinical-ai-must-prove-itself/",
  "newsroom": "health-biotech",
  "beat": null,
  "newsroom_title": "Health & Biotech",
  "angle": "Clinical AI vendors are now selling concordance with humans, explainability, and licensed data — not accuracy alone.",
  "headline": "Clinical AI's new pitch isn't accuracy — it's proof it matches a human",
  "dek": "Komodo's AI matched a human analyst step for step on cohort selection, a hospital AI study stresses explainability by data source, and Tempus licensed its oncology data platform through 2031. The pitch is proof, not performance.",
  "published_at": "2026-10-24T12:31:34Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 4 sources",
    "newsroom": "Health & Biotech"
  },
  "source_count": 4,
  "publishers": [
    "komodo-health-perspectives",
    "medrxiv-health-ai-informatics",
    "medrxiv-neurology-mental-health",
    "tempus-ai-news-releases"
  ],
  "key_facts": [
    {
      "text": "Komodo Health says its Marmot AI produced identical counts at all eight cohort selection steps in a blinded comparison with a human analyst, with 16 of 17 comorbidity components matching and mean comorbidity scores of 1.1 for Marmot versus 1.5 for manual analysis.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792775874540996476",
          "claim_ids": [
            "1792776118648540785",
            "1792776118654405421",
            "1792776118656253613"
          ]
        }
      ]
    },
    {
      "text": "Komodo Health says discrepancies between the pipelines were traced to derived-variable implementation, and validation of downstream analyses including resource utilization and cost outcomes is ongoing.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792775874540996476",
          "claim_ids": [
            "1792776118657720760",
            "1792776118660312997"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, multimodal fusion of routine electronic health record data improves in-hospital detection of lung cancer and heart failure compared with laboratory data alone, but explanation faithfulness varies by data source, with laboratory-based explanations failing weight-randomization checks.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792774449964395821",
          "claim_ids": [
            "1792774887727858510",
            "1792774887785678096"
          ]
        }
      ]
    },
    {
      "text": "In a preprint on Rett syndrome classification from resting-state EEG, Random Forest performed best among classical models, EEGNet best among deep learning models, 4-second epochs offered the best trade-off of performance and robustness, and delta and theta oscillations with centro-frontal regions were key contributors.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792769962198335407",
          "claim_ids": [
            "1792770292653161362",
            "1792770292656952477",
            "1792770292657459929",
            "1792770292659913662"
          ]
        }
      ]
    },
    {
      "text": "Tempus says it entered a multi-year agreement under which Bristol Myers Squibb will license multimodal oncology data from Tempus through 2031 and gain access to the Lens Platform.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792771213973335620",
          "claim_ids": [
            "1792771447336838685",
            "1792771447347238630",
            "1792771447332883505"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Matching the human",
      "paragraphs": [
        {
          "markdown": "The clearest version of the new pitch comes from Komodo Health, which says it ran a blinded study comparing its Marmot AI platform against a human analyst for real-world evidence generation. The study focused on a non-ambulatory Duchenne muscular dystrophy cohort, and both pipelines produced identical counts at all eight cohort selection steps. That is the kind of result a buyer can check: not a score on a hidden test set, but a step-by-step match with the person the tool is supposed to assist. Komodo Health also reports that the initial cohort size was 30,221 patients and the final cohort size was 1,130 patients, so the match held through substantial narrowing.",
          "cites": [
            {
              "manifest_id": "1792775874540996476",
              "claim_ids": [
                "1792776118639690799",
                "1792776118642013769",
                "1792776118648540785"
              ]
            },
            {
              "manifest_id": "1792775874540996476",
              "claim_ids": [
                "1792776118650626340",
                "1792776118651320324"
              ]
            }
          ]
        },
        {
          "markdown": "The study is more persuasive because it reports where the match broke. Age, region, and race and ethnicity distributions matched exactly between the two pipelines, and 16 of 17 Charlson Comorbidity Index components matched, but the mean comorbidity score was 1.5 in the manual analysis and 1.1 in the Marmot analysis, and payer mix differed between the two pipelines. Komodo Health says the discrepancies were traced to derived-variable implementation, which is another way of saying the two sides defined a computed variable differently. Three repeated Marmot executions returned identical outputs under the tested conditions, which speaks to repeatability, but the company also says validation of downstream analyses, including healthcare resource utilization and cost outcomes, is ongoing. The honest boundary is part of the pitch: the tool matched a human on cohort selection, and the rest is not yet proven.",
          "cites": [
            {
              "manifest_id": "1792775874540996476",
              "claim_ids": [
                "1792776118652060063",
                "1792776118654405421",
                "1792776118656253613",
                "1792776118657720760"
              ]
            },
            {
              "manifest_id": "1792775874540996476",
              "claim_ids": [
                "1792776118659548501",
                "1792776118660312997"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Explaining itself, by source",
      "paragraphs": [
        {
          "markdown": "A preprint on in-hospital detection of lung cancer and heart failure makes a parallel argument for a different kind of proof. Multimodal fusion of routine electronic health record data, combining lab results, radiology reports, and ECGs, significantly improves in-hospital detection of lung cancer and heart failure compared with laboratory data alone. The study also reports that FedProx federated training recovers 82 percent of centralized performance gains for rare outcomes like lung cancer, outperforming FedAvg, which matters for hospitals that cannot pool patient data centrally. Detection improves when sources are combined, and training can stay distributed without giving up most of the gain.",
          "cites": [
            {
              "manifest_id": "1792774449964395821",
              "claim_ids": [
                "1792774887727858510",
                "1792774887740773185"
              ]
            }
          ]
        },
        {
          "markdown": "But the preprint adds a check that many accuracy reports skip: explanation faithfulness varies by data source, with laboratory-based explanations failing weight-randomization checks. An explanation that does not survive a randomization check is not really explaining the model's decision, even if it looks plausible. The implication for buyers is that explainability is not one property of a model but a separate test for each data source feeding it, and a model can be explainable for radiology while misleading for labs.",
          "cites": [
            {
              "manifest_id": "1792774449964395821",
              "claim_ids": [
                "1792774887785678096"
              ]
            }
          ]
        },
        {
          "markdown": "A second preprint, on classifying Rett syndrome from resting-state EEG, shows the same discipline applied to model selection. Random Forest achieved the best performance among the classical machine learning models, EEGNet achieved the highest deep learning performance, and 4-second EEG epochs offered the optimal trade-off between model performance and robustness. Delta and theta oscillations, along with centro-frontal brain regions, are key contributors to classification, which gives clinicians a physiological story to check rather than a black-box verdict. The pattern across both preprints is consistent: the result worth reporting is not only which model won, but under what conditions and for which signals.",
          "cites": [
            {
              "manifest_id": "1792769962198335407",
              "claim_ids": [
                "1792770292653161362",
                "1792770292656952477",
                "1792770292657459929",
                "1792770292659913662"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Proof as a product",
      "paragraphs": [
        {
          "markdown": "Tempus shows the commercial side of the same shift. The company says it entered a multi-year agreement with Bristol Myers Squibb under which Bristol Myers Squibb will license multimodal oncology data from Tempus through 2031 and gain access to the Lens Platform. Tempus says the licensed data supports biological and clinical insight identification, novel drug target discovery, and therapeutic development across the oncology portfolio, and that the Lens Platform allows researchers to train and execute advanced AI models. The company also says the existing relationship includes applying AI and real-world data to optimize clinical trial design and neuroscience initiatives, so the new agreement expands a working relationship rather than starting one.",
          "cites": [
            {
              "manifest_id": "1792771213973335620",
              "claim_ids": [
                "1792771447332883505",
                "1792771447336838685",
                "1792771447347238630"
              ]
            },
            {
              "manifest_id": "1792771213973335620",
              "claim_ids": [
                "1792771447338079008",
                "1792771447341674985",
                "1792771447343402356",
                "1792771447387482430",
                "1792771447388371025",
                "1792771447390935009"
              ]
            }
          ]
        },
        {
          "markdown": "Put the three items together and the market's new question comes into focus. Buyers are no longer asking only how accurate a clinical AI tool is; they are asking whether it matches a human on the same task, whether its explanations hold up per data source, and whether its data and platform are licensed as durable infrastructure. Tempus says its agreement expands a long-standing strategic relationship, and Komodo Health is careful to say downstream validation is ongoing. That combination, confidence about what was proven plus clarity about what was not, is becoming the price of admission. Vendors that can only quote a score will find the conversation has moved on to concordance, and concordance is a harder test to pass.",
          "cites": [
            {
              "manifest_id": "1792775874540996476",
              "claim_ids": [
                "1792776118660312997"
              ]
            },
            {
              "manifest_id": "1792771213973335620",
              "claim_ids": [
                "1792771447387482430"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "Whether Marmot's downstream analyses, including resource utilization and cost outcomes, match human analysis once validation completes.",
    "Whether federated training gains for rare outcomes hold across hospital systems beyond the preprint's dataset.",
    "Whether the Lens Platform agreement changes drug discovery timelines, which neither company quantifies in the cited claims."
  ],
  "decision_card": {
    "useful_for": [
      "Health system buyers weighing AI tools for evidence generation, detection, and drug discovery.",
      "Clinical AI teams deciding what proof a skeptical buyer will ask for next."
    ],
    "not_for": [
      "Clinical deployment decisions, which need prospective validation beyond these studies.",
      "Anyone treating a vendor concordance study as proof of downstream outcomes, which Komodo says is still being validated."
    ],
    "compare_against": [
      "Vendor accuracy claims without a human comparison, an explanation check, or a stated validation scope.",
      "Single-source models presented as replacements for multi-source clinical workflows."
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-10-24T12"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1792775874540996476",
    "1792774449964395821",
    "1792769962198335407",
    "1792771213973335620"
  ],
  "manifests": [
    {
      "manifest_id": "1792775874540996476",
      "title": "Evaluating Marmot for Real-World Evidence Generation: A Blinded Human-Analyst Concordance Study in Duchenne Muscular Dystrophy",
      "source_name": "komodo-health-perspectives",
      "source_url": "https://komodohealth.com/perspectives/voices-marmot-for-rwe-generation-a-blinded-study",
      "published_date": "2026-10-23"
    },
    {
      "manifest_id": "1792774449964395821",
      "title": "Explainable Multimodal AI for In-Hospital Detection of Lung Cancer and Heart Failure Using Routine Data: Development, Internal Validation, and Simulated Federated Evaluation",
      "source_name": "medrxiv-health-ai-informatics",
      "source_url": "https://medrxiv.org/content/10.64898/2026.10.22.26366034v1.full",
      "published_date": "2026-10-23"
    },
    {
      "manifest_id": "1792769962198335407",
      "title": "EEG Biomarkers for Rett Syndrome: A Comparative Analysis of Machine Learning and Deep Learning with Explainable AI",
      "source_name": "medrxiv-neurology-mental-health",
      "source_url": "https://medrxiv.org/content/10.64898/2026.10.21.26366077v1.full",
      "published_date": "2026-10-23"
    },
    {
      "manifest_id": "1792771213973335620",
      "title": "Release Details",
      "source_name": "tempus-ai-news-releases",
      "source_url": "https://investors.tempus.com/news-releases/news-release-details/tempus-expands-strategic-collaboration-bristol-myers-squibb-0",
      "published_date": "2026-10-23"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1792775874540996476",
      "1792774449964395821",
      "1792769962198335407",
      "1792771213973335620"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; 3 of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1792775874540996476",
      "all": "for id in 1792775874540996476 1792774449964395821 1792769962198335407 1792771213973335620; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-10-24/clinical-ai-must-prove-itself/",
    "json": "https://hangingcontext.com/stories/2026-10-24/clinical-ai-must-prove-itself/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-10-24/clinical-ai-must-prove-itself/index.md",
    "newsroom": "https://hangingcontext.com/n/health-biotech/",
    "edition": "https://hangingcontext.com/editions/2026-10-24T12/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-10-24/biology-ai-tooling-layer/",
      "headline": "Biology's tooling layer levels up: nanoparticle shape controls mRNA delivery, RNA structures at scale",
      "slug": "biology-ai-tooling-layer"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-23/organoids-close-in-on-disease/",
      "headline": "The gap between the dish and the patient keeps narrowing",
      "slug": "organoids-close-in-on-disease"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-22/de-novo-ligand-platforms/",
      "headline": "De novo discovery delivers first-in-class ligands for three hard targets",
      "slug": "de-novo-ligand-platforms"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-22/cancers-exploitable-biology/",
      "headline": "Cancer's weak spots, three ways: a metabolite, a mechanical signal, an immune checkpoint",
      "slug": "cancers-exploitable-biology"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Ask what Synorb covers",
    "text": "The MCP tool synorb-stream-search answers questions like \"do you have this publisher?\" without using any Manifest allowance.",
    "cta": {
      "label": "Connect and ask",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-10-24/clinical-ai-must-prove-itself/"
  }
}