{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1489",
  "slug": "ai-safety-governance-measured",
  "edition": "2026-11-13T00",
  "url": "https://hangingcontext.com/stories/2026-11-13/ai-safety-governance-measured/",
  "newsroom": "artificial-intelligence",
  "beat": "policy",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Safety governance gets measured: OpenAI funds child-safety research, Vals joins NIST, and a benchmark catches models guessing.",
  "headline": "AI safety governance starts getting measured",
  "dek": "OpenAI pledged $2 million for research on AI and children's safety, Vals AI joined the reconvened NIST Artificial Intelligence Consortium, and a Scale AI and Singapore IMDA benchmark found models often answer legal questions without asking for the missing facts.",
  "published_at": "2026-11-13T00:51:22Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 3 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 3,
  "publishers": [
    "openai-news",
    "scale-ai-blog",
    "vals-ai-blog"
  ],
  "sources": [
    "openai-news",
    "scale-ai-blog",
    "vals-ai-blog"
  ],
  "distinct_source_count": 3,
  "key_facts": [
    {
      "text": "OpenAI pledged $2 million to support research on AI's impact on children's safety and wellbeing. The $2 million funding supports the scientific research network under the Coalition for Children's Rights and Protection in the Age of AI.",
      "cites": [
        {
          "claim_ids": [
            "1794497094462752242",
            "1794497094467752244"
          ],
          "manifest_id": "1794496841890751675"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Vals AI announced its participation in the reconvened NIST Artificial Intelligence Consortium, which aims to establish new measurement science for AI.",
      "cites": [
        {
          "claim_ids": [
            "1794457508567583516",
            "1794457508570583517"
          ],
          "manifest_id": "1794457235617582225"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Scale AI and Singapore's IMDA introduced SG-LegalClarify, a benchmark for evaluating legal reasoning and ambiguity recognition under Singapore law. The benchmark evaluates 19 AI models across 1,860 legal scenarios spanning 10 domains of Singaporean law.",
      "cites": [
        {
          "claim_ids": [
            "1794508991957838020",
            "1794508991960838021"
          ],
          "manifest_id": "1794508731871836653"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Without access to web search, the highest-performing model in the benchmark satisfied 69.8% of mandatory grading criteria, and approximately 39% of scenarios with missing facts received no clarifying question from the models.",
      "cites": [
        {
          "claim_ids": [
            "1794508991965838024",
            "1794508991974838030"
          ],
          "manifest_id": "1794508731871836653"
        }
      ],
      "self_reported": true
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Two million dollars for research on children",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794497094462752242",
                "1794497094467752244",
                "1794497094469752245"
              ],
              "manifest_id": "1794496841890751675"
            }
          ],
          "markdown": "OpenAI pledged $2 million to support research on AI's impact on children's safety and wellbeing. The $2 million funding supports the scientific research network under the Coalition for Children's Rights and Protection in the Age of AI, and the research initiative aims to develop tools to measure AI risks and opportunities for children. The pledge is small next to the sums in AI, but the purpose is specific: measurement tools, not marketing, and research on children rather than on adults."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794497094470752246",
                "1794497094471752247"
              ],
              "manifest_id": "1794496841890751675"
            }
          ],
          "markdown": "Research teams funded by the initiative retain control over their conclusions, and research findings from the initiative will be published, including negative or inconclusive results. Those two commitments are the ones that make the pledge credible: the funder does not control the conclusions, and the results come out even when they are inconvenient."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794497094466752243",
                "1794497094473752248",
                "1794497094474752249"
              ],
              "manifest_id": "1794496841890751675"
            }
          ],
          "markdown": "OpenAI signed the Charter of Engagement for Safer AI for Children at the Paris Peace Forum, and supports youth AI safety legislation that includes age-appropriate safeguards by default. OpenAI plans to engage with European policymakers regarding the EU Kids Act. The pledge sits inside a policy move: a charter, support for legislation with safeguards by default, and engagement with the EU Kids Act, which is where the rules for children's AI will actually be written."
        }
      ]
    },
    {
      "heading": "A consortium to measure AI itself",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794457508567583516",
                "1794457508570583517",
                "1794457508595583523"
              ],
              "manifest_id": "1794457235617582225"
            }
          ],
          "markdown": "Vals AI announced its participation in the reconvened NIST Artificial Intelligence Consortium, which aims to establish new measurement science for AI. The consortium supports initiatives from America's AI Action Plan (2025). Measurement science is the unglamorous foundation of governance: before anyone can regulate AI, someone has to agree on how to measure what a model can do, and NIST is the body that does that work for the US government."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794457508573583519",
                "1794457508582583521",
                "1794457508606583527"
              ],
              "manifest_id": "1794457235617582225"
            }
          ],
          "markdown": "Rayan Krishnan asserts that independent evaluation is essential for measuring AI capabilities, performance and risks, and Glenn Parham states that independent evaluations provide a common basis for comparing AI performance across government and society. Vals AI is an independent AI evaluation company based in San Francisco, and works with domain experts to build benchmarks grounded in real-world tasks. Krishnan is the Co-Founder and CEO of Vals AI, and Parham is the Head of Public Sector at Vals AI. The argument is that evaluation must be independent of the companies being evaluated, or the scores are advertising."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794457508601583524",
                "1794457508603583525"
              ],
              "manifest_id": "1794457235617582225"
            }
          ],
          "markdown": "NIST does not evaluate commercial products under the Consortium, and NIST does not endorse any product or service used in the Consortium. Those limits are worth stating plainly: joining the consortium is a seat at the table where measurement is defined, not a government stamp on a company's product."
        }
      ]
    },
    {
      "heading": "Models answer legal questions without asking what is missing",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794508991957838020",
                "1794508991960838021",
                "1794508991962838022",
                "1794508991963838023"
              ],
              "manifest_id": "1794508731871836653"
            }
          ],
          "markdown": "Scale AI and Singapore's IMDA introduced SG-LegalClarify, a benchmark for evaluating legal reasoning and ambiguity recognition under Singapore law. The benchmark evaluates 19 AI models across 1,860 legal scenarios spanning 10 domains of Singaporean law, and contains 1,100 original questions written by qualified legal professionals with expertise in Singapore law. Of the 1,100 questions, 760 deliberately omit information needed to determine the correct answer. The design is the point: a good lawyer asks what is missing before answering, and the benchmark tests whether a model does the same."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794508991965838024",
                "1794508991966838025",
                "1794508991969838027",
                "1794508991968838026"
              ],
              "manifest_id": "1794508731871836653"
            }
          ],
          "markdown": "Without access to web search, the highest-performing model in the benchmark satisfied 69.8% of mandatory grading criteria, and the average performance across all 19 models without web search was 53.8% of mandatory grading criteria. Providing web search improved the average performance to 63.0%, and the highest-performing model with web access satisfied 77.8% of mandatory criteria. Even the best model with search leaves more than a fifth of the mandatory criteria unsatisfied, and the average model fails nearly half."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794508991974838030",
                "1794508991980838033",
                "1794508991973838028",
                "1794508991976838031"
              ],
              "manifest_id": "1794508731871836653"
            }
          ],
          "markdown": "Without web search, approximately 39% of scenarios with missing facts received no clarifying question from the models. Among web search-enabled trials with missing facts where models asked about every relevant missing fact, only 55.6% reached the correct legal conclusion. Models satisfied only 56.5% of applicable rule-recall criteria when using web access, and asked unnecessary questions in approximately 28% of scenarios where enough information had already been provided. The models fail in both directions: they answer when they should ask, and ask when they should answer. Asking every relevant question still leaves nearly half the conclusions wrong, which means the problem is not only missing facts but reasoning with the facts once they arrive."
        }
      ]
    },
    {
      "heading": "What measurement buys",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794497094469752245"
              ],
              "manifest_id": "1794496841890751675"
            },
            {
              "claim_ids": [
                "1794457508570583517"
              ],
              "manifest_id": "1794457235617582225"
            }
          ],
          "markdown": "The research initiative aims to develop tools to measure AI risks and opportunities for children, and the NIST AI Consortium aims to establish new measurement science for AI. The two efforts share a premise: governance starts with measurement, and measurement has to be built by people who are not selling the product."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794508991965838024"
              ],
              "manifest_id": "1794508731871836653"
            }
          ],
          "markdown": "The SG-LegalClarify results show why that premise matters. Without access to web search, the highest-performing model satisfied 69.8% of mandatory grading criteria, which is a number a buyer can use. A pledge, a consortium seat and a benchmark are not safety by themselves, but they are the instruments that let everyone else check whether safety claims are true."
        }
      ]
    }
  ],
  "unresolved": [
    "Whether the children's safety research produces measurement tools regulators actually adopt",
    "What measurement standards the reconvened NIST consortium will publish first",
    "Whether models improve at asking clarifying questions or keep answering past missing facts"
  ],
  "decision_card": {
    "not_for": [
      "Treating a pledge or a consortium seat as proof of safety",
      "Deploying models on legal questions without a human check"
    ],
    "useful_for": [
      "Policy teams tracking AI evaluation standards",
      "Executives weighing model risk in regulated work"
    ],
    "compare_against": [
      "Your own evaluation of the models you deploy",
      "The SG-LegalClarify results for your jurisdiction"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-13T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1794496841890751675",
    "1794457235617582225",
    "1794508731871836653"
  ],
  "manifests": [
    {
      "title": "Supporting research on AI and children's safety",
      "source_url": "https://openai.com/index/children-first-declaration",
      "manifest_id": "1794496841890751675",
      "source_name": "openai-news",
      "published_date": "2026-11-12"
    },
    {
      "title": "Vals AI Announces Participation in Reconvened NIST Artificial Intelligence Consortium",
      "source_url": "https://vals.ai/blogs/nist-ai-consortium",
      "manifest_id": "1794457235617582225",
      "source_name": "vals-ai-blog",
      "published_date": "2026-11-12"
    },
    {
      "title": "Scale and Singapore's IMDA Advance AI Evaluation with New Legal Reasoning Benchmark",
      "source_url": "https://scale.com/blog/scale-imda-legal-reasoning-benchmark",
      "manifest_id": "1794508731871836653",
      "source_name": "scale-ai-blog",
      "published_date": "2026-11-12"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1794496841890751675",
      "1794457235617582225",
      "1794508731871836653"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1794496841890751675",
      "all": "for id in 1794496841890751675 1794457235617582225 1794508731871836653; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-13/ai-safety-governance-measured/",
    "json": "https://hangingcontext.com/stories/2026-11-13/ai-safety-governance-measured/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-13/ai-safety-governance-measured/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-13T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-15/enterprise-ai-consolidates-measured-returns/",
      "headline": "Enterprise AI grows up: fewer bets, harder numbers, power contracts",
      "slug": "enterprise-ai-consolidates-measured-returns"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/back-office-agents-boring-work-receipts/",
      "headline": "The back office gets agents that do the boring work, with the receipts attached",
      "slug": "back-office-agents-boring-work-receipts"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/agents-turn-attacker-during-evals/",
      "headline": "The agents went off script in October, and the safety data says it was not a fluke",
      "slug": "agents-turn-attacker-during-evals"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/agents-doing-real-work-proof-campus-ward/",
      "headline": "Agents are doing the work now: a proof, a campus and a hospital ward",
      "slug": "agents-doing-real-work-proof-campus-ward"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Connect your agent over MCP",
    "text": "Add https://mcp.synorb.com/mcp to your agent, then call the tool synorb-get-demo-key if it has no key yet. It needs no email and no form. Keys are free for 72 hours, and new keys are limited per network.",
    "cta": {
      "label": "How to connect",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-13/ai-safety-governance-measured/"
  }
}