{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1154",
  "slug": "gpt6-cyber-security-evaluations",
  "edition": "2026-11-06T00",
  "url": "https://hangingcontext.com/stories/2026-11-06/gpt6-cyber-security-evaluations/",
  "newsroom": "artificial-intelligence",
  "beat": "models-labs",
  "newsroom_title": "Artificial Intelligence",
  "angle": "OpenAI's security model looks strong on scripted tests and ordinary targets, and stops at hardened ones",
  "headline": "GPT-6 Cyber aces the scripted attacks and stalls on hardened targets",
  "dek": "Two independent evaluations of GPT-6 Cyber agree on the shape of the result: it finds more, finishes scripted attacks, and still fails against hardened targets.",
  "published_at": "2026-11-06T00:34:18Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 3 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 3,
  "publishers": [
    "coderabbit-private-company",
    "irregular-research",
    "openai-news"
  ],
  "sources": [
    "coderabbit-private-company",
    "irregular-research",
    "openai-news"
  ],
  "distinct_source_count": 3,
  "key_facts": [
    {
      "text": "CodeRabbit found GPT-6 Cyber caught 22 selected issues versus 19 without it, with 66% more high- and medium-severity findings — but says it still needs to check how many of the extra findings are real vulnerabilities.",
      "self_reported": false,
      "cites": [
        {
          "manifest_id": "1793910947336909136",
          "claim_ids": [
            "1793911197907900547",
            "1793911197909665703",
            "1793911197910705370"
          ]
        }
      ]
    },
    {
      "text": "Irregular Research found GPT-6 Cyber solved all 11 CyScenarioBench scenarios and all 22 Atomic challenges at least once, and 90 of 288 FrontierCyber challenges — with no successful attacks on fully hardened targets.",
      "self_reported": false,
      "cites": [
        {
          "manifest_id": "1793909926967313802",
          "claim_ids": [
            "1793910275821505235",
            "1793910275822875989",
            "1793910275824664135",
            "1793910275825045746"
          ]
        }
      ]
    },
    {
      "text": "GPT-6 Astra, the general model, solved 96 of 288 FrontierCyber challenges versus 90 for the security variant, and neither solved any of the 15 highest-difficulty challenges.",
      "self_reported": false,
      "cites": [
        {
          "manifest_id": "1793909926967313802",
          "claim_ids": [
            "1793910275832146781",
            "1793910275834389772"
          ]
        }
      ]
    },
    {
      "text": "Merge, which builds on OpenAI technology, reports tool calls through its own systems succeed 99.9% of the time, versus around 60% for public MCP servers.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793922749815035285",
          "claim_ids": [
            "1793923005755037716",
            "1793923005756037717"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "What the evaluators found",
      "paragraphs": [
        {
          "markdown": "CodeRabbit evaluated GPT-6 Cyber, a cybersecurity variant of OpenAI's GPT-6 models trained for security research, inside its CodeRabbit Secure platform, with access provided through the OpenAI Daybreak Defense Network. The evaluation covered Python, JavaScript, TypeScript, Go, PHP, Rust, Ruby, C, and C#. The ensemble with GPT-6 Cyber caught 22 selected issues compared with 19 without it, and reported 66% more high- and medium-severity findings. Injection and server-side request forgery showed the largest percentage increases, and denial of service added 98 findings while authorization bypass added 77.",
          "cites": [
            {
              "manifest_id": "1793910947336909136",
              "claim_ids": [
                "1793911197905910291",
                "1793911197912483674",
                "1793911197913922565",
                "1793911197915950961"
              ]
            }
          ]
        },
        {
          "markdown": "Irregular Research, which collaborated with OpenAI on its evaluation, tested the model across three offensive security benchmarks: FrontierCyber, CyScenarioBench, and Atomic Challenges. GPT-6 Cyber solved all 11 CyScenarioBench scenarios at least once and all 22 Atomic challenges at least once, and 90 of 288 FrontierCyber challenges. FrontierCyber tests vulnerability discovery and exploitation against real software and hardware, and the evaluators observed no successful attacks on fully hardened targets. On Atomic Challenges, the model reached a 97% success rate in vulnerability research and exploitation, 98% in network attack simulation, and 63% in evasion.",
          "cites": [
            {
              "manifest_id": "1793909926967313802",
              "claim_ids": [
                "1793910275818523432",
                "1793910275821505235",
                "1793910275822875989",
                "1793910275824664135",
                "1793910275825045746"
              ]
            }
          ]
        },
        {
          "markdown": "The comparison with the general model is the part that should slow down any simple story about specialization. GPT-6 Astra solved 96 of 288 FrontierCyber challenges, more than the security variant's 90, and neither model solved any of the 15 highest-difficulty FrontierCyber challenges. Where GPT-6 Cyber did pull ahead was on multi-stage work: it completed 11 of 11 CyScenarioBench scenarios against 8 of 11 for Astra, with an average success rate of 61%, up 7 percentage points. The pattern is that the security tuning helps most on long, staged operations, not on raw exploitation.",
          "cites": [
            {
              "manifest_id": "1793909926967313802",
              "claim_ids": [
                "1793910275832146781",
                "1793910275834389772",
                "1793910275835465260",
                "1793910275836471136"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "The false-positive problem nobody has solved yet",
      "paragraphs": [
        {
          "markdown": "CodeRabbit is candid about the gap in its own result: it needs to assess how many of the additional findings are real vulnerabilities. A 66% rise in high- and medium-severity findings is only good news if the extra findings are true positives. Between October 23 and November 3, teams ran more than 13,000 Deep Scans across more than 2,700 organizations, so the volume of findings at stake is large — and a model that cries wolf more often will be tuned out by the developers it is supposed to help. Any team adopting GPT-6 Cyber for review should measure precision, not just recall, before trusting the count.",
          "cites": [
            {
              "manifest_id": "1793910947336909136",
              "claim_ids": [
                "1793911197910705370",
                "1793911197917925863"
              ]
            }
          ]
        },
        {
          "markdown": "Merge, a startup that builds product and business tools with OpenAI technology, offers a useful counterpoint from the defensive side of the same week. It uses GPT-6 Astra, ChatGPT, and Codex across product reviews, marketing, and business development planning, and reports that tool calls through its own systems succeed 99.9% of the time, compared with around 60% for public MCP servers. Its Merge Agent Handler provides secure tool access for agents, and its Gateway provides model routing and cost controls. The implication is that the reliability gap in agent tooling is currently a plumbing problem as much as a model problem — and the plumbing is where a builder can actually control the outcome.",
          "cites": [
            {
              "manifest_id": "1793922749815035285",
              "claim_ids": [
                "1793923005745037710",
                "1793923005755037716",
                "1793923005756037717",
                "1793923005748037712"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "What it means for developers",
      "paragraphs": [
        {
          "markdown": "For a developer deciding whether to put a security-tuned model into a review pipeline, the week's evidence supports a narrow conclusion: GPT-6 Cyber is good at finding known classes of weakness and completing staged attack scenarios, and it is not a skeleton key. CyScenarioBench measures multi-stage offensive operations over long horizons, and Atomic Challenges measure focused skills in vulnerability research, network attack simulation, and evasion — exactly the skills that matter for an attacker working through a real system, and exactly where the model still hits a ceiling on hardened targets.",
          "cites": [
            {
              "manifest_id": "1793909926967313802",
              "claim_ids": [
                "1793910275830936070",
                "1793910275831173949"
              ]
            }
          ]
        },
        {
          "markdown": "The practical move is to treat the model as a first pass with a human in the loop, and to track how many of its findings survive review. Merge's chief executive Shensi Ding advises executives to experiment with AI in low-risk areas before scaling, which is the right posture here: run the security model on non-critical repositories first, measure the true-positive rate, and only then let it near the code that matters. A model that finds 66% more issues is only an improvement if the extra issues are real.",
          "cites": [
            {
              "manifest_id": "1793910947336909136",
              "claim_ids": [
                "1793911197910705370"
              ]
            },
            {
              "manifest_id": "1793922749815035285",
              "claim_ids": [
                "1793923005766037724"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "How many of CodeRabbit's additional GPT-6 Cyber findings are real vulnerabilities rather than false positives.",
    "Whether GPT-6 Cyber's advantage on multi-stage operations holds against targets harder than the current benchmarks.",
    "What the Daybreak Defense Network's access terms mean for who can evaluate future security models independently."
  ],
  "decision_card": {
    "useful_for": [
      "Developers deciding whether to put a security-tuned model into code review",
      "Security teams weighing what offensive benchmarks actually prove"
    ],
    "not_for": [
      "Anyone wanting a verdict on whether GPT-6 Cyber is safe to deploy — both evaluators flag open questions"
    ],
    "compare_against": [
      "Abnormal Security survey on agent behavioural monitoring",
      "a16z on AI lowering the cost of exploit development"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-06T00-ai"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793910947336909136",
    "1793909926967313802",
    "1793922749815035285"
  ],
  "manifests": [
    {
      "manifest_id": "1793910947336909136",
      "title": "GPT-6 Cyber Security Evaluation: CodeRabbit's Results",
      "source_name": "coderabbit-private-company",
      "source_url": "https://coderabbit.ai/blog/gpt-6-cyber-security-evaluation",
      "published_date": "2026-11-05"
    },
    {
      "manifest_id": "1793909926967313802",
      "title": "Assessing GPT-6 Cyber Against Offensive Security Benchmarks We evaluated GPT-6 Cyber across FrontierCyber, CyScenarioBench, and Atomic Challenges. It solved all 11 CyScenarioBench scenarios and all 22 Atomic challenges at least once, and 90 of 288 FrontierCyber challenges. We observed no successful attacks on fully hardened targets in our FrontierCyber evaluations. Read more",
      "source_name": "irregular-research",
      "source_url": "https://irregular.com/research/assessing-gpt-6-cyber-against-offensive-security-benchmarks",
      "published_date": "2026-11-05"
    },
    {
      "manifest_id": "1793922749815035285",
      "title": "Merge builds product and business tools with OpenAI technology",
      "source_name": "openai-news",
      "source_url": "https://openai.com/index/merge-builds-with-openai",
      "published_date": "2026-11-05"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793910947336909136",
      "1793909926967313802",
      "1793922749815035285"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793910947336909136",
      "all": "for id in 1793910947336909136 1793909926967313802 1793922749815035285; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-06/gpt6-cyber-security-evaluations/",
    "json": "https://hangingcontext.com/stories/2026-11-06/gpt6-cyber-security-evaluations/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-06/gpt6-cyber-security-evaluations/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-06T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/voice-agents-listen-now/",
      "headline": "Voice agents now judge how you sound, not just what you say. The testing has to change",
      "slug": "voice-agents-listen-now"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/rag-retrieval-trust-gap/",
      "headline": "RAG's problem is trust, not retrieval: poisoned context slips past relevance, preprints find",
      "slug": "rag-retrieval-trust-gap"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/openai-week-gpt61-images-pierre/",
      "headline": "OpenAI shipped a model, an image engine, an acquisition and a security pipeline in one week",
      "slug": "openai-week-gpt61-images-pierre"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/kv-cache-serving-squeeze/",
      "headline": "The KV cache is the serving bottleneck. Four new methods attack it from different sides",
      "slug": "kv-cache-serving-squeeze"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Save what you follow as a Beacon",
    "text": "A Beacon is a saved query that your agent pulls with an explicit lookback window. Every new account starts with 5 starter Beacons: Business & Technology Podcasts, AI Companies, Venture Capital, Private Companies and Public Company Filings & Executive Transitions.",
    "cta": {
      "label": "See Beacons",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-06/gpt6-cyber-security-evaluations/"
  }
}