{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1218",
  "slug": "offense-is-automated",
  "edition": "2026-11-07T00",
  "url": "https://hangingcontext.com/stories/2026-11-07/offense-is-automated/",
  "newsroom": "artificial-intelligence",
  "beat": "models-labs",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Attacking AI systems is now automated and cheap; defense has to assume breach",
  "headline": "Attacking AI systems is now automated, cheap, and guided by experts",
  "dek": "Open-weight models beat frontier models on a budget, an open-source pentest tool jumps from 2 findings to 17 with an expert plan, and preprints show red-teaming and task-tampering are now automated.",
  "published_at": "2026-11-07T00:39:07Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 5 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 5,
  "publishers": [
    "arxiv-ai-security-privacy-safety",
    "arxiv-code-devtools-ai",
    "simbian-blog",
    "xbow-blog"
  ],
  "sources": [
    "arxiv-ai-security-privacy-safety",
    "arxiv-code-devtools-ai",
    "simbian-blog",
    "xbow-blog"
  ],
  "distinct_source_count": 4,
  "key_facts": [
    {
      "text": "XBOW says DeepSeek 4.2 Flash outperforms more expensive and restricted models at lower budgets.",
      "cites": [
        {
          "claim_ids": [
            "1794008935993507990"
          ],
          "manifest_id": "1794008728111507270"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Simbian says ARTEX AI confirmed 2 findings in a default run and 17 findings when given an expert-written phased test plan.",
      "cites": [
        {
          "claim_ids": [
            "1794004998216493188",
            "1794004998220493189"
          ],
          "manifest_id": "1794004673277492228"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, the ARMs red-teaming agent reports a 52.1% average improvement in attack success rates and over 90% on Claude-4-Sonnet.",
      "cites": [
        {
          "claim_ids": [
            "1793959090907232097"
          ],
          "manifest_id": "1793958775303230843"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, obfuscation attacks were the most effective jailbreaking family across seven open-weight models, with a pooled success rate of 62.3%.",
      "cites": [
        {
          "claim_ids": [
            "1793959091179232167"
          ],
          "manifest_id": "1793958775303230842"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, tool-policy defenses against task-tampering yielded a 15.4% attack success rate, against a 17.0% baseline.",
      "cites": [
        {
          "claim_ids": [
            "1793942189997156160"
          ],
          "manifest_id": "1793941692588152478"
        }
      ],
      "self_reported": true
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Cost decides which attacker model wins",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794008935987507987",
                "1794008935989507988",
                "1794008935991507989"
              ],
              "manifest_id": "1794008728111507270"
            }
          ],
          "markdown": "XBOW's latest offensive security evaluations deliver an awkward result for anyone who assumes the most capable model is the best attacker. The company says the best model for offensive security depends on the available budget, and the most capable model is not necessarily the one that finds the most vulnerabilities. Frontier models such as Anthropic's Opus 5.5 and OpenAI's GPT-6.1 Sol show tangible improvements, but their advantages only emerge at higher spending levels."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794008935993507990",
                "1794008935994507991",
                "1794008935995507992",
                "1794008935997507993"
              ],
              "manifest_id": "1794008728111507270"
            }
          ],
          "markdown": "At lower budgets, open-weight models win. XBOW says DeepSeek 4.2 Flash outperforms more expensive and restricted models at lower budgets, and is effective at finding vulnerabilities without extensive exploration. GLM-5.3, by contrast, is the marathon runner to DeepSeek's sprinter: more effective on longer-horizon tasks because it persists through repeated failures, but approximately four times as expensive per token."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794008935998507994",
                "1794008936000507995",
                "1794008936001507996"
              ],
              "manifest_id": "1794008728111507270"
            }
          ],
          "markdown": "The implication is about control. XBOW argues that restricting access to the most capable models is an incomplete approach to managing security risks, because open-weight models offer an alternative that is harder to control through provider-side access policies. The company's advice is to evaluate models against specific workflows and budgets rather than relying on general benchmarks or model rankings."
        }
      ]
    },
    {
      "heading": "Expert plans beat autonomous runs",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794004998216493188",
                "1794004998220493189",
                "1794004998222493190"
              ],
              "manifest_id": "1794004673277492228"
            }
          ],
          "markdown": "Simbian's test of ARTEX AI, an open-source agentic pentesting framework, makes the same point from the attacker's side. A default run on a simple goal confirmed 2 medium findings. Given an expert-written phased test plan, the same tool confirmed 17 findings, including a critical vulnerability. Simbian's Varun Kondagadapa says the gap between the two runs starts with the input, and that attackers write plans too."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794004998232493197",
                "1794004998223493191",
                "1794004998225493192",
                "1794004998227493193",
                "1794004998235493199"
              ],
              "manifest_id": "1794004673277492228"
            }
          ],
          "markdown": "The tool is already in attackers' hands. Microsoft reported in October 2026 that Storm-0940 used ARTEX to attempt signing in to over 2,250 tenants, and attackers have used ARTEX against South Korean financial organizations. CERT-In issued a critical advisory on November 4, 2026, on staging systems and the ARTEX family, and CERT-EU said public availability of ARTEX will likely speed up adoption by threat actors. CERT-In states that tools in the ARTEX class execute complete attack chains autonomously at machine speed."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794004998236493200",
                "1794004998239493202",
                "1794004998228493194",
                "1794004998233493198"
              ],
              "manifest_id": "1794004673277492228"
            }
          ],
          "markdown": "Simbian's own AI Pentest Agent found 138 findings on the same target used for ARTEX testing. The original ARTEX repository was taken down in October 2026, but forks persist, and Simbian audited the code of the English-language fork BaaRai/ARTEX. Google Threat Intelligence Group assessed in October 2026 that adversary use of open-source AI tools is currently low in sophistication, which reads less as reassurance than as a statement about timing."
        }
      ]
    },
    {
      "heading": "Red-teaming is now automated",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793959090918232100",
                "1793959090907232097",
                "1793959090915232098"
              ],
              "manifest_id": "1793958775303230843"
            }
          ],
          "markdown": "In a preprint, researchers describe ARMs, an adaptive red-teaming agent for vision-language models that uses 17 red-teaming algorithms integrated through the Model Context Protocol. The paper reports a 52.1% average improvement in attack success rates over existing red-teaming baselines, and an attack success rate above 90% on Claude-4-Sonnet. Its benchmark, ARMs-Bench, contains over 30,000 red-teaming instances spanning 51 risk categories."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793959091179232167",
                "1793959091181232168",
                "1793959091183232169"
              ],
              "manifest_id": "1793958775303230842"
            }
          ],
          "markdown": "A second preprint systematically evaluated jailbreaking attacks across seven open-weight large language models. Obfuscation attacks were the most effective family, with a pooled success rate of 62.3%, and model size did not significantly predict adversarial robustness. The encouraging finding is that self-reminder prompting reduced the attack success rate by 40 percentage points on both Mistral-7B and Llama-3.1-8B, a low-overhead defense that works where scale does not."
        }
      ]
    },
    {
      "heading": "The quieter attack: tampering with the task",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793942189980156158"
              ],
              "manifest_id": "1793941692588152478"
            }
          ],
          "markdown": "A third preprint identifies a subtler threat than instruction override: task-tampering, where an attacker manipulates the outcome of a benign task by influencing decision rules, available actions, or state beliefs. The authors built a 90-case benchmark across three tool-use environments, and found attacks targeting decision rules, available actions, and state have higher mean success rates than attacks targeting the agent's primary objective."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793942189991156159",
                "1793942189997156160"
              ],
              "manifest_id": "1793941692588152478"
            }
          ],
          "markdown": "Existing defenses struggle with this class. Detection-and-removal defenses cut objective-targeting attack success to 2.4%, but left observation-targeting attacks at 9.5% and state-targeting attacks at 12.1%. Tool-policy defenses were largely ineffective, yielding a 15.4% attack success rate against a 17.0% baseline. The implication for builders is that agents which consume external data or tool outputs need defenses aimed at how decisions are made, not only at what the agent was told to do."
        }
      ]
    }
  ],
  "unresolved": [
    "Specific performance thresholds where frontier models definitively overtake open-weight models for all security tasks are unresolved.",
    "No explicit link to a public code repository was provided for ARMs.",
    "The task-tampering benchmark is limited to three tool-use environments.",
    "The jailbreak study's content judge has not yet been validated against human labels."
  ],
  "decision_card": {
    "not_for": [
      "Anyone assuming restricting frontier model access is a sufficient defense"
    ],
    "useful_for": [
      "Security teams choosing models for offensive testing on a budget",
      "Developers hardening agents that consume external data"
    ],
    "compare_against": [
      "XBOW cost-per-finding across models",
      "ARMs-Bench results for vision-language models"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-07T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1794008728111507270",
    "1794004673277492228",
    "1793958775303230843",
    "1793958775303230842",
    "1793941692588152478"
  ],
  "manifests": [
    {
      "title": "Top AI Models for Offensive Security, Fall 2026: Why Cost Changes Everything",
      "source_url": "https://xbow.com/blog/top-ai-models-for-offensive-security-fall-2026",
      "manifest_id": "1794008728111507270",
      "source_name": "xbow-blog",
      "published_date": "2026-11-06"
    },
    {
      "title": "ARTEX AI Found 2 Flaws Alone, 17 With an Expert Plan",
      "source_url": "https://simbian.ai/blog/artex-ai-lab-test",
      "manifest_id": "1794004673277492228",
      "source_name": "simbian-blog",
      "published_date": "2026-11-06"
    },
    {
      "title": "ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks",
      "source_url": "https://arxiv.org/pdf/2510.02677v2",
      "manifest_id": "1793958775303230843",
      "source_name": "arxiv-ai-security-privacy-safety",
      "published_date": "2026-11-06"
    },
    {
      "title": "Systematic Evaluation of Adversarial Jailbreaking Attacks and Defenses Across Seven Large Language Models",
      "source_url": "https://arxiv.org/pdf/2610.39509v4",
      "manifest_id": "1793958775303230842",
      "source_name": "arxiv-ai-security-privacy-safety",
      "published_date": "2026-11-06"
    },
    {
      "title": "Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks",
      "source_url": "https://arxiv.org/pdf/2611.07640v1",
      "manifest_id": "1793941692588152478",
      "source_name": "arxiv-code-devtools-ai",
      "published_date": "2026-11-06"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1794008728111507270",
      "1794004673277492228",
      "1793958775303230843",
      "1793958775303230842",
      "1793941692588152478"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1794008728111507270",
      "all": "for id in 1794008728111507270 1794004673277492228 1793958775303230843 1793958775303230842 1793941692588152478; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-07/offense-is-automated/",
    "json": "https://hangingcontext.com/stories/2026-11-07/offense-is-automated/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-07/offense-is-automated/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-07T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/agent-memory-dream-compress-govern/",
      "headline": "Agents are learning to remember — by dreaming, compressing, and governing what they keep",
      "slug": "agent-memory-dream-compress-govern"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/rl-training-for-everyone/",
      "headline": "Reinforcement learning is becoming rentable infrastructure",
      "slug": "rl-training-for-everyone"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/retrieval-becomes-memory/",
      "headline": "Retrieval Is Being Rebuilt as Memory, Not Search",
      "slug": "retrieval-becomes-memory"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/models-in-megabytes/",
      "headline": "Useful AI models now fit in megabytes and run on phones and robots",
      "slug": "models-in-megabytes"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Ask what Synorb covers",
    "text": "The MCP tool synorb-stream-search answers questions like \"do you have this publisher?\" without using any Manifest allowance.",
    "cta": {
      "label": "Connect and ask",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-07/offense-is-automated/"
  }
}