The red teams price their attacks now: jailbreaks measured by cost, post-generation checkers bypassed, email agents phished

Attack success gets a price tag

Security researchers are starting to price jailbreaks the way attackers actually experience them. In a preprint posted October 23, Rohan Menon and Chih-Hong Cheng introduce ASR_STEP, a metric that penalizes iterative jailbreak strategies for every call they make to the victim model. The logic is operational, not academic: repeated queries raise the chance of detection and intervention in production, so an attack that needs hundreds of tries is less dangerous than its headline success rate suggests. m:…0373

When the authors re-scored JailBreakBench, the popular iterative attacks GCG and PAIR came out less effective than previously reported - GCG down 7.0 to 8.7 percentage points, PAIR down 3.4 to 12 - while single-step attacks like PAP were unaffected by the new metric. That draws a line that matters for budgets: threat profiles that look similar on a leaderboard behave very differently under real monitoring. This suggests the relevant question for a security team is not whether the model can be jailbroken at all, but how expensive each successful jailbreak is, and whether production monitoring catches the attempt before it lands. m:…0373

The safety checker is now the target

A second preprint, Mirage, goes after a part of the pipeline most teams assumed was their last line of defense. Shanghao Shi and co-authors demonstrate a multimodal jailbreak that bypasses post-generation safety checkers in diffusion models - including Stable Diffusion 1.5, SDXL, and Kandinsky 2.2 - by concealing adversarial targets within benign input images. The attack exploits shared representation spaces so a perturbed image lines up with the adversarial target while staying clear of the concepts the checkers scan for, and it needs only limited black-box queries with no internal model access. m:…7086

The implication is uncomfortable for anyone who placed a checker at the end of the pipeline and called the system safe. If the checker itself can be evaded as one more component in the attack path, then bolting a filter onto the output is a strategy with a known, published workaround. m:…7086

Email agents keep getting phished

The agent version of the same story comes from AgentPhishing, a benchmark by Justin Tan, Miao Li, and Ding Zhao that tests LLM email agents against adaptive, persuasive attacks instead of static datasets. The authors built PhishOpt, a method that optimizes phishing emails against each individual agent, and the results are blunt: of nine agents evaluated, four detected fewer than 10% of phishing attempts while complying with malicious requests more than 45% of the time, even though the agents had passed conventional phishing evaluations. m:…6385

As Zico Kolter, the Gray Swan co-founder and chief scientist and an OpenAI board member interviewed on the Madrona blog on October 22, put it: "Prompt injection is analogous to phishing for AI agents." The AgentPhishing numbers put a price on that analogy. Static defenses that pass their own tests can still fold when the attacker adapts to the specific agent reading the inbox. m:…5662 m:…6385

The defense is being measured too

The fourth preprint is on the other side of the ledger. MPIDetector, by Yiheng Li and Jingyue Li, uses LLM-based reasoning with few-shot in-context learning to spot malicious prompt injections across text and visual inputs, without fine-tuning. The authors report 90.91% accuracy with 100% recall on textual prompt injection, 99.28% accuracy with 100% precision on visual prompt injection, and a 99.80% detection rate against 500 highly obfuscated web-agent injections in the WebInject set. m:…1748

Those numbers are the authors' own, on the authors' own benchmarks, in a preprint that has not been peer-reviewed - and that caveat applies to all four papers. But the pattern is the point: defenses are now being scored with the same cost-and-coverage discipline that the attackers just got. The arms race has a bookkeeping standard. m:…1748 m:…0373

What this changes for budgets

Kolter's broader point gives the batch its frame. AI security, he told the Madrona blog, splits into two jobs - hardening infrastructure against attackers and addressing the vulnerabilities inside the model itself - and red-teaming the model is distinct from standard penetration testing. His recommendation is to red-team the model itself, separately from the software around it, and to test each new capability adversarially before it ships. m:…5662

This suggests the planning move for an AI executive: measure security against operations, not leaderboards. Cost-aware metrics like ASR_STEP change which attacks count as credible threats; benchmarks like AgentPhishing change which agent deployments are ready for real inboxes; and defenses like MPIDetector get scored the same way. Kolter's framing - that prompt injection cannot be fully patched away - implies the budget follows mitigation and detection, not prevention alone. The honest caveat: every number above is self-reported by its authors, on their own benchmarks, in papers that have not yet been peer-reviewed. m:…5662 m:…0373