{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1365",
  "slug": "social-engineering-agents",
  "edition": "2026-11-10T12",
  "url": "https://hangingcontext.com/stories/2026-11-10/social-engineering-agents/",
  "newsroom": "artificial-intelligence",
  "beat": "policy",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Frontier agents are easy to socially engineer, and a single clean retest proves nothing because the same attack often works on the next try.",
  "headline": "Agents fall for social engineering, and one clean retest proves nothing",
  "dek": "A new benchmark finds frontier models verify suspicious requests less than a third of the time, while other research shows attacks that fail once can succeed on repeat.",
  "published_at": "2026-11-10T12:44:58Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 6 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 6,
  "publishers": [
    "arxiv-ai-security-privacy-safety",
    "centific-blog",
    "lumenova-ai-blog"
  ],
  "sources": [
    "arxiv-ai-security-privacy-safety",
    "centific-blog",
    "lumenova-ai-blog"
  ],
  "distinct_source_count": 3,
  "key_facts": [
    {
      "text": "The SEA benchmark's five-dimensional evaluation framework across eight daily-life domains finds frontier models show proactive verification rates below 32% under social engineering attacks.",
      "cites": [
        {
          "claim_ids": [
            "1794302716414658515",
            "1794302716416658516"
          ],
          "manifest_id": "1794302247390656027"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In noisy cybersecurity investigations, language agents reach 0.82 to 0.88 recall but only 0.61 to 0.72 precision.",
      "cites": [
        {
          "claim_ids": [
            "1794302710647657811"
          ],
          "manifest_id": "1794302247390656036"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Only 278 of 660 identified attack-safeguard relationships in agentic AI have been directly assessed in existing literature.",
      "cites": [
        {
          "claim_ids": [
            "1794302700087657106"
          ],
          "manifest_id": "1794302247390656028"
        }
      ],
      "self_reported": true
    },
    {
      "text": "NIST researchers found repeating the same attack 25 times raised its success rate from 57% to 80%, and the Cloud Security Alliance states a single refusal does not prove resilience.",
      "cites": [
        {
          "claim_ids": [
            "1794245015522277245",
            "1794245015524277246"
          ],
          "manifest_id": "1794244618298274128"
        }
      ],
      "self_reported": false
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Agents do not check before they act",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794302716414658515",
                "1794302716416658516"
              ],
              "manifest_id": "1794302247390656027"
            }
          ],
          "markdown": "A new benchmark for social engineering attacks on AI agents, with a five-dimensional evaluation framework across eight daily-life domains, finds that current frontier models are highly susceptible. The models tested showed proactive verification rates below 32% when subjected to social engineering attacks, meaning that in most cases the agent did not independently check a suspicious request before acting on it. That is the number that should make any team deploying a customer-facing agent pause: verification is the difference between an agent that checks and one that obeys."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794302716417658517"
              ],
              "manifest_id": "1794302247390656027"
            }
          ],
          "markdown": "There is a constructive result in the same paper. The authors propose META-SEA, a co-evolutionary training framework, and report that it enables open-source models like Qwen3-8B to achieve defense performance comparable to closed-source models. That matters because it suggests the vulnerability is not a permanent property of open models: training that pairs attackers and defenders can close much of the gap. But the baseline result stands on its own. An agent that verifies less than a third of the time is an agent that will follow a plausible instruction from a stranger, and most production agents face strangers all day."
        }
      ]
    },
    {
      "heading": "The threat map is bigger than prompt injection",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794302700079657105",
                "1794302700095657107"
              ],
              "manifest_id": "1794302247390656028"
            }
          ],
          "markdown": "A systematization of the attack literature overturns a common assumption about where the risk sits. Tool, MCP and action-layer attacks account for the largest share of atomic attacks in agentic AI systems, while the prompt-injection and action-hijacking class has the broadest paper coverage. In other words, researchers write the most about prompt injection, but the attacks that actually exist in the largest numbers live at the tool and action layer, where an agent does things rather than says things."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794302700087657106"
              ],
              "manifest_id": "1794302247390656028"
            },
            {
              "claim_ids": [
                "1794302710643657809",
                "1794302710647657811"
              ],
              "manifest_id": "1794302247390656036"
            }
          ],
          "markdown": "The same study finds that only 278 of 660 identified attack-safeguard relationships have been directly assessed in existing literature, so most proposed defenses have never been tested against the attacks they claim to stop. A separate benchmark for sparse-signal cybersecurity investigation, with 74 adversarial scenarios, shows what untested defenses look like in practice: current agent architectures exhibit high recall of 0.82 to 0.88 but low precision of 0.61 to 0.72 when operating in noisy environments. The agents find most of the real threats and also flag a large number of false alarms, and those failures are consistent across reasoning tiers and are not fixed by varying confounder density or event volume."
        }
      ]
    },
    {
      "heading": "Attackers now use agents too",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794302701815657225",
                "1794302701826657226"
              ],
              "manifest_id": "1794302247388656013"
            }
          ],
          "markdown": "The red-teaming side is also automating. A preprint on scaling adaptive prompt injection attacks with agent swarms shows that agentic attackers can reliably trick models into following instructions in tool output, even for models previously considered robust in the GraySwan Arena. Human red-teaming for prompt injection is nearing saturation for the Gemini model family, the authors say, which means human testers are finding fewer new failures while automated attackers keep finding them. The authors argue that static benchmarks are insufficient and that proactive, automated red-teaming is essential for future agent security."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794302701846657227"
              ],
              "manifest_id": "1794302247388656013"
            },
            {
              "claim_ids": [
                "1794245015525277247",
                "1794245015527277248"
              ],
              "manifest_id": "1794244618298274128"
            }
          ],
          "markdown": "One methodological idea in that paper is worth noting: evaluating prompt injection robustness through non-malicious actions separates instruction-hierarchy adherence from general security knowledge, so a failure tells you the model followed the wrong instruction rather than that it lacks safety knowledge. Lumenova makes the same point from the testing side, arguing that red teaming for AI agents should include indirect attack paths like documents, emails and tool results, and that automated penetration testing of agents requires checking automated judge verdicts against human reviews. An agent's most dangerous input is often not the user's message but the document the user asked it to read."
        }
      ]
    },
    {
      "heading": "A clean retest is not proof",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794245015514277244",
                "1794245015522277245"
              ],
              "manifest_id": "1794244618298274128"
            }
          ],
          "markdown": "The most practical finding in this batch concerns how teams verify fixes. AI agents are non-deterministic, Lumenova argues, meaning an attack that fails once may succeed on a later try without any changes to the system. NIST researchers found that repeating the same attack 25 times increased the success rate from 57% to 80%, and the Cloud Security Alliance states that a single refusal does not prove resilience. Standard penetration testing relies on clean retests to verify security fixes, but for agents that approach is unreliable: a successful retest may simply mean the attack failed by chance."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794245015528277249",
                "1794245015531277251",
                "1794245015530277250"
              ],
              "manifest_id": "1794244618298274128"
            },
            {
              "claim_ids": [
                "1794244832523275373",
                "1794244832543275376",
                "1794244832495275372",
                "1794244832570275383"
              ],
              "manifest_id": "1794244463574273585"
            }
          ],
          "markdown": "Lumenova's prescription is to document the specific rule or control that blocked the attack, because a rule on an action holds regardless of how the instruction is worded, and to require retesting after any change to the agent, its tools or its model. Untested attack categories should be reported as untested rather than passed. Centific's PAVE benchmark makes the same point for personal assistants: it tests task completion and restraint by checking action logs, device data and final replies, because transcript-based evaluations can fail to detect failed saves, dropped preferences and unauthorized actions. In one case an assistant deleted all three reminders after a hidden instruction in a note, and GPT-4o-mini's verdict differed from PAVE's on 7 of 20 tasks when viewing only the final reply. Taken together, the sources point to a testing standard that looks very different from current practice: repeated attacks, verified through rules rather than refusals, across the tool layer where most attacks actually live, with the honest admission that most safeguards have never been assessed at all."
        }
      ]
    }
  ],
  "unresolved": [
    "Whether META-SEA training generalizes beyond the SEA benchmark's eight domains to real customer-facing deployments.",
    "How attack success rates change when agents have access to real money movement or account changes, rather than benchmark tasks.",
    "Which safeguards among the 382 unassessed attack-safeguard pairs actually work in production."
  ],
  "decision_card": {
    "not_for": [
      "Teams evaluating agents only on task success rates",
      "Anyone treating a refused attack as a fixed vulnerability"
    ],
    "useful_for": [
      "Security teams red-teaming production agents",
      "Executives approving customer-facing agents",
      "Compliance teams setting agent test standards"
    ],
    "compare_against": [
      "Single-run red-team pass rates as proof of safety",
      "Prompt-injection filters as the whole defense"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-10T12"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1794302247390656027",
    "1794302247390656036",
    "1794302247390656028",
    "1794244618298274128",
    "1794302247388656013",
    "1794244463574273585"
  ],
  "manifests": [
    {
      "title": "Toward Safety-Aware AI Agents in Realistic Social Engineering Attack Scenarios",
      "source_url": "https://arxiv.org/pdf/2611.10037v1",
      "manifest_id": "1794302247390656027",
      "source_name": "arxiv-ai-security-privacy-safety",
      "published_date": "2026-11-10"
    },
    {
      "title": "LEGION: Benchmarking Language Agents on Sparse-Signal Investigation Under Adversarial Noise",
      "source_url": "https://arxiv.org/pdf/2611.11896v1",
      "manifest_id": "1794302247390656036",
      "source_name": "arxiv-ai-security-privacy-safety",
      "published_date": "2026-11-10"
    },
    {
      "title": "Agents of Chaos: A Systematization of Knowledge (SoK) on Malicious Attacks on Agentic AI Systems",
      "source_url": "https://arxiv.org/pdf/2611.10843v1",
      "manifest_id": "1794302247390656028",
      "source_name": "arxiv-ai-security-privacy-safety",
      "published_date": "2026-11-10"
    },
    {
      "title": "AI Agents Penetration Testing: Why a Clean Retest Isn't Proof",
      "source_url": "https://lumenova.ai/blog/penetration-testing-for-ai-agents",
      "manifest_id": "1794244618298274128",
      "source_name": "lumenova-ai-blog",
      "published_date": "2026-11-09"
    },
    {
      "title": "Scaling Adaptive Prompt Injection Attacks with Agent Swarms",
      "source_url": "https://arxiv.org/pdf/2611.11861v1",
      "manifest_id": "1794302247388656013",
      "source_name": "arxiv-ai-security-privacy-safety",
      "published_date": "2026-11-10"
    },
    {
      "title": "How to evaluate AI personal assistants for task completion and safety",
      "source_url": "https://centific.com/blog/ai-personal-assistant-evaluation",
      "manifest_id": "1794244463574273585",
      "source_name": "centific-blog",
      "published_date": "2026-11-09"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1794302247390656027",
      "1794302247390656036",
      "1794302247390656028",
      "1794244618298274128",
      "1794302247388656013",
      "1794244463574273585"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1794302247390656027",
      "all": "for id in 1794302247390656027 1794302247390656036 1794302247390656028 1794244618298274128 1794302247388656013 1794244463574273585; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-10/social-engineering-agents/",
    "json": "https://hangingcontext.com/stories/2026-11-10/social-engineering-agents/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-10/social-engineering-agents/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-10T12/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/gpu-cold-start-warm-pools/",
      "headline": "The GPU bottleneck moved from buying chips to starting them",
      "slug": "gpu-cold-start-warm-pools"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/ai-rules-arrive-from-every-direction/",
      "headline": "AI rules are arriving from every direction at once",
      "slug": "ai-rules-arrive-from-every-direction"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/agent-incidents-66-days-undisclosed/",
      "headline": "66 days to daylight: agent incidents surface late, and usually by accident",
      "slug": "agent-incidents-66-days-undisclosed"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-10/agent-runtime-security-controls/",
      "headline": "The kill switch is not enough: agent security moves to the runtime",
      "slug": "agent-runtime-security-controls"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Claims with a source link",
    "text": "A Manifest holds a Signal of structured claims and a Brief, plus the source it came from. Your agent can check a sentence against it.",
    "cta": {
      "label": "Read the guide",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-10/social-engineering-agents/"
  }
}