{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1543",
  "slug": "software-factory-verification-bottleneck",
  "edition": "2026-11-13T12",
  "url": "https://hangingcontext.com/stories/2026-11-13/software-factory-verification-bottleneck/",
  "newsroom": "artificial-intelligence",
  "beat": "developers",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Agents now write whole systems, so the engineer's job has moved from writing code to building the gate that checks it.",
  "headline": "The agents wrote 200 services. Nobody reviewed a line.",
  "dek": "AWS's emulator of 200-plus services was written entirely by agents with no human code review. The companies selling the verification layer are growing tenfold in a year.",
  "published_at": "2026-11-13T12:46:21Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 5 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 5,
  "publishers": [
    "clay-blog",
    "coderabbit-private-company",
    "qodo-publishing",
    "snorkel-ai-research"
  ],
  "sources": [
    "clay-blog",
    "coderabbit-private-company",
    "qodo-publishing",
    "snorkel-ai-research"
  ],
  "distinct_source_count": 4,
  "key_facts": [
    {
      "text": "Qodo reports the AWS Emulator, a local copy of more than 200 AWS services, was written entirely by autonomous agents without human code review.",
      "cites": [
        {
          "claim_ids": [
            "1794539857083057084",
            "1794539857079057083"
          ],
          "manifest_id": "1794539414455053362"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Snorkel reports Fable 5.2 achieved a 71% pass@1 score on 26 frontier coding tasks, against 53% for Fable 5.1.",
      "cites": [
        {
          "claim_ids": [
            "1794510972401846468",
            "1794510972411846470"
          ],
          "manifest_id": "1794510730605846119"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Qodo cites Gartner's prediction that 50% of enterprises adopting autonomous coding agents without real-time guardrails will decommission or restrict them by 2028 due to quality or security incidents.",
      "cites": [
        {
          "claim_ids": [
            "1794499221996763692"
          ],
          "manifest_id": "1794498932867760730"
        }
      ],
      "self_reported": true
    },
    {
      "text": "CodeRabbit says it grew from $10 million to $100 million in annual recurring revenue in just over a year, serving more than 20,000 customers and 400,000 monthly active users.",
      "cites": [
        {
          "claim_ids": [
            "1794504091876794764",
            "1794504091878794765"
          ],
          "manifest_id": "1794503841970793584"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Clay says the AI models it uses now rarely write bugs, and those that are written are typically caught by review agents.",
      "cites": [
        {
          "claim_ids": [
            "1794505131641800658"
          ],
          "manifest_id": "1794504921896799465"
        }
      ],
      "self_reported": true
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "The factory arrives",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794539857083057084",
                "1794539857079057083",
                "1794539857085057085",
                "1794539857087057087"
              ],
              "manifest_id": "1794539414455053362"
            }
          ],
          "markdown": "AWS has released a local testing environment for more than 200 of its services, and the striking part is not the product. It is how it was built. Qodo reports the AWS Emulator was written entirely by autonomous agents without human code review. Instead of line-by-line review, the project relies on a robust automated merge gate that includes security agents performing adversarial reviews. The engineer's job, in this workflow, is to define the rules and build the merge gates. The limits are stated plainly too: the emulator does not evaluate IAM policies the way production AWS does, and AWS advises against using it for security testing. The agents built the system. The humans built the test that decides whether the system ships."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794510972401846468",
                "1794510972411846470",
                "1794510972419846471",
                "1794510972427846473",
                "1794510972455846477"
              ],
              "manifest_id": "1794510730605846119"
            }
          ],
          "markdown": "The models have earned that trust at the level of mechanics. Snorkel reports its Fable 5.2 model achieved a 71 percent pass-at-one score on a set of 26 frontier coding tasks, up from 53 percent for Fable 5.1. Mid-run terminal failures observed in the earlier version are nearly absent, and the remaining failures are concentrated in planning and reasoning — stable misreadings of task requirements — rather than crashes and malformed output. The model recorded 302 instances of self-verification, with 71 percent rated as strong. The agents have largely stopped failing at typing. They fail, now, at understanding what was asked."
        }
      ]
    },
    {
      "heading": "Verification is the bottleneck",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794499221915763690",
                "1794499221949763691",
                "1794499221996763692",
                "1794499222081763695"
              ],
              "manifest_id": "1794498932867760730"
            }
          ],
          "markdown": "Qodo's argument this week is that most organisations calling their setup a software factory are getting the definition wrong. A factory, it says, is a delivery system where AI agents plan, build, review and release software within defined boundaries, checks and evidence — not a workflow with agents in it. The bottleneck in such a system is verification, not code generation, and teams that add agents to every stage without improving review capacity often end up with longer queues. The stakes are quantified by Gartner, cited by Qodo: by 2028, half of enterprises adopting autonomous coding agents without real-time guardrails will decommission or restrict them due to quality or security incidents. Verification, the checklist says, should be proportional to the change's blast radius."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794505131639800657",
                "1794505131641800658",
                "1794505131642800659",
                "1794505131643800660"
              ],
              "manifest_id": "1794504921896799465"
            }
          ],
          "markdown": "Clay, a company that builds with agents daily, describes what that shift looks like inside a review process. It has moved the primary goal of code review from bug-finding to high-level architectural scrutiny — data schemas, service boundaries, system organisation — because, it says, the AI models it uses now rarely write bugs, and those that are written are typically caught by review agents. Complex architectural discussions now have to happen before a pull request is opened, and the developer experience team is building a PR bot to score the risk level of pull requests, with auto-approval for low-risk changes. The review meeting is no longer where bugs are found. It is where the architecture is argued about."
        }
      ]
    },
    {
      "heading": "The verification business",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794504091876794764",
                "1794504091878794765",
                "1794504091887794771",
                "1794504091888794772",
                "1794504091886794770"
              ],
              "manifest_id": "1794503841970793584"
            }
          ],
          "markdown": "The market is paying for the gate rather than the generator. CodeRabbit says it has grown from 10 million to 100 million dollars in annual recurring revenue in just over a year, serving more than 20,000 customers and 400,000 monthly active users, by selling an AI-driven validation layer for software development — independent code review, security and triage. Its Deep Scans program, funded with 1 million dollars of free scans, surfaced more than 43,000 critical or high-severity findings in its first ten days, and the company joined the OpenAI Daybreak Defense Network in October 2026. The week's story, taken together, is not that agents can write code. It is that the industry's scarcest resource is now the check on the code, and the companies that sell the check are the ones compounding."
        }
      ]
    }
  ],
  "unresolved": [
    "Whether merge-gate verification scales to systems larger than the AWS Emulator",
    "Whether architecture-first review holds up when agents write most of the code",
    "Whether the verification layer consolidates into a few vendors or stays fragmented"
  ],
  "decision_card": {
    "not_for": [
      "Shipping agent-written code with no independent check",
      "Calling an agentic workflow a factory before the gates exist"
    ],
    "useful_for": [
      "Engineering leaders building agent-driven delivery",
      "Executives deciding what the engineering org does next"
    ],
    "compare_against": [
      "Your own review queue length against the 9-hour assisted figure",
      "The cost of a verification layer against the cost of a bad merge"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-13T12"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1794539414455053362",
    "1794510730605846119",
    "1794498932867760730",
    "1794503841970793584",
    "1794504921896799465"
  ],
  "manifests": [
    {
      "title": "Agents Wrote All of AWS Emulator's 200+ Services. What's the Engineer's Job Now?",
      "source_url": "https://qodo.ai/blog/aws-emulator-agents-merge-gate",
      "manifest_id": "1794539414455053362",
      "source_name": "qodo-publishing",
      "published_date": "2026-11-12"
    },
    {
      "title": "Fable 5.2 vs Fable 5.1: Coding Benchmark Results",
      "source_url": "https://snorkel.ai/blog/fable-5-2-vs-fable-5-1-coding-benchmark",
      "manifest_id": "1794510730605846119",
      "source_name": "snorkel-ai-research",
      "published_date": "2026-11-12"
    },
    {
      "title": "Software Factory Checklist: 10 Questions to Ask Before You Call It a Factory",
      "source_url": "https://qodo.ai/blog/software-factory-checklist",
      "manifest_id": "1794498932867760730",
      "source_name": "qodo-publishing",
      "published_date": "2026-11-12"
    },
    {
      "title": "CodeRabbit Passes $100 million in ARR",
      "source_url": "https://coderabbit.ai/blog/coderabbit-100-million-arr",
      "manifest_id": "1794503841970793584",
      "source_name": "coderabbit-private-company",
      "published_date": "2026-11-12"
    },
    {
      "title": "Less time on code, more time on architecture",
      "source_url": "https://clay.com/blog/code-reviews-agents",
      "manifest_id": "1794504921896799465",
      "source_name": "clay-blog",
      "published_date": "2026-11-12"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1794539414455053362",
      "1794510730605846119",
      "1794498932867760730",
      "1794503841970793584",
      "1794504921896799465"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1794539414455053362",
      "all": "for id in 1794539414455053362 1794510730605846119 1794498932867760730 1794503841970793584 1794504921896799465; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-13/software-factory-verification-bottleneck/",
    "json": "https://hangingcontext.com/stories/2026-11-13/software-factory-verification-bottleneck/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-13/software-factory-verification-bottleneck/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-13T12/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/nvidia-jensen-exit-deal-machine-stalls/",
      "headline": "Nvidia loses the chief who was its strategy, and the succession race starts without a plan",
      "slug": "nvidia-jensen-exit-deal-machine-stalls"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/enterprise-ai-sold-on-governance/",
      "headline": "Enterprise AI is now sold on governance first, models second",
      "slug": "enterprise-ai-sold-on-governance"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/cutting-ai-bill-four-levers/",
      "headline": "The AI bill is now a line item, and four levers are cutting it without cutting capability",
      "slug": "cutting-ai-bill-four-levers"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-14/agents-need-task-scoped-access/",
      "headline": "Your agents have standing access to everything. The fix is to give them almost nothing, briefly.",
      "slug": "agents-need-task-scoped-access"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Claims with a source link",
    "text": "A Manifest holds a Signal of structured claims and a Brief, plus the source it came from. Your agent can check a sentence against it.",
    "cta": {
      "label": "Read the guide",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-13/software-factory-verification-bottleneck/"
  }
}