The document pipeline under AI search is now measured like infrastructure

Parsing now has a severity score

LlamaIndex introduced ParseBench V2 on 12 November, a document parsing benchmark the company describes as an industry standard for evaluating the accuracy of document parsing. The company says ParseBench-2 introduces severity-tuned scoring for document parsing benchmarks, and expands the document corpus to 4,300 cases, including 1,025 charts, 330 error types for measuring failure modes, and forms as a sixth evaluation dimension. The point of severity tuning, as the company presents it, is to penalise parsing errors by their impact on downstream use, not to treat every error as equal. m:…9304

The leaderboard the company reports puts its own product first, so the numbers should be read as the vendor's benchmark. LlamaIndex says LlamaParse Agentic Plus scored 89.8 on ParseBench-2. The company also says chart parsing scores range from 1.2 to 92.6 across evaluated systems, making charts the widest gap in the test, and that direct vision models achieved a median Visual Grounding score of 64.5. LlamaIndex also says it introduced OCRia-1.5, a new family of document parsing models. m:…9304

Fast mode is cheaper, with a trade-off stated plainly

Datalab, writing on 11 November, prices the same pipeline work in plain per-page terms. The company says Fast mode now runs Surya 2.6, its new OCR model, and that the cost with data sharing is $1 per 1,000 pages, down from $3, while the cost without data sharing is $1.33 per 1,000 pages. The company says the median time per page is 2.6 seconds, down from 3.9 seconds, and that the OmniParseBench score for Fast mode is 91.6, up from 90.3. Table extraction accuracy in Fast mode is 91.2, according to the company. m:…8948

Datalab also states where the new model got worse, which is the part of the post a buyer should not skip. The company says handwriting accuracy in Fast mode fell from 69.1 to 54.3, and old scanned text accuracy fell from 72.2 to 63.0. The company says users should use balanced or accurate modes for handwritten documents, old scans, or large-format pages. That is a clear trade-off: the cheaper, faster mode is better on tables and current documents, and worse on the document types where reading is hardest. m:…8948

Sensitive documents need a different test

LlamaIndex's second post, published 11 November, shows what happens when the documents themselves are the attack surface. The company says it uses Trace for agentic dynamic application security testing, and that Trace agents covered the LlamaParse platform in four days. LlamaIndex says hidden text in PDFs can be extracted by LlamaParse even if it is not visible to human readers, and that LlamaParse processes sensitive files including financial filings, insurance claims, and contracts. The company says it fixed or mitigated every issue identified in the platform during the security test. m:…1163

The company's conclusion puts the remaining responsibility on the customer, not the parser. LlamaIndex says it believes the responsibility for deciding which extracted values to trust lies with the customer's pipeline, and recommends treating documents from suppliers, applicants, or policyholders as untrusted input. The company recommends using do_not_cache=true for sensitive files. The message is consistent across the week's posts: parsing quality and security testing are now separate, measurable parts of the same stack, and a team that only measures the model is measuring the wrong thing. m:…1163

The test itself ran on outside muscle. LlamaIndex says Trace runs its agents on frontier models through Anthropic's Cyber Verification Program and OpenAI's Daybreak program, and that its agents covered the LlamaParse platform in four days. A four-day sweep is what makes this repeatable for every release rather than a one-off audit. m:…1163