---
title: Debugging agents means querying traces, not scrolling logs
description: Braintrust, LangChain, Databricks and Andesite all point the same way: keep full traces, query them like data, and let humans judge what they show.
canonical: https://hangingcontext.com/stories/2026-11-04/debugging-agents-queryable-traces/
last-updated: 2026-11-04
url: https://hangingcontext.com/stories/2026-11-04/debugging-agents-queryable-traces/
published: 2026-11-04T00:44:46Z
edition: 2026-11-04T00
newsroom: artificial-intelligence
beat: developers
trust: confirmed
source_count: 4
cited_manifest_ids: ['1793746603911950390', '1793743309826466556', '1793748537581021644', '1793746596619817612']
---

# Debugging agents means querying traces, not scrolling logs

*Braintrust, LangChain, Databricks and Andesite all point the same way: keep full traces, query them like data, and let humans judge what they show.*

Confirmed · 4 Manifests · Artificial Intelligence · Published 2026-11-04 00:44 UTC

## Key facts

- Traditional tracing products sample heavily and discard traces quickly, while agent traces serve as the authoritative record for how an agent behaves during long-running, probabilistic tasks. (self-reported) [cites: 1793746603911950390]
- Braintrust's Trace Lake supports fast ingest, SQL-like querying of nested fields and continuous analysis; the company says full-text searches in production A/B tests ran more than twice as fast with its Nitro engine. (self-reported) [cites: 1793746603911950390]
- LangSmith lets queue items be single runs or full multi-turn threads, and human-labeled examples can align LLM judges, update online evaluators and build golden datasets for continuous testing. (self-reported) [cites: 1793743309826466556]
- Fetch Rewards reviewers inspect trace metadata to verify tool usage and data grounding, check product-card images against represented products, and confirm offer availability; engineers use reviewer reasoning to tune automated judge logic. (self-reported) [cites: 1793743309826466556]
- Claude Platform now supports native OpenTelemetry export to Databricks MLflow, where Unity Catalog controls access to trace data that can be queried with SQL. (self-reported) [cites: 1793748537581021644]

## Sampling breaks down

Traditional tracing products sample heavily and discard traces quickly, Braintrust says, and that approach was designed for deterministic, multi-service software, where the goal was understanding bottlenecks. Agent work is different: agent traces serve as the authoritative record for how an agent behaves during long-running, probabilistic tasks. When an agent's answer depends on a chain of tool calls and intermediate steps, a sampled log leaves the team guessing about the steps that were thrown away. [cites: 1793746603911950390]

Braintrust's answer is a Trace Lake that supports fast ingest, SQL-like querying of nested fields and continuous analysis, the company says, with Topics to classify traces, Patterns to investigate recurring issues and Loop automations for scheduled investigations. Its Nitro engine separates waiting for storage from processing the data it returns; the company says full-text searches in production A/B tests ran more than twice as fast as previous iterations, and other queries 15% faster. Notion's AI modeling team uses Brainstore to run searches in seconds that previously took hours, the company says. [cites: 1793746603911950390]

## Humans still have to read the thread

Querying only helps if someone can judge what the trace shows. LangChain argues human expertise is required to identify quality and explain why an interaction succeeded or failed, and reviewing full threads lets experts see user intent and the agent's decision path. An agent trace records user messages, model responses, tool calls and intermediate steps. Flagging a trace provides a signal but does not explain how to change agent behavior, a distinction that matters when a team must decide what to fix. [cites: 1793743309826466556]

> **From Synorb** — Claims with a source link. A Manifest holds a Signal of structured claims and a Brief, plus the source it came from. Your agent can check a sentence against it. [Read the guide](https://synorb.com/connect/start)


LangSmith lets queue items be single runs or full multi-turn threads, and human-labeled examples can align LLM judges, update online evaluators and build golden datasets for continuous testing. Fetch Rewards uses annotation queues to assess its AI Rewards Assistant, the company says, with reviewers inspecting trace metadata to verify tool usage and data grounding, checking product-card images against represented products and confirming offer availability for specific users. Engineers then use reviewer reasoning to tune automated judge logic. [cites: 1793743309826466556]

## Traces that other systems can query

Databricks' MLflow now takes native OpenTelemetry export from the Claude Platform, the company says: Claude's instrumentation generates spans exported using the OpenTelemetry Protocol, Databricks receives them through Zerobus Ingest and stores them in Unity Catalog tables linked to an MLflow experiment, where Unity Catalog controls access and the data can be queried with SQL. MLflow's Analyze trace action uses Genie Code to investigate agent executions, Issues in beta uses it to identify recurring quality and operational problems, and Review Queues let domain experts assess recorded requests and responses. [cites: 1793748537581021644]

Andesite pushes the same idea further into security work: its agents form hypotheses, collect supporting or contradicting claims and link every claim to specific query results, with an evidence panel in the chat interface and audit logs recording high-level system and user actions, the company says. It cites a figure that 81% of CISOs worry AI agents have too much access. This suggests debugging is becoming less about reading logs and more about querying evidence, but the evidence is only as good as the traces teams keep, and the tools for keeping them are still largely sold as vendor platforms. [cites: 1793746596619817612]

## What is still unresolved

- Do OpenTelemetry-based traces stay portable across vendors, or lock into one platform's tables?
- How much of a trace must be kept for audit, versus what debugging alone needs?
- Can automated judges close the gaps human reviewers find, or only narrow them?

## Decision card

- **Useful for:** Teams debugging agents in production; Platform engineers choosing observability tooling; Reviewers setting up human evaluation queues
- **Not for:** Teams running agents only in short demos; Anyone expecting sampled logs to explain a long agent run
- **Compare against:** Sampled tracing products; Manual log review; Automated judges without human-labeled examples

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1793746603911950390` | The Trace Lake | [braintrust-blog](https://braintrust.dev/blog/trace-lake) | 1793746901114441616, 1793746901112041068, 1793746901115195392, 1793746901123162195, 1793746901108212208, 1793746901120771524, 1793746901121241162, 1793746901124460138, 1793746901118531928 | 11-03 00:00 |
| `1793743309826466556` | Why human insight is a strategic advantage for building better agents | [langchain-blog](https://langchain.com/blog/human-insight-strategic-advantage-agents) | 1793743630042265448, 1793743630047451343, 1793743630049035479, 1793743630051097907, 1793743630053024342, 1793743630055431328, 1793743630056129941, 1793743630058514336, 1793743630035435124, 1793743630043432841, 1793743630039351210, 1793743630040413504, 1793743630052330730 | 11-03 00:00 |
| `1793748537581021644` | Claude Platform now supports native OpenTelemetry observability in Databricks MLflow | [databricks-blog](https://databricks.com/blog/claude-platform-now-supports-native-opentelemetry-observability-databricks-mlflow) | 1793748895883605388, 1793748895889798347, 1793748895885161204, 1793748895887708244, 1793748895892562534, 1793748895894537670, 1793748895895672323 | 11-03 00:00 |
| `1793746596619817612` | Evidentiary AI™: A First Principle at Andesite | [andesite-ai-blog](https://andesite.ai/blog/evidentiary-ai-a-first-principle-at-andesite) | 1793746899906876789, 1793746899912010967, 1793746899864151369, 1793746899731389575 | 11-03 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1793746603911950390`
- `1793743309826466556`
- `1793748537581021644`
- `1793746596619817612`

```sh
for id in 1793746603911950390 1793743309826466556 1793748537581021644 1793746596619817612; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Ask what Synorb covers. The MCP tool synorb-stream-search answers questions like "do you have this publisher?" without using any Manifest allowance. [Connect and ask](https://synorb.com/connect/start)

## Related stories

- [Video models are becoming world simulators](https://hangingcontext.com/stories/2026-11-04/video-models-become-world-simulators/)
- [Robot benchmarks are measuring the wrong thing](https://hangingcontext.com/stories/2026-11-04/robot-evaluation-reality-check/)
- [Reasoning effort is becoming a dial, not a default](https://hangingcontext.com/stories/2026-11-04/reasoning-effort-becomes-a-dial/)
- [Open agent models arrive ready to build on](https://hangingcontext.com/stories/2026-11-04/open-agent-models-go-usable/)

## Links

- HTML: https://hangingcontext.com/stories/2026-11-04/debugging-agents-queryable-traces/
- JSON: https://hangingcontext.com/stories/2026-11-04/debugging-agents-queryable-traces/index.json
- Markdown: https://hangingcontext.com/stories/2026-11-04/debugging-agents-queryable-traces/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-11-04/debugging-agents-queryable-traces/. Terms: https://hangingcontext.com/terms/reuse/