AMD's AI Stack Is Filling In, Layer by Layer

Agents running on AMD hardware, in production

The most concrete item in AMD's week is not a chip announcement but a runtime that has already been running. PrimusClaw is an open-source managed agent runtime that runs in Kubernetes clusters next to AMD Instinct GPUs, and it includes a session store, an agent harness and per-session sandboxes. AMD says it has been in production use at AMD since April 2026, and the two deployments have hosted more than 40,000 agent sessions and logged over 100,000 agent-hours, serving more than 70 users. The deployments have processed more than 4 million tool calls and more than 500 billion prompt tokens, with the median run on the main deployment lasting 3.4 hours. m:…6835

Those numbers are AMD's own, and they should be read as a usage report rather than a benchmark, but the shape of them matters. Long-running agent sessions with millions of tool calls are exactly the workload that exposes a weak software stack, because every session needs isolation, state and tooling that survives for hours. A managed runtime that has absorbed that load in production is a different kind of evidence from a launch slide, and it suggests AMD is dogfooding its agent infrastructure on its own clusters before asking anyone else to trust it. m:…6835

Serving: stop guessing, start simulating

Serving a large language model is a configuration problem before it is a hardware problem. LLM serving requires selecting engine, parallel layout, precision and scheduler settings, and there are thousands of potential candidates for LLM serving configurations. Each trial ties up one or more multi-GPU nodes for minutes, and measuring every candidate in searches would take about 25,000 GPU-hours. Most teams currently try a few guesses and keep the best configuration, which is why ROCm InferaSim is designed to cut LLM serving trial costs: an incorrect configuration guess results in extra GPU costs or missed latency targets. m:…6834

Disaggregated serving adds another decision, since it requires a split between prefill and decode, and that split interacts with every other setting. A simulator that searches the configuration space without burning GPU-hours is therefore not a convenience but the difference between a tuned deployment and a guessed one. The implication for teams running on AMD hardware is that the tooling gap is closing at the layer where money is actually spent: not peak flops, but the hours burned finding a configuration that meets a latency target. m:…6834

Kernels that beat the vendor default, and gains that do not survive

At the kernel layer, a preprint called ZYFA reports FP8 attention kernels built for AMD's MI355X that beat the vendor's own AITER kernels. In a preprint, ZYFA kernels achieve 1.16 to 1.76 times the throughput of vendor AITER kernels on AMD MI355X for long-context prefill, and reduce numerical error by 62 to 78% compared with AITER kernels. Ping-pong scheduling allows overlapping softmax computation with matrix multiplications on MI355X, compensating for lack of hardware instruction tracking. For long-context inference, where attention dominates cost, that is a meaningful improvement on AMD's own hardware, reported by researchers rather than by the vendor. m:…9587

A second preprint offers the counterweight, and it is the more useful of the two for anyone planning capacity. FP8 GEMM performance retains high capability gain, but unfused quantization causes per-layer slowdowns in LLM inference, and 2:4 structured sparsity retains at most about half of its theoretical speedup in hipSPARSELt. Shared LLM serving performance is constrained by hardware queue availability rather than numeric precision. The lesson is that theoretical speedups erode as they pass through libraries and shared serving, so a stack is only as good as the layer where the gains leak away. m:…9593

The stack, end to end

The CPU side of the same story is quieter but consistent with the pattern. AMD says optimising BERT-Base CPU inference on EPYC processors with ZenDNN and zentorch delivers up to 47.3% average gains over ONNX Runtime. That is a vendor-reported number for a classic encoder workload, and it matters less for its size than for what it signals: the optimisation work is happening in software libraries, not only in new silicon. m:…0867

Put the pieces together and the picture is a stack filling in from both ends. PrimusClaw covers agent sessions, InferaSim covers serving configuration, ZYFA covers attention kernels, and the capability-gain analysis covers the honesty check on where speedups actually survive. None of this proves AMD's stack matches the incumbent on every workload, and the preprint results still need independent reproduction. But a runtime with 40,000 sessions behind it, a simulator that replaces 25,000 GPU-hours of guessing, and kernels that beat the vendor default add up to something developers can evaluate on evidence rather than on a slogan. m:…6835 m:…6834 m:…9587