The memory wall is now the product

The memory wall

NVIDIA's post this week is unusually direct about the constraint shaping its roadmap. Larger models, longer contexts and agentic applications are increasing GPU memory pressure, and the company's answer is full-stack codesign: designing GPUs, CPUs, networking, storage and software together to optimise memory usage, rather than simply adding more high-bandwidth memory. NVIDIA was the first to adopt HBM3e and HBM4 technologies into an accelerated computing stack, but HBM capacity per GPU rose roughly 3% from Blackwell Ultra to the Vera Rubin platform — a small gain against a fast-growing demand. m:…3957

The techniques NVIDIA highlights are about using memory better. NVFP4 format reduces the space required for weights compared with FP8. CMX context memory storage is shared across servers and skips long-term data protection overhead. Within an NVL72 system, model weights can be distributed across up to 72 GPUs. The headline number is that agentic throughput per dollar rose by up to 30x on SemiAnalysis AgentX between Blackwell Ultra and Vera Rubin, and the GB300 NVL72 delivers up to 10x higher throughput per dollar than the H200 platform at 100 tokens per second per user. m:…3957

The wafer-scale defence

Cerebras is fighting a different battle: credibility. CEO Andrew Feldman publicly challenged a report by Culper Research that claimed Cerebras wafers fail after about two years. The company's defence is built on numbers. TSMC qualifies Cerebras wafers for 10 years of continuous operation at 105°C, Arrhenius modelling indicates a wafer held at 85°C would last 33 years before heat-induced failure, and the average junction temperature of deployed CS-3 wafers is less than 65°C. Less than 0.5% of all CS-3 wafers deployed since 2024 have been decommissioned, and Cerebras silicon is primarily retired for economic reasons rather than physical wear. m:…2378

Cerebras says it uses a patented assembly design to moderate temperature changes on the wafer, does not use the PariPoser material named in the report, and uses a 3 to 5 year depreciation schedule for its internal and cloud production assets. KPMG conducts annual audits of Cerebras' management estimates, including useful life. The subtext is that wafer-scale computing's economics depend on the wafer lasting, and a short seller's claim about durability is a claim about the business model. Cerebras answered with qualification data and audit references rather than adjectives. m:…2378

Buffers across campuses

Broadcom's contribution is about distance. AI training workloads are increasingly spreading across campuses and regions, and the company tested buffer sizing across 10, 30 and 100 km links. Fiber optic cabling adds approximately 100 microseconds of round-trip time for every 10 kilometres of distance, and traditional reactive congestion control becomes insufficient as workloads spread. Broadcom found a buffer threshold of 1.5 times the bandwidth-delay product keeps flow completion time near-ideal and retransmission at zero, and that the threshold holds under both NSCC and RCCC congestion-control protocols. m:…5175

The cost of that threshold scales with distance. At 100T switch throughput, 1.5x BDP requires 1.9 GB of buffer at 10 km, 5.6 GB at 30 km and 18.8 GB at 100 km. Below 1.5x BDP, retransmission rates increase and flow completion time degrades. Broadcom recommends a hierarchical, hybrid buffering model for multi-site AI clusters, with Jericho switches supporting a dual-tier buffering architecture and Thor NICs handling advanced congestion control at the endpoint. The message for anyone building a multi-site cluster is that the network is not a detail: at 100 km, the buffer alone is a capacity planning decision. m:…5175

Faster attention on AMD

Zyphra's ZYFA release shows the same memory-and-efficiency pressure from the software side, and on AMD hardware. ZYFA provides FP8 and BF16 prefill attention kernels for AMD Instinct MI355X GPUs, and its FP8 kernels achieve 1.16 to 1.76x the throughput of AMD's default AITER kernels on captured activations. FP8 matrix instructions offer twice the peak throughput of BF16 on the MI355X, and the MI355X executes 8-bit floating-point matrix instructions at twice the peak rate of BF16. The kernels also reduce numerical error by 62 to 78% compared with AITER's FP8 kernel at head dimension 128. m:…3381

The techniques are specific: a Mean P method that computes FP8 probabilities in 6 cycles per score compared with 14 cycles for exact P, and a Hadamard rotation of Q and K that halves the input quantisation error in Mistral-7B. ZYFA kernels support head dimension 128 and the head shape of MLA in DeepSeek-V2, and its BF16 kernels achieve 1.01 to 1.02x the throughput of AITER's default kernels. The BF16 result is honest about the limits — near parity — while the FP8 result is where the gain lives. For operators on AMD hardware, the choice between precisions is now a measured trade-off rather than a guess. m:…3381

What this means

Four announcements, one constraint. Memory pressure is the binding limit on agentic AI at scale, and every vendor in the stack is now selling its answer: NVIDIA with codesign and context memory, Cerebras with wafer-scale durability, Broadcom with deep buffers across campuses, Zyphra with faster low-precision attention. The buyers' question is no longer which chip is fastest in isolation, but which combination of memory, networking and software delivers the most useful work per byte and per dollar. m:…3957 m:…2378 m:…5175 m:…3381

There is also a credibility lesson in the week's posts. Cerebras answered a short seller with qualification data and audits. NVIDIA published a throughput-per-dollar number tied to a named benchmark. Broadcom published buffer sizes in gigabytes at three distances. Zyphra published throughput and error numbers against a named baseline. m:…2378 m:…3957 m:…5175 m:…3381