Inference hardware goes heterogeneous

Two kinds of silicon in one rack

Supermicro is collaborating with SambaNova to incorporate SN50 Reconfigurable Dataflow Unit chips into its Data Center Building Block Solutions. The partnership aims to deploy heterogeneous inference systems that pair NVIDIA GPUs with SambaNova's chips: NVIDIA GPUs will perform the prefill phase and SN50 RDUs the decode phase. The company says Supermicro has an initial plan to procure 10,000 SN50 units for these systems. m:…5603

Supermicro will integrate, validate and support the new disaggregated inference systems, which use specialized hardware for distinct phases of large language model processing. The idea is that agentic inference is not one workload: different phases have different compute profiles, so matching each phase to the hardware built for it should beat running everything on one chip type. m:…9907

Initial deployment will prioritize NeoClouds and sovereign clouds before expanding to enterprise data centers. That ordering says something about the first buyers: cloud operators building inference capacity at scale, and sovereign clouds that want inference infrastructure they can control, are the early market for heterogeneous systems. m:…9907 m:…5603

The connectivity business behind it

Astera Labs filed an 8-K disclosing record third quarter 2026 results, with revenue of $584.9 million, up 154% year over year. The company reported GAAP net income of $182.9 million and diluted earnings per share of $0.99. Management attributed growth to the Scorpio fabric switch family and announced new product initiatives including Aries 7 and Orion reference architectures, with fourth quarter revenue guidance of $730 million to $750 million. m:…8325

Astera's numbers are a useful check on the heterogeneous story. If inference systems split across chip types and vendors, the fabric connecting them becomes more important, not less, and a connectivity supplier growing at this pace suggests the build-out is real. The filing is also a reminder that this part of the stack is already a public-company business with audited numbers, not just partnership announcements. m:…8325

Cooling as a design constraint

The third piece is thermal. Accelsius is supplying its NeuCool IR150 two-phase direct-to-chip cooling infrastructure to Villanova University for use by the NSF Center for Energy-Smart Electronic Systems, known as ES2. The partnership supports ES2's mission to enhance US data center energy efficiency by 20-35% through advanced thermal management research. m:…7535

The company says the technology is expected to be installed at Villanova University in early October 2026. Denser, more heterogeneous racks make cooling a first-order design problem: the hardware mix only helps if the heat it produces can be removed efficiently, and direct-to-chip cooling is one of the answers being tested at research scale. m:…7535

Put together, the three items describe an inference stack that is no longer a single chip decision. The compute is heterogeneous, the fabric between chips is a growth business in its own right, and cooling is a constraint that shapes what can actually be deployed. Buyers choosing inference infrastructure now are choosing a system, not a chip. m:…9907 m:…8325 m:…7535

The sequencing matters too. Supermicro is starting with NeoClouds and sovereign clouds, the buyers building inference capacity fastest and with the most control over their stacks, before expanding to enterprise data centers. If heterogeneous systems prove themselves there, enterprise buyers will inherit a tested architecture rather than an experiment. m:…9907