Kimi K3.1 Pro lands with a serving stack ready on day one

A release with a runway

Moonshot AI released Kimi K3.1 Pro on November 4, 2026, and the serving work started before the model did. Moonshot and Inferact coordinated on the K3 architecture and serving requirements ahead of release, covering pre-launch tuning, native vLLM integration, and deployment validation, so the model arrived with production plumbing already attached. m:…7498

The model itself is a 3T-class open-weight model with a 1-million-token context window, built for long-horizon work. On Runpod it is available on public endpoints alongside a Highspeed variant, and developers can pick between the two per request on a single endpoint, choosing complex reasoning or a faster agent loop depending on the job. m:…5064

AMD, meanwhile, offered support on Instinct MI355X GPUs from day zero, using its ATOM framework and ROCm software stack with optimized inference kernels. That matters because a model that only runs well on one vendor's hardware on launch week narrows who can actually use it; day-zero support on a second hardware family widens the door immediately. m:…7889

The managed-service option

Verda launched Verda Inference in preview on November 4 in partnership with Moonshot AI, starting with Kimi K3.1 Pro. It is a managed model-as-a-service offering: Verda handles the GPUs, the models, and the serving in its own data centers in Europe, and exposes an OpenAI-compatible API so developers do not have to rebuild their client code. m:…9854

The company says the service delivers up to 10x faster time to first token on long-context agentic workloads than standard caching, because it uses a shared cache to reuse context across agent steps. In internal testing, the company says, the cache reused about 99% of context. Those are vendor numbers, not independent benchmarks, but they describe the design clearly: agent workloads repeat context, so caching it is the product. m:…9854

Verda also says its engineering team increased agent session density by 16x on each GPU in six weeks, and that users pay only for the tokens they use. If that density holds outside Verda's own testing, the cost story for long-running agents changes: the expensive part of an agent is not one big answer, it is dozens of steps that keep re-reading the same context. m:…9854

Why this pattern is worth copying

The Inferact collaboration shows what a complete serving story looks like. The Kimi K3 implementation in vLLM includes Kimi Delta Attention, hybrid cache management, and native vision inputs, plus reasoning and tool-call parsers and prefill-decode disaggregation. Inferact also trained a speculative decoding model to raise generation throughput, and contributes its production optimizations back to the open-source vLLM community. m:…7498

For a developer, the practical upshot is choice without rework. Runpod's endpoint is OpenAI compatible, Verda's API is OpenAI compatible, and vLLM is the open serving layer underneath. A team can prototype on one of them and move to another without rewriting the integration, and can trade reasoning depth for speed per request rather than per deployment. m:…5064

What remains unproven is how the new serving options behave under real production load. Verda Inference is in preview, so its pricing, limits, and support terms can still change, and the qualification work Inferact performed for the Kimi K3 model shows how much testing sits behind a day-one claim. The model is out; the serving track record is still being written, and the teams that adopt early will be the ones writing it. m:…9854 m:…7498