Same model, different results: the provider is now part of the model

The serving stack changes the answer

FriendliAI's post, published 12 November, states the point directly: the company says inference providers' engineering choices shape model results as much as the model itself. The company says quantization methods, GPU kernels, and tool-call parsers are key engineering choices that affect model quality. That matters because two providers serving the same open-weight model can return different outputs, at different speeds, for different prices, even though the model name on the invoice is the same. m:…5625

FriendliAI gives numbers from its own measurements to show the size of the gap. The company says routing requests to higher-quality providers reduced tool-call error rates by 88% for GLM-5 and 80% for GLM-4.7 in March 2026. The company also cites a June 2026 University of Washington study that identified 90 real-world silent errors in vLLM and SGLang, and says preserving model quality requires ongoing validation of serving changes against a trusted baseline. The throughput and error figures are the company's own reports, and the study is cited by FriendliAI rather than independently verified here. m:…5625

Speed numbers depend on the setup

The speed figures in this week's posts are tied to specific setups, and the posts say so. FriendliAI says it achieved a median time to first token of 266 ms for GLM-5.3 on Vercel AI Gateway between 13 and 20 October 2026, and recorded a median output rate of 132 tokens/s for GLM-5.3 in September 2026. The company also says its inference delivered approximately 2.1 times the aggregate throughput of vLLM on four NVIDIA B200 GPUs. SambaNova, writing on 11 November, says GLM-5.3 is available on SambaCloud at over 200 tokens/s, and scores 28.3 on Terminal-Bench 3.0 and 66.9 on DeepSWE v1.1. Those are serving and benchmark claims from the vendors, not launch news about the GLM models themselves. m:…5625 m:…7942

AMD's two posts, both published 11 November, explain why a single speed number is not enough for long-context work. AMD says GLM-5.2 uses a Mixture-of-Experts architecture, incorporates sparse attention mechanisms, and is designed for long-context processing, and that long input lengths create bottlenecks in GPU compute, memory bandwidth, and inter-GPU communication. AMD says tuning a single kernel is insufficient, and that optimising the model requires coordinated tuning of parallelisation, KV-cache layout, collective communication, kernels, and serving frameworks. On GLM-5.3-Flash, AMD says the model runs on a single eight-GPU Instinct MI300X node, the FP8 checkpoint is approximately 306 GB, and the KV cache footprint grows with request length and the number of requests in flight, so that when the pool is exhausted the scheduler stops admitting new requests. m:…6955 m:…6954

Self-hosting is a utilisation question

OpenRouter's self-hosting guide, published 12 November, frames the choice as utilisation, not ideology. OpenRouter says self-hosting an LLM is only cost-effective when traffic keeps the GPU busy enough of the time, because GPU rent is charged every hour regardless of utilisation, while API costs are based on token usage. The guide says a 27B model on one rented RTX 3090 breaks even at about 12% load based on 8 October 2026 prices, while DeepSeek V4.1 Flash on four rented H200s never breaks even against the cheapest zero-data-retention endpoint, and on two B200s needs 76% load at the lowest rent found to break even. m:…7070

OpenRouter also says most privacy requirements do not necessitate self-hosting, because the provider.zdr setting limits requests to zero-data-retention endpoints, and EU and US domains keep requests in-region. The guide says organisations can size self-hosted hardware for base traffic and route peaks to the API. That hybrid pattern is OpenRouter's recommendation, and it fits the wider point of this week's posts: the decision is about how the model is served, under what data rules, and at what sustained load, not only about which model is named. m:…7070

Test your own workload

OpenRouter's benchmarking guide, also published 12 November, warns against trusting public rankings for a specific job. The guide says its benchmark uses 60 synthetic tasks to evaluate LLM performance, with 40 baseline tasks and 20 hard tasks, and reports that GPT-6.1 Sol passed 180 of 180 trials, Luna passed 162 of 180, and DeepSeek passed 157 of 180 with the lowest response cost in the study. The study found a Spearman correlation of 0.109 between its synthetic benchmark and public rankings, which OpenRouter presents as weak agreement, suggesting general benchmarks may not predict performance on a specific workload. m:…7069

Scale Ventures, writing on 12 November, adds the business context for providers themselves. The firm says open source serving engines like vLLM and SGLang have commoditised inference serving, and that inference providers typically see gross margins of 10 to 15% when reselling compute. Scale Ventures says providers that climb the stack to offer fine-tuning and RL infrastructure create higher customer stickiness, and describes the AWS playbook as cutting prices on the commodity layer while building higher-value products on top. For a buyer, the combined message is practical: measure quality, latency and cost on the team's own workload, re-check after serving changes, and treat the provider as part of the system being bought. m:…6630