---
title: Same model, different results: the provider is now part of the model
description: FriendliAI, Scale Ventures, OpenRouter, AMD and SambaNova published on 11 and 12 November on how serving choices change speed, accuracy and cost, and why teams should test the serving stack, not just the model.
canonical: https://hangingcontext.com/stories/2026-11-12/same-model-different-results/
last-updated: 2026-11-12
url: https://hangingcontext.com/stories/2026-11-12/same-model-different-results/
published: 2026-11-12T12:42:13Z
edition: 2026-11-12T12
newsroom: artificial-intelligence
beat: infrastructure
trust: confirmed
source_count: 7
cited_manifest_ids: ['1794466541266625625', '1794445658838547070', '1794448164412556955', '1794435763181487942', '1794448164411556954', '1794445658838547069', '1794451533839566630']
---

# Same model, different results: the provider is now part of the model

*FriendliAI, Scale Ventures, OpenRouter, AMD and SambaNova published on 11 and 12 November on how serving choices change speed, accuracy and cost, and why teams should test the serving stack, not just the model.*

Confirmed · 7 Manifests · Artificial Intelligence · Published 2026-11-12 12:42 UTC

## Key facts

- FriendliAI says inference providers' engineering choices shape model results as much as the model itself. (self-reported) [cites: 1794466541266625625]
- FriendliAI says it recorded a median output rate of 132 tokens/s for GLM-5.3 in September 2026 and a median time to first token of 266 ms on Vercel AI Gateway between 13 and 20 October 2026. (self-reported) [cites: 1794466541266625625]
- OpenRouter says a 27B model on one rented RTX 3090 breaks even at about 12% load based on 8 October 2026 prices, while a large mixture-of-experts model on rented H200s never beats the cheapest zero-data-retention API endpoint. [cites: 1794445658838547070]
- AMD says long input lengths in GLM-5.2 create bottlenecks in GPU compute, memory bandwidth, and inter-GPU communication. (self-reported) [cites: 1794448164412556955]
- SambaNova says GLM-5.3 is available on SambaCloud at over 200 tokens/s and scores 28.3 on Terminal-Bench 3.0. (self-reported) [cites: 1794435763181487942]

## The serving stack changes the answer

FriendliAI's post, published 12 November, states the point directly: the company says inference providers' engineering choices shape model results as much as the model itself. The company says quantization methods, GPU kernels, and tool-call parsers are key engineering choices that affect model quality. That matters because two providers serving the same open-weight model can return different outputs, at different speeds, for different prices, even though the model name on the invoice is the same. [cites: 1794466541266625625]

FriendliAI gives numbers from its own measurements to show the size of the gap. The company says routing requests to higher-quality providers reduced tool-call error rates by 88% for GLM-5 and 80% for GLM-4.7 in March 2026. The company also cites a June 2026 University of Washington study that identified 90 real-world silent errors in vLLM and SGLang, and says preserving model quality requires ongoing validation of serving changes against a trusted baseline. The throughput and error figures are the company's own reports, and the study is cited by FriendliAI rather than independently verified here. [cites: 1794466541266625625]

## Speed numbers depend on the setup

The speed figures in this week's posts are tied to specific setups, and the posts say so. FriendliAI says it achieved a median time to first token of 266 ms for GLM-5.3 on Vercel AI Gateway between 13 and 20 October 2026, and recorded a median output rate of 132 tokens/s for GLM-5.3 in September 2026. The company also says its inference delivered approximately 2.1 times the aggregate throughput of vLLM on four NVIDIA B200 GPUs. SambaNova, writing on 11 November, says GLM-5.3 is available on SambaCloud at over 200 tokens/s, and scores 28.3 on Terminal-Bench 3.0 and 66.9 on DeepSWE v1.1. Those are serving and benchmark claims from the vendors, not launch news about the GLM models themselves. [cites: 1794466541266625625, 1794435763181487942]

> **From Synorb** — Claims with a source link. A Manifest holds a Signal of structured claims and a Brief, plus the source it came from. Your agent can check a sentence against it. [Read the guide](https://synorb.com/connect/start)


AMD's two posts, both published 11 November, explain why a single speed number is not enough for long-context work. AMD says GLM-5.2 uses a Mixture-of-Experts architecture, incorporates sparse attention mechanisms, and is designed for long-context processing, and that long input lengths create bottlenecks in GPU compute, memory bandwidth, and inter-GPU communication. AMD says tuning a single kernel is insufficient, and that optimising the model requires coordinated tuning of parallelisation, KV-cache layout, collective communication, kernels, and serving frameworks. On GLM-5.3-Flash, AMD says the model runs on a single eight-GPU Instinct MI300X node, the FP8 checkpoint is approximately 306 GB, and the KV cache footprint grows with request length and the number of requests in flight, so that when the pool is exhausted the scheduler stops admitting new requests. [cites: 1794448164412556955, 1794448164411556954]

## Self-hosting is a utilisation question

OpenRouter's self-hosting guide, published 12 November, frames the choice as utilisation, not ideology. OpenRouter says self-hosting an LLM is only cost-effective when traffic keeps the GPU busy enough of the time, because GPU rent is charged every hour regardless of utilisation, while API costs are based on token usage. The guide says a 27B model on one rented RTX 3090 breaks even at about 12% load based on 8 October 2026 prices, while DeepSeek V4.1 Flash on four rented H200s never breaks even against the cheapest zero-data-retention endpoint, and on two B200s needs 76% load at the lowest rent found to break even. [cites: 1794445658838547070]

OpenRouter also says most privacy requirements do not necessitate self-hosting, because the provider.zdr setting limits requests to zero-data-retention endpoints, and EU and US domains keep requests in-region. The guide says organisations can size self-hosted hardware for base traffic and route peaks to the API. That hybrid pattern is OpenRouter's recommendation, and it fits the wider point of this week's posts: the decision is about how the model is served, under what data rules, and at what sustained load, not only about which model is named. [cites: 1794445658838547070]

## Test your own workload

OpenRouter's benchmarking guide, also published 12 November, warns against trusting public rankings for a specific job. The guide says its benchmark uses 60 synthetic tasks to evaluate LLM performance, with 40 baseline tasks and 20 hard tasks, and reports that GPT-6.1 Sol passed 180 of 180 trials, Luna passed 162 of 180, and DeepSeek passed 157 of 180 with the lowest response cost in the study. The study found a Spearman correlation of 0.109 between its synthetic benchmark and public rankings, which OpenRouter presents as weak agreement, suggesting general benchmarks may not predict performance on a specific workload. [cites: 1794445658838547069]

Scale Ventures, writing on 12 November, adds the business context for providers themselves. The firm says open source serving engines like vLLM and SGLang have commoditised inference serving, and that inference providers typically see gross margins of 10 to 15% when reselling compute. Scale Ventures says providers that climb the stack to offer fine-tuning and RL infrastructure create higher customer stickiness, and describes the AWS playbook as cutting prices on the commodity layer while building higher-value products on top. For a buyer, the combined message is practical: measure quality, latency and cost on the team's own workload, re-check after serving changes, and treat the provider as part of the system being bought. [cites: 1794451533839566630]

## What is still unresolved

- How FriendliAI's 266 ms and 132 tokens/s figures compare on other gateways, since they are tied to Vercel AI Gateway and September 2026 measurements.
- Whether SambaNova's Terminal-Bench and DeepSWE scores were independently reproduced.
- What sustained load a given team's traffic actually reaches, which decides the self-hosting break-even in practice.

## Decision card

- **Useful for:** AI developers choosing an inference provider for a named model; Teams deciding between self-hosting and an API; Engineers setting up serving benchmarks on their own workload
- **Not for:** Choosing a model from a public ranking alone; Assuming the same model name means the same output everywhere
- **Compare against:** vLLM and SGLang self-hosted serving; Other providers serving the same model; Zero-data-retention API endpoints in-region

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1794466541266625625` | Same Model, Different Results: Why Your Inference Provider Matters | [friendliai-blog](https://friendli.ai/blog/same-model-different-results) | 1794466903307626467, 1794466903317626472, 1794466903315626471, 1794466903311626468, 1794466903314626470, 1794466903312626469, 1794466903324626477, 1794466903319626473 | 11-12 00:00 |
| `1794445658838547070` | Self-Hosted LLM vs API: When Does Self-Hosting Pay Off | [openrouter-rss](https://openrouter.ai/blog/insights/self-hosted-llm-vs-api) | 1794446014046549002, 1794446014047549003, 1794446014044549001, 1794446014042549000, 1794446014048549004, 1794446014050549005, 1794446014051549006, 1794446014053549007, 1794446014054549008, 1794446014063549014 | 11-12 00:00 |
| `1794448164412556955` | Accelerating GLM-5.2 on AMD Instinct MI355X: EvalScope Benchmarks with Prefix Caching | [amd-rocm-blog](https://rocm.blogs.amd.com/artificial-intelligence/glm5.2-mi355x-kvcache/README.html) | 1794448388156557268, 1794448388151557263, 1794448388153557265, 1794448388155557267, 1794448388158557269, 1794448388159557271 | 11-11 00:00 |
| `1794435763181487942` | GLM-5.3 Now Running on SambaCloud | [sambanova-blog](https://sambanova.ai/blog/glm-5.3-now-running-on-sambacloud) | 1794436012259488990, 1794436012261488991, 1794436012262488992 | 11-11 00:00 |
| `1794448164411556954` | Parallelism for Long-Context Serving: GLM-5.3-Flash on MI300X | [amd-rocm-blog](https://rocm.blogs.amd.com/artificial-intelligence/glm53flash-mi300x-vllm/README.html) | 1794448388123557253, 1794448388127557254, 1794448388133557257, 1794448388137557259 | 11-11 00:00 |
| `1794445658838547069` | How to Benchmark LLMs | [openrouter-rss](https://openrouter.ai/blog/tutorials/how-to-benchmark-llms) | 1794446013542548914, 1794446013552548919, 1794446013545548915, 1794446013550548918, 1794446013547548916, 1794446013549548917 | 11-12 00:00 |
| `1794451533839566630` | Inference providers have their head in the clouds | [scale-ventures-blog](https://scalevp.com/blog/inference-providers-have-their-head-in-the-clouds) | 1794451985061567665, 1794451985056567662, 1794451985064567667, 1794451985059567664 | 11-12 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1794466541266625625`
- `1794445658838547070`
- `1794448164412556955`
- `1794435763181487942`
- `1794448164411556954`
- `1794445658838547069`
- `1794451533839566630`

```sh
for id in 1794466541266625625 1794445658838547070 1794448164412556955 1794435763181487942 1794448164411556954 1794445658838547069 1794451533839566630; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Ask what Synorb covers. The MCP tool synorb-stream-search answers questions like "do you have this publisher?" without using any Manifest allowance. [Connect and ask](https://synorb.com/connect/start)

## Related stories

- [Fable, GLM and Hy4 landed in the same week. The test that matters is the one you run yourself](https://hangingcontext.com/stories/2026-11-12/new-models-week-fable-glm-hy4/)
- [AI writes the code now. Humans can't read it fast enough — and open source just proved it](https://hangingcontext.com/stories/2026-11-15/ai-prs-broke-open-source-review/)
- [Agents claim work they never did. Now we can count how often](https://hangingcontext.com/stories/2026-11-15/agents-claim-work-they-never-did/)
- [Throughput is the new model spec, and the gap between hosts is now fortyfold](https://hangingcontext.com/stories/2026-11-14/inference-throughput-race-kimi-deepseek/)

## Links

- HTML: https://hangingcontext.com/stories/2026-11-12/same-model-different-results/
- JSON: https://hangingcontext.com/stories/2026-11-12/same-model-different-results/index.json
- Markdown: https://hangingcontext.com/stories/2026-11-12/same-model-different-results/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-11-12/same-model-different-results/. Terms: https://hangingcontext.com/terms/reuse/