Selling tokens is a race to the bottom, so inference sellers build platforms

What production-ready means

A serverless GPU provider is ready for production when you can control eight things, writes J Jim Cai in fal's guide: the warm floor and the ceiling, cold starts, multi-GPU runners, logs and exportable metrics, what happens when a request fails, where it runs and what capacity backs it, who can call it, and the most it can cost per hour. The list is a procurement checklist disguised as a tutorial, and each item maps to a failure mode teams have already hit. m:…3036

On fal, queue requests are retried on a new runner for server errors, timeouts and connection errors, up to 10 attempts in total, and requests that fail with an HTTP 5xx status are not charged. Metrics come from a Prometheus-format endpoint covering runners by state, queue size, requests completed, latency buckets, and GPU and VRAM use, and the most a user can spend per hour is max concurrency multiplied by number of GPUs multiplied by the hourly rate. m:…3036

fal publishes a 99.99% uptime SLA, the guide says, with per-second billing while a runner is alive, covering setup, idle, running, draining and teardown, while waiting for hardware and pulling the image are not billed. Minimum concurrency keeps runners alive whatever the traffic, with a default of zero, so an application scales to zero, and the fal.distributed module splits one request across a runner's GPUs or runs several outputs at once. m:…3036

Why the platform, not the token

fal's own story fits the same pattern. The Sourcery podcast episode is titled around fal's $1.65 billion Series E and F led by Sequoia and Bessemer, and the founders describe a transition from data infrastructure to generative media, with about 50% of revenue coming from video models. The company maintains a flat, two-level hierarchy using high-agency individual contributors and coding agents. m:…1229

Token pricing itself is part of the problem. A study of metering for inference services argues that under batched serving no per-request meter can be simultaneously additive, independent of co-resident traffic, and exactly cost-reflective. Its proposed architecture uses reference inference units and occupancy-indexed ratings, which limit statement energy error to 9 percent under measured traffic, compared with 70.5 percent under fixed-occupancy rating. Tenant mispricing falls from 7.36 times under flat token counting to 1.06 times after occupancy-based reconciliation. For a buyer, that means a low token price is not a complete price: the metering method decides whether the bill tracks the work done under sharing. m:…1487

What to check before buying

Read together, the fal accounts and the metering study give buyers a short test. First, can the provider show the eight controls, including a cost ceiling, retry behaviour and exportable metrics, rather than only a price per token? Second, does its metering reflect sharing, since flat token counting misprices tenants when requests share hardware? Third, can it demonstrate the demand and cost structure behind its prices, since a platform that cannot show its controls or its metering leaves the buyer to discover the real bill after deployment? m:…3036 m:…1487

The open question is how far metering reform goes. If flat token counting misprices tenants by more than seven times under sharing, the winning inference vendors will look less like commodity exchanges and more like specialist platforms whose metering, controls and capacity reflect the actual work, and the token price will be the least interesting number in the contract. m:…1487