Post-training goes mainstream: 4-bit RL, zero-latency adapters, $9 GPU-hours

A service, not a lab

On October 19, Together AI introduced Together Post-Training, a service for building custom models through reinforcement learning and supervised fine-tuning, and announced promotional pricing of $9 per GPU-hour of B200 usage for the first eight weeks. The company says users define their own training loop while it co-optimizes the systems underneath, and that experiments run on dedicated Model Resources — GPU allocations the company says do not compete with other users' requests. The service supports LoRA training and full-weight experiments, multi-LoRA adapter serving that lets users swap adapters without starting a separate production deployment for each run, and asynchronous RL aimed at keeping GPUs busy during long agentic rollouts. m:…4825

The early-adopter numbers come from the company's own announcement, so read them as customer testimony rather than audited benchmarks. Deel reported running up to five concurrent RL experiments on the same GPU capacity using the platform, and Jack & Jill said it matched a closed-source model's quality in offline evaluations using an open-source model the company described as significantly cheaper to serve. If those results generalize, the implication is straightforward: the skill set that once required a training-infrastructure team is being packaged into a metered service, and the differentiator moves from who can train to who knows what to train on. m:…4825

4-bit RL stops being a gamble

In a preprint published October 20, researchers diagnose why native 4-bit RL training is unreliable: in NVFP4 reinforcement learning, matching model weights and numerical formats across the training and sampling engines is insufficient to guarantee consistent policy evaluation. Their framework, miniTE, targets that execution gap directly — a cross-engine numerical alignment toolkit plus W4A4 GEMM computation — and the authors report that it improves numerical execution consistency without modifying the RL objective itself. m:…4097

The reported numbers: 33,540 tokens per second on a single NVIDIA B300 GPU node, which the authors put at 1.39x the throughput of BF16 and 1.47x of standard TE NVFP4 for complete learner updates. Those are preprint results on one hardware node, but the direction matters: if 4-bit RL becomes numerically boring, the cost floor of post-training drops with it. m:…4097

Adapters with no serving tax

A second preprint, also published October 20, takes aim at the adapter layer. SOS-LoRA decomposes the total LoRA rank across K static experts with cross-expert orthogonal initialization and fixed multi-scale scaling, which the authors say mitigates interference from shared input directions. The practical claim: the adapter stays fully mergeable, with zero inference-time parameter or latency overhead. On benchmarks the authors report consistent gains over matched-budget LoRA baselines on Llama 2 and 3, GLUE, GSM8K, and MATH. Code is published on GitHub. m:…5418

The layer that got cheap

Step back and the three moves are the same story told three ways. A managed service turns RL and SFT into an operational expense; a precision-training fix makes the cheapest hardware precision numerically trustworthy; a better adapter design raises fine-tune quality without taxing inference. None of this invents a better base model — that is the point. Post-training is being commoditized into tooling, and the scarce input is becoming the reward signal and the data, not the compute. For developers, the near-term question is not whether to customize models but how to price the experiment: at promotional $9-per-GPU-hour B200 time, five parallel RL runs, and mergeable adapters, the cost of finding out what works is falling fast. m:…4825 m:…4097 m:…5418