Self-Training, Team Training, and the Collaboration Gap

Self-improvement without curated data

In a preprint, researchers introduce On-Policy Self-Training, a method that allows foundation models to improve performance without human-curated data. Existing on-policy self-distillation approaches rely on ground-truth question-answer pairs or human-curated problem sets, according to the authors, while OPST uses self-generated question-answer pairs and privileged reasoning demonstrations instead. m:…8716

The filter is the mechanism that keeps the self-generated data honest. OPST is guided by coarse, high-level topic descriptions, and its pseudo-verification filter keeps a problem only if the model can reproduce its answer in a fresh context, and requires the model to reproduce the answer under a paraphrase, according to the authors. A claim that survives rephrasing is treated as more reliable than one that only holds in a single wording. m:…8716

The authors report that OPST achieves self-improvement on standard language benchmarks and on Visual Question Answering, and that models trained with OPST retain capabilities on domains they are not self-trained on. Self-distillation on synthetic privileged information enables a model to improve across multiple domains, according to the authors, and OPST outperforms alternative training strategies in their comparisons. m:…8716

Training the team, not just the model

In a second preprint, MultiAgent-OPD trains a hierarchical multi-agent team via on-policy distillation and reinforcement learning. The team is a shared-parameter Leader-Searcher-Answerer group trained end to end, according to the authors, and the Leader learns from team success while the Answerer learns from candidate correctness. m:…6585

The training problem it addresses is credit assignment. Jointly training a Leader and subagents is difficult because final answers provide only indirect feedback, according to the authors, and LLM judge scores for subagents depend on the judge's own assessment of response quality. MultiAgent-OPD uses a frozen teacher for local process supervision, with the teacher model trained in a Searchers-Answerer workflow using answer correctness and scoring student-generated search and summary tokens. m:…6585

The reported results are 38.68% macro avg@4 across seven datasets, exceeding Prompted Critic by 4.06 percentage points and C-MAS + RL by 2.61 percentage points, according to the authors. Inference for MultiAgent-OPD requires only the trained student model, which means the teacher is a training-time cost, not a deployment dependency. m:…6585

Collaboration is the weak link

Kolab approaches the same multi-agent question from the human side. It is a dataset of 115+ hours of screen recordings of 50 professionals from 40 occupations collaborating with agents, according to the authors, designed for long-horizon tasks requiring ongoing interaction with human experts rather than one-shot automation. m:…3930

Most agentic benchmarks focus on automation rather than effective real-world collaboration, according to the study, and frontier AI models overestimate their collaboration quality with experts. Collaborative agents must anticipate whether experts will adopt, modify, or reject contributions to guide their next steps, according to the authors — a prediction task, not just a generation task. m:…3930

Longer workflow context does not consistently improve the ability of frontier models to predict collaboration quality, according to the study, and methods for improving agent performance in long-horizon collaboration remain an open research challenge. Read together, the three papers describe a field that can now train models and teams with less human data, but cannot yet reliably tell when a human partner will accept the result. m:…3930

What to test before trusting it

For builders, the practical test for self-training is whether gains survive outside the reported benchmarks and whether a human partner accepts the output. OPST reports retained capabilities on domains it was not self-trained on, according to the authors, but Kolab shows that acceptance by an expert is a separate prediction problem that longer context alone does not solve. m:…8716 m:…3930