Teaching a model to write its own homework: SPADE and the case for a learnable curriculum

The bottleneck nobody talks about

Every RL pipeline for language agents eventually runs into the same wall: where do the tasks come from. You can hand-curate a benchmark, but curation doesn't scale past a few thousand examples and it freezes the moment you ship it. You can synthesize tasks with a fixed generator, but a fixed generator has a fixed difficulty ceiling — the model saturates it and keeps training on problems it already solves, which is a wasted gradient step dressed up as progress. You can use a frozen verifier, but frozen verifiers don't know when the model has outgrown them.

The common failure mode is a static goal distribution meeting a learner that isn't static. As the model improves, the gap between "what the environment pool offers" and "what would actually teach the model something" widens, and self-improvement stalls out — not because the model can't learn more, but because nothing is left to learn from.

SPADE: Self-Play in Adaptive Synthetic Executable Environments (Liu, Yu, Jiang, Qu, Zhao, et al., submitted 19 Aug 2026) takes a direct shot at that bottleneck: stop treating environment design as a one-time authoring problem and make it a second thing the model learns to do.

One model, two jobs

SPADE runs a single LLM in two roles. The Environment Designer writes complete training environments as executable code — an OpenAI Gym-style reset()/step() interface, plus the reward function and the verification logic that scores an attempt. The Reasoning Agent is the same underlying model, playing the opposite side: it receives the environment the Designer just wrote and tries to solve it.

What makes this more than a wrapper around "LLM writes practice problems for itself" is that the interface is stateful and multi-turn on both ends. An environment isn't a static prompt-answer pair — it has state transitions, so the same reset()/step() contract covers a math derivation, a coding task with a test suite, or a multi-step tool-use episode where the agent has to call functions in sequence and recover from bad intermediate results. One interface, one authoring format, spanning reasoning problems and agentic tool use.

The regret signal that keeps things honest

The hard part of self-generated curricula has always been calibration. A Designer with no feedback either converges to trivially easy tasks (which are easy to "pass" and look productive on a loss curve while teaching nothing) or drifts into unsolvable ones (which just produce noise, since there's no gradient signal in uniform failure either). SPADE's answer is to estimate the Reasoning Agent's regret on each generated environment: the gap between the reward it gets with a privileged hint and the reward it gets without one.

That gap is informative in a way raw pass/fail isn't. If the agent solves the task fine either way, the hint didn't matter — the task was already within reach, regret is near zero, and it's not worth training on. If the agent fails even with the hint, the task is out of reach regardless of what it's told, regret collapses again, and it's also not worth training on. Regret peaks in the middle: where the hint is the difference between failure and success, meaning the task is exactly at the boundary of what the agent can currently do. The Designer is trained to push this regret signal up, which in practice means learning to write environments that sit right at that boundary — hard enough to matter, feasible enough to be learnable.

This is a concrete, differentiable stand-in for the harder-to-operationalize idea of "zone of proximal development" that shows up all over curriculum learning and unsupervised environment design. What's notable here isn't the concept — regret-driven curricula have precedent — it's that SPADE gets it working with a single LLM authoring executable code as the task representation, at a scale (30B parameters) where the environments are genuinely open-ended rather than drawn from a fixed template family.

What actually made it work

The paper is candid that the regret objective alone wasn't sufficient. Two additions were load-bearing. First, grounding the Environment Designer on documents sampled from a large pretraining corpus — rather than letting it free-generate from nothing, which tends to degenerate into a narrow, repetitive set of task shapes. Second, giving the Designer an accumulated environment memory, so it isn't rediscovering the same difficulty calibration from scratch on every generation and can build on what has and hasn't worked before. Neither is a surprising ingredient in isolation, but the paper's ablations treat them as necessary, not optional — the self-play loop degrades without them.

The numbers

At 30B parameters, SPADE improves over the strongest fixed-environment baseline by +5.3 points averaged across eight held-out math, science, code, and reasoning benchmarks. In the tool-use setting, it lifts BFCL-v4 multi-turn by +5.7 and ACEBench-Agent by +13.9 — the larger of the two, on the benchmark that stresses longer agentic sequences, which is consistent with the core claim: static curricula lose the most ground precisely where tasks are long-horizon and hard to hand-author. On the games setting, the paper reports that SPADE's margin over the strongest baseline grows with model scale, which is the direction you want to see if the goal is a training method that keeps paying off rather than one that closes as models get better.

My read

What I find worth taking seriously here isn't any single benchmark number — it's the framing choice. Most agentic RL work treats the environment as fixed infrastructure and puts all the learning into the policy. SPADE puts the environment on the same side of the learning problem as the policy. That's a meaningful reframing because it means the ceiling on "how much useful training signal is available" is no longer set by how much a human curated in advance — it's set by how well the Designer role can be trained, which is itself an empirical question with room to improve.

I'd treat the specific benchmark deltas as early evidence rather than a settled result — the paper is explicitly work in progress, single scale, and self-play curricula have a history of finding degenerate shortcuts that only show up once you look past the headline numbers (reward hacking the verifier, or the Designer converging on a narrow family of "easy to regret-optimize" tasks). The grounding-on-pretraining-documents and memory ablations are the right kind of detail to watch in follow-up work, since that's usually where these systems quietly break. But as a research direction — make environment design itself the thing that scales with the model instead of the thing that caps it — this is one of the more concrete steps I've seen toward open-ended self-improvement that isn't just relabeled data augmentation.