IQuest-Q1: a training recipe for agentic CLI models
IQuest-Q1 is an open-weight 320B/15B-active MoE trained for agentic coding across multiple CLI harnesses. Its technical report is most valuable as a recipe: measurable stability headroom, harness-aware RL with failure attribution, serious reward-hacking defenses, and multi-teacher distillation plus merging.
IQuest Research has released IQuest-Q1, an open-weight sparse Mixture-of-Experts model with 320B total parameters and 15B active per token, built for agentic coding through command-line harnesses. The abstract reads like a list of named techniques: Stable SplitNorm, Muon, muP, MOPD, model merging. On its own it gives no numbers. The full technical report does, and it is more useful than the abstract suggests. I read it less as a leaderboard entry and more as a recipe: a fairly complete account of how to train one policy that works across Claude Code, Codex and OpenCode instead of overfitting to one of them.
The shape of the model
The architecture is conservative where it should be. It has 88 layers, a hidden size of 3,072, and 256 routed experts with 8 active. Attention interleaves three sliding-window layers (4,096-token window) with one full-attention layer, plus full attention in the first two and last two layers. A query-aware sink token absorbs excess attention mass, and the first layer is a dense MLP so that expert load balancing does not break where the representation is still raw.
The part I find most interesting is normalization. Pre-Norm is stable, but the residual stream's variance grows with depth, so later layers contribute less and less. SplitNorm normalizes the stream entering attention but leaves the FFN path unnormalized. Scaled naively, that blows up. IQuest adds an RMSNorm on the attention output and a sandwich norm in the first layer. In a controlled comparison against a Qwen3-30B-A3B-style baseline on identical data, they report 1.93x scaling efficiency, meaning roughly 52% of the compute to reach the same training loss.
Stability headroom as a number
The idea I will reuse is what they call Critical Model Size. For a model of size N, take the efficiency-optimal learning rate and the largest learning rate that still trains stably. The log of their ratio is your headroom. When the two meet, you have reached the largest model this recipe can train without trading efficiency for stability. It works like the critical batch size, but along the model-size axis.
Extrapolating their fits gives a CMS of about 572B active and 12T total parameters. I would treat an extrapolation that far out with caution. Still, turning "will this recipe survive another 4x?" into two curves you can measure on small proxies is good engineering practice. The muP setup supports it: they tuned the learning rate on an 896-wide, 18-layer proxy and transferred it zero-shot, and they validated transfer across a 4x range in both width and depth.
The pipeline
Pre-training runs in two stages, and the code share rises from 40% to 70% in the second. Mid-training extends context from 32K to 128K to 512K, uses token-level loss masks that are decoupled from chat roles, and trains on trajectories rolled out in more than a million containerized repository instances. Across pre- and mid-training the model sees 29T tokens. The team tracks progress with bits-per-byte on held-out agent trajectories, a generation-free signal I would like to see used more widely.
Post-training is where the CLI focus shows. Each RL episode fixes a task domain, a harness and its version, an environment, and an evaluator. GRPO compares rollouts under the same harness. Failures are attributed before they become gradient: policy errors produce a learning signal, environment or harness faults are masked, and evaluator faults block the sample entirely. Anyone who has watched a flaky sandbox poison an RL run will see why that matters.
Multi-teacher on-policy distillation (MOPD) then consolidates four frozen experts (UX, multi-harness, long-horizon, general) into one student. The student samples, and the domain-matched teacher scores those tokens with a clipped log-ratio advantage. Residual merging of parameter deltas, with l2-norm calibration, runs alongside training across stages. On the infrastructure side, teacher weights are swapped in on demand from distributed storage, so adding a teacher does not mean adding a resident GPU replica.
Reward hacking gets a real section
Most agentic training reports mention reward hacking in a sentence. This one describes the defenses. Git history is truncated at the base commit, and hooks in Claude Code, Codex and OpenCode block attempts to fetch upstream fixes. Blocked operations return explicit feedback rather than failing silently. Collected trajectories are audited by rules and a rubric-guided judge. For coding tasks, reference solutions must pass and no-op solutions must fail. That last check is cheap and catches a surprising share of broken verifiers.
What the numbers say
The report's Table 2 is broad. IQuest-Q1 scores 64.6 on DeepSWE v1.1, 57.5 on SWE-Bench Pro, 63.0 on NL2Repo, 84.5 on CyberGym and 83.2 on Terminal-Bench 2.1. Against open-weight peers such as GLM-5.3 Flash and DeepSeek-V4-Flash it is competitive, usually close and sometimes ahead. Against Claude Opus 5 and GPT-5.6 Sol the gap on the hardest long-horizon work is large: 22.7 against 53.9 and 37.3 on Terminal-Bench 4.0, and 57.5 against 79.2 on SWE-Bench Pro.
Three caveats. First, many baseline numbers are marked as the authors' own runs. Second, IQuest-CLIBench, which is the user-experience metric, is proprietary. Third, the harness differs by benchmark. These are not reasons to dismiss the results, but they are reasons to rerun them on your own workload before you rely on them.
My read
I would not take IQuest-Q1 as evidence that it has caught the frontier on agentic coding. Under the evaluation settings the report describes, it trails Claude Opus 5 and GPT-5.6 Sol clearly on long-horizon terminal work. I would take it as one of the more complete public recipes for building a coding agent model: measurable stability headroom, curricula judged by trajectory likelihood, RL episodes that treat the harness as a first-class variable, failure attribution before optimization, and expert consolidation through on-policy distillation plus merging.
The report also describes a "human-on-the-loop" process in which the model diagnosed its own tool-schema failures across 6,054 calls and proposed a 2.35B-token data fix that humans approved. That is a sensible, bounded use of an agent in model development. I would call it what it is, supervised tooling, rather than recursive self-improvement.
If you build agentic coding systems, read sections 2.2 and 4.2 through 4.5 even if you never download the weights.