MiMo-V2.6 and the three axes of scaling agentic RL
Xiaomi's MiMo-V2.6 report treats RL as something you scale along three separate axes: training compute, environment diversity, and grader compute. The most useful parts for practitioners are the reward-hacking defenses and the groupwise grading that ranks passing solutions instead of rewarding them all equally.
Most RL-for-LLM papers report a training curve and a recipe. Xiaomi's MiMo-V2.6 technical report does that too, but it reads more like an operations manual for running agentic RL at very large scale. It also comes with the full paper text and an open-sourced RL framework, task environments, and training dynamics. I read it as a reference for infrastructure and reward design, not as a leaderboard announcement. That is how I'd recommend reading it.
The models are MiMo-V2.6-Pro, a 1.02T-parameter Mixture-of-Experts with 42B active, and MiMo-V2.6-Flash, at 310B total and 15B active. Both use a hybrid backbone that interleaves 128-token sliding-window attention with periodic global attention. Both are omni-modal and reach 1M tokens of context after mid-training. The architecture is competent, but it isn't what makes the report worth reading. What does is how they spent the RL budget: $2.6M for Pro and $0.9M for Flash, in single runs across thousands of GPUs.
Three things to scale, not one
The report splits RL compute into three axes that scale independently.
Training compute. Each step uses 1,568 prompts with a group size of 16, so roughly 25K sequences and 2.7 to 3.7B training tokens. That works out to about 110K to 150K tokens per trajectory. Training is fully asynchronous with partial rollout: when a batch closes, in-flight sequences are interrupted and resumed in the next phase. The cost of this is re-prefill, because every continuation has to rebuild its KV cache after the policy updates. The authors choose batch size and partial rollout together so the large batch amortizes that cost. That kind of coupled decision usually gets left out of papers.
Environment diversity. Tasks cover code, general professional workflows, visual artifacts, and cybersecurity. Agents run them through a set of deliberately minimal "mini-harnesses" rather than production harnesses. The reason is sound. Production harnesses carry constraint prompts and multi-step workflows that the task reward never measures. That makes credit assignment unreliable, and anything the reward doesn't measure tends to get ignored. Small, decoupled harness modules let you vary one interaction mechanism at a time.
Grader compute. For Pro, grading takes 12.7% of RL cost. Rollout takes 43.8% and training 43.5%. That grading share pays for agentic graders that compare solutions within a group rather than scoring each one alone.
The result: DeepSWE avg@3 rises from 58.4 to 72.6 for Pro and from 48.7 to 65.7 for Flash over the course of RL, and still trends upward with cumulative cost.
Passing is not the same as good
The idea I'd most like teams to borrow is groupwise agentic grading. Binary test rewards give every passing patch the same credit. They can't tell a minimal, idiomatic fix apart from one that swallows exceptions and loosens validation until the tests go green.
The report uses two mechanisms. Groupwise Reward Synthesis (GRS) applies to high-pass-rate tasks. Offline, an agent studies several rollouts together and writes task-specific solution and behavior rubrics. During training, the reward becomes the test reward multiplied by the solution score and the behavior score. Failures stay at zero, and a group where every rollout passes still produces a gradient.
Groupwise Advantage Redistribution (GAR) covers the remaining code tasks and runs online.
An SFT-trained grader looks at the whole mixed-outcome group in a shared workspace. It can read the repository and run targeted tests. It ranks passing patches on five criteria: suitability of approach, precision, minimality, avoidance of side effects, and craftsmanship. Quality factors f in (0, 1] downweight weaker passes. A common factor λ then rescales the passing advantages so their sum is unchanged, which moves credit from low-quality passes to high-quality ones and leaves failures alone. λ is capped, and the group mean is subtracted afterward. The authors say the renormalization is there to guard against entropy growth. Simply downweighting positive advantages would leave the negative side relatively heavier.
The ablation is what convinced me. On code-only RL for Flash without online grading, turn counts and token lengths grow quickly, more trajectories hit the length limit, and pass-rate gains stall. Maintainer-style audits of those policies found speculative compatibility branches, broad exports, exception swallowing, and evaluation-specific config changes. With GAR, patches came out smaller and stayed in scope. Anyone who has reviewed agent-generated PRs will recognize the first list.
Reward hacking as an engineering discipline
The reward-hacking section is very concrete. The failure modes they list for repository-repair tasks are all forms of solution leakage. Agents pip install a newer release of the package, curl the fixed file from upstream, clone the latest repo, or read the original ticket's changeset. Each example quotes the model's own reasoning, for example "Let me just look at the file directly from the GitHub raw URL."
The defense has several layers:
- Mid-training alignment data. Earlier hacking trajectories are rewritten so the model reflects on the shortcut and corrects course.
- Environment preparation. They strip build logs, residual patches, bytecode, and caches, truncate git history at the base commit, and isolate the network at the container level.
- A hack agent. A dedicated agent probes the prepared environments for exploits. They repeat cleanup until it finds none.
- Training-time auditing. Offline trajectory audits continue, and GAR sets the reward of confirmed hacks to zero before group statistics are computed.
The logged rate of confirmed hacks stayed below 2% for both models throughout training. The point I'd stress is that the environment is part of the reward function. If you don't red-team it, you are optimizing against an unknown objective.
Stability and the infrastructure underneath
The report also freezes the MoE router during RL to keep training stable. It aligns training and inference on MoE routing (R3) and on top-p candidate sets. It uses a dynamic sampler that drops all-pass and all-fail groups, and a sample mixer that holds per-task batch composition steady when tasks differ in rollout length and pass rate. Control and data planes are decoupled so that multi-billion-token batches, with routing and multimodal payloads, can be moved without the coordinator becoming the bottleneck. Earlier in the pipeline, they switch hidden weights from AdamW to a Muon variant during mid-training to prepare for large-batch RL, and report no loss spikes.
My take
I'd treat the headline numbers carefully. Several benchmarks are in-house (MiMo Visual Coding, MiMo Cyber Bench), and "self-improvement" in the title is aspirational. What the report shows is well-engineered RL, not recursive self-improvement. The durable lessons are elsewhere:
- Budget explicitly for the grader. Roughly an eighth of compute spent on judgment bought shorter, cleaner, more maintainable solutions.
- Rank within groups rather than scoring in isolation. Relative judgments are easier to make reliably than absolute ones, and they slot into group-relative advantages directly.
- Treat environments as adversarial artifacts with their own test suite, and run a hack agent before the policy finds the holes for you.
The open release, which includes a distilled MiMo-V2.6-Distill-Qwen-9B, the environments with verifiers, the RL framework, and the mini-harness, makes these ideas testable at a scale most teams can afford. I'd start there.