Training a small model to survive a harness that keeps changing

Most agent systems separate two things that used to be bundled together: the model's weights, and the harness around it — the skill library, tool schemas, system prompt, and the hook functions that validate inputs and outputs. That separation is what makes agents ship fast. A product team can add a skill, rename a tool, or tighten a retry policy on a Tuesday afternoon without anyone touching the model. The TaoLive AIGC LLM Team at Alibaba built their digital-avatar livestreaming agent this way, and then ran into the failure mode that separation quietly creates: the harness moves faster than the model can adapt to it.

Their agent runs Taobao Live's AI streamers — answering viewer questions about products, running marketing strategy, holding a conversation, all under a hard latency budget because it's live video, not a chat window. A frontier model can handle a shifting harness zero-shot; it's smart enough to read a new tool schema and use it correctly. But a frontier model is too slow for a live stream. A compact model hits the latency target, but the compact models the team tried had a specific and non-obvious failure: they weren't just weak in general, they were overfit to the exact harness they were trained against. Rename a skill, reorder a prompt block, change a retry hook, and accuracy dropped — not because the task changed, but because the surface form did.

The fix: train the model to expect the harness to move

The paper's answer is Harness-Aware Training (HAT), built around a component called Harness-State Augmentation (HSA). HSA doesn't touch the task. It takes the four things that make up a harness and applies task-preserving transformations to each of them during training, so the model never gets to memorize one fixed configuration:

• Skill identifiers and content — synthetic decoy skills are added, real ones are renamed and their descriptions rewritten by a strong LLM, subsets are masked, and rules within a skill are paraphrased and reordered.
• Tool schemas — tools are renamed and their descriptions rewritten while the underlying function stays identical, so the model can't just pattern-match on a tool's name.
• Prompt structure — instruction blocks and the items within them are reordered, and numeric constraints (reply length, how many turns are left) are perturbed within valid ranges.
• Hook functions — retry behavior, message structure, and hook-triggered edits get controlled variants to simulate how these actually change in production.

The premise is simple: if a model only ever sees one harness during training, it can't tell the difference between "this is how the task works" and "this is how today's harness happens to be phrased." Training across many valid harness variants forces it to learn the former.

Three stages, because each one fixes what the last one breaks

HAT isn't a single fine-tuning pass — it's three stages, and each exists because of a specific failure the previous one introduces.

HSA-SFT starts from about 10K real live-streaming examples — viewer queries, dialogue history, product context, paired with harness configurations. HSA generates augmented harness variants of each, a strong teacher model produces trajectories under both the original and augmented versions, and the results are filtered for accuracy and effectiveness before training on them. This is where the model first learns to treat the harness as variable input rather than fixed context.

Domain-heavy SFT has a known cost: it narrows the model. Instruction-following on held-out tasks degrades. The paper measures this directly — a Fixed-Harness SFT baseline drops IFEval by 7.7 points relative to the base model. General On-Policy Distillation is the correction: on a general instruction dataset (Tulu3), the student is trained to match the base model's output distribution via KL divergence, pulling back the general capability that narrow SFT erodes, before the model specializes further.

HSA-RL is the final stage — reinforcement learning inside an augmented simulator that implements real skill loading, tool execution, and hooks over multi-round interactions. Reward is split across accuracy, effectiveness, tool-call rationality, skill-selection quality, and a length penalty against padded reasoning, optimized with GRPO/GDPO-style group-relative advantages. This is what pushes robustness past what SFT alone achieves, since RL lets the model experience its own failures against a shifting harness rather than only imitating a teacher's successes.

The numbers, and why the deployment matters more than the benchmarks

On Live-Stream QA, HAT reaches 94.8 against a base-model score of 80.3 — and against 93.0 for the strongest general-purpose LLM the team tested, despite HAT running on a compact model under a hard latency budget. On Harness-Variant QA — the benchmark built specifically to test robustness to harness changes — HAT scores 94.6 against a base of 75.4. And critically, HAT reaches 83.5 on IFEval, avoiding the 7.7-point regression that plain harness-specific SFT causes. That's the paper's central claim in one comparison: you can specialize a model to a harness without sacrificing its general instruction-following, if you correct for it explicitly.

None of that would mean much without production validation, and this is the part I'd flag as the more unusual contribution. The system runs on a single NVIDIA H20 GPU, with P50 latency of 3.4 seconds and P95 of 8.1 seconds — inside a live video interaction. It's deployed in Taobao Live's digital-avatar service, and the team reports positive online A/B results on confirmed-receipt GMV and item-page views against an 80/20 traffic split. Offline benchmarks tell you a model generalizes across harness variants you constructed. An A/B test on real commerce traffic tells you it survives contact with a harness nobody constructed — the one product and ops teams are still actively changing.

Why this matters beyond livestreaming

The skill/tool/prompt/hook split is common — agent frameworks are converging on some version of it, and the reason is the same one TaoLive had: it lets non-model teams iterate without a retraining cycle. But that architectural choice implicitly assumes the model is robust to the harness moving underneath it, and that assumption usually goes untested until an ops team renames a tool in production and accuracy quietly drops. HAT is a reasonably direct way to test that assumption on purpose, during training, instead of finding out in an incident review. The specific augmentations — rename skills and tools, reorder prompt blocks, vary hook retry behavior — read less like a novel technique and more like someone finally writing down the list of things a harness actually changes in practice, and training against all of them at once.