Agent memory needs a backward pass

Most agent memory systems I see in production are write-once. The agent finishes a task, a memory step distils a lesson from the trajectory, the lesson goes into a store, and later tasks retrieve it. Nothing ever goes back and asks whether that lesson actually helped. A new paper from MIT, Meta and Yale, Recurrent Harness for Agentic Continual Learning (Xiao et al., 2026), names this gap precisely and proposes a fix that is almost embarrassingly simple: give the harness a backward pass.

I think the framing matters more than the specific method, so I want to walk through both.

The problem with forward-only memory

The authors model a standard continual-learning harness as three roles around a fixed LLM: a decision agent that answers each query, a memory agent that turns each finished trajectory into a reflection, and a retrieval step that pulls the top-k reflections for the next query. Information flows one way: trajectory to reflection to store to the next decision.

Their toy example makes the failure obvious. The agent sorts {3, 7, 4} in descending order and the memory agent writes "put larger numbers first." The next query asks for ascending order. The stale reflection is retrieved, the agent gets it wrong, and the memory agent dutifully writes a new reflection, "put smaller numbers first." But the old one is still in the store, and because the next query shares numbers with the first, it gets retrieved again and misleads again.

The paper notes that existing curation methods (add, delete, merge in the style of Mem0 or ACE) keep a memory bank tidy, but they do not incorporate feedback from how a memory performed downstream. Methods that do optimise harness state against outcomes usually need replay or a held-out validation set, which a strictly online stream does not have. And with frontier API models there are no gradients to lean on.

Backward revision

ReHarness keeps forward read and forward write, and adds one step between them.

1. Forward read. Retrieve the top-k reflections for the current query (k = 5 in the main experiments) and let the decision agent act.
2. Backward revision. After the reward comes back, a revision agent revisits each retrieved reflection independently, given three things: the reflection, the trajectory it was originally distilled from, and the trajectory it just influenced. It rewrites the reflection so that, had it been used instead, it would have helped on the current task.
3. Forward write. The memory agent writes a new reflection from the current trajectory, using the revised reflections as demonstrations.

The constraint that makes this work is in step 2. The new trajectory is used only as feedback; any new content in the revised reflection must be grounded in the reflection's own source trajectory. That stops the revision from simply copying the latest answer into memory. In the sorting example, the revision turns "put larger numbers first" into "larger first only if descending": a limiting condition that was always implicit in the original evidence but got lost in the summary.

If you come from optimisation, this is the textual analogue of credit assignment. The forward pass stores a hypothesis; the backward pass tightens its domain of validity when it fails.

What the numbers say

The evaluation covers three benchmarks (CL-Bench, MemoryBench, MemoryArena), 16 tasks, and three backbones (Claude Opus 4.8, GPT-5.5, Gemini 3.7 Flash), all under a strict streaming protocol: each instance seen once, no training or validation split. The metric is normalised gain over the same backbone with no memory.

• ReHarness reaches an average normalised gain of 0.240, against 0.168 for Top-K Raw and 0.164 for Full Raw, and ranks first or tied in eight of nine benchmark-backbone combinations. The gains over every baseline are significant under paired Wilcoxon tests with Bonferroni correction.
• The raw-context baselines are the ones to beat. Simply retrieving prior trajectories outperforms most structured memory and reflection methods, several of which go negative on some settings. That matches what I see in practice: a lossy summary is often worse than the raw log.
• Removing the revision agent on CL-Bench with Claude drops the gain from 0.217 to 0.144. A counterfactual that re-applies the revised memory to the same instance reaches 0.335, which is not a legal online result but shows the revisions carry real information.
• On stability (the first instance after a distribution shift) the forward-only memory and reflection methods average negative gains: their accumulated memory actively hurts when the world changes. ReHarness has the highest average stability and the highest plasticity on every backbone.

The qualitative examples are the most convincing part. In a database-exploration task, the agent learns that the graded value lives in a column called prc. After a schema shift adds prc_v2, that reflection becomes wrong. One failure later, the revision agent rewrites it to "do not assume which column is graded; enumerate all price-like columns," and the next retrieval gets the answer in two queries.

Cost and grounding

Two things I would check before adopting this, and the paper checks both.

Grounding: decomposing revised reflections into atomic claims, over 95% stay entailed by the source trajectory, leakage from the new trajectory sits at 0.8 to 2.1%, and grounding drops by at most 0.3% per revision. The constraint mostly holds.

Cost: revision adds k extra LLM calls per instance, but compact reflections save far more input context than raw trajectories. On one CL-Bench rollout with Claude, ReHarness used about 41.6M tokens against 52.7M for Top-K Raw and 142.1M for Full Raw. In dollars with caching it lands at roughly $254, between Top-K Raw ($85) and Full Raw ($314). So it is not the cheap option; it is the better-performing option at a cost below full history.

Where it breaks

The authors are candid about the weak spot. On exploitable poker, ReHarness barely improves, because hand-level reward is weakly coupled to strategy quality. Backward revision is only as good as the feedback signal: if a single outcome is noisy, rewriting a reflection after every outcome is rewriting on noise. Raw-context baselines are more robust here because they defer interpretation to decision time.

There is also a storage cost: to revise a reflection you need its source trajectory, so the raw logs have to be kept anyway. And revising too many reflections per step (large k) degrades results, which the authors attribute to reflection drift.

My take

The idea I am taking away is not the specific prompts but the invariant: a memory that influenced a decision should be accountable to that decision's outcome, and its revision should be constrained to the evidence it was built from. That is cheap to add to any reflection-style harness I already run. I would gate it on reward quality, revising only where the signal is diagnostic (tests, schemas, verifiable answers) and leaving stochastic domains to raw retrieval. Forward-only memory was always half of a learning loop. This paper closes the other half without touching weights.