title: "Bilevel Reflection shows why a transcript-only critic cannot improve memory in every environment" slug: "bilevel-reflection-transcript-only-memory-gate" excerpt: "A new multi-agent study separates reflection from verification and argues that shared memory should change only after an external test shows a real improvement." category: "AI Research" tags:
- "multi-agent systems"
- "agent memory"
- "verification"
- "coding agents"
- "SRMA" author: "ISH Team" read_time: "7"
An agent writes a lesson after a failed run. Another model decides the lesson sounds reasonable and saves it to shared memory. The next run now carries that advice as context, whether it was useful or poisonous.
Bilevel Coordinated Reflection treats that memory update as a state change, not a nicer prompt. Its sharpest result is an impossibility theorem. A gate that sees only generated text cannot improve memory uniformly across environments that produce the same transcript but assign opposite value to the same reflection. A more capable textual critic still lacks the missing information.
The paper proposes Stochastic Reflective Memory Ascent (SRMA) instead. It evaluates the current memory and a candidate under the same grounded protocol, then commits the update only when measured verifier risk strictly decreases. Tests, simulator state, executed API responses, or a formal checker can supply the external signal.
The operational rule is simple: treat a memory write like a deployment. Generate proposals freely, but require evidence before they become durable context.
Why the transcript cannot settle it
The proof constructs two environments with identical text-generation laws. In one, a class of reflections is corrective and another is harmful. The second environment swaps those meanings. A transcript-only gate sees the same proposal stream and history in both, so it makes the same acceptance decisions.
For every such gate and evaluation horizon, at least one environment ends with expected error no lower than the starting error. If the gate accepts an ambiguous proposal with positive probability, the worse environment becomes strictly worse when initial error is below one half. Free-form reflection accepts everything and approaches an error of one half in both constructed environments. A grounded gate observes environmental value, accepts only the corrective class, and reaches zero error geometrically in the construction.
This is a minimax theorem, not a claim that text critique never works. The paper excludes cases where the transcript certifies correctness, such as a fully checkable proof. The limit applies when correctness depends on state that the text does not expose: a hidden resource cap, a simulator, a live API response, or the current repository.
That distinction belongs beside our warning that agent memory is not factual evidence. A memory item can be clear, confident, and internally consistent while remaining wrong about the system it describes.
SRMA tests a reflection before saving it
The language model remains the proposal engine. The system first runs a fixed evaluation protocol with the current memory and calculates verifier risk. The model proposes a reflection, the system creates a candidate memory, and the same protocol runs again. The candidate survives only if risk is strictly lower.
Using the same protocol matters. Two uncontrolled one-shot generations can reward sampling luck instead of better memory. If evaluation is stochastic, the paper's confidence gate takes repeated independent probes and accepts only when the candidate's confidence interval falls below the current memory's interval.
The guarantees require strong assumptions. The verifier must be calibrated so zero verifier risk certifies zero task error. Useful proposals must keep appearing with enough probability, and accepted proposals must reduce risk proportionally. Under those conditions, the authors derive geometric or polynomial convergence rates. With an incomplete test suite, the guarantee covers verifier risk only, not true task utility.
This resembles the deployment pattern in TRUSS shadow runs. A generated behavior change runs in a controlled copy before it reaches the live system. Reflection needs the same separation between proposal and promotion.
Grounding won in the controlled experiments
The researchers use a hidden-cap Resource Contest and three Overcooked layouts with frozen MiniMax-M2.7 agents. Every metric comes from environment ground truth rather than an LLM judge. In Overcooked, an exact breadth-first-search value table measures joint-action steps to the next delivery.
Grounded SRMA beat a text-only self-gate on all three layouts. Scores improved by 14.3%, 27.3%, and 30.0%. It reached the first delivery in 22, 26, and 32 mean steps, compared with 26, 35, and 45 for self-gating. In the gate analysis, free-form reflection accepted every harmful proposal, the self-gate accepted 34.5%, and grounded SRMA accepted 6.2%. Final risk was 0.42, 0.28, and 0.14 respectively.
The 6.2% result deserves attention. SRMA is monotone in the verifier's own risk, but the researchers classify downstream harm with an independent oracle. Those remaining bad accepts measure verifier-to-oracle miscalibration. Grounding still depends on the quality of the grounder.
Stochastic evaluation produced a clear cost-quality curve. A one-shot verifier falsely accepted 28.4% of worsening proposals. Five fixed probes reduced that to 6.8% at 225 verifier calls. An adaptive confidence gate reached a similar 7.1% rate with 82 mean calls, which was 63.6% fewer than the fixed-five setup.
The controlled results say more than the headline SWE-bench comparison. On 500 tasks, a Kimi K2.5 SRMA system resolved 72.2%, while the public mini-SWE-agent v2 reference reported 70.8%. Those are not controlled method-only runs, as the paper notes. The matched-budget comparisons are cleaner. Kimi free-form multi-agent reflection resolved 58.4% versus 72.2% for SRMA. DeepSeek mini-SWE v2 resolved 68.2% versus 71.4% for Bilevel SRMA.
The official code repository currently includes Resource Contest and Overcooked, along with prompts, configurations, tests, and per-seed trajectories. Its README says the SWE-bench scenario will be released separately. Readers can inspect the controlled mechanisms now, but cannot yet reproduce the paper's software-repair result from the public checkout.
Give memory its own promotion lane
A useful audit trail keeps the raw reflection proposal, accepted memory, and verifier evidence separate. The critic should not overwrite durable memory in place.
The verifier needs to observe the system the memory claims to describe. A code agent can run tests, compilation, static checks, and the relevant reproduction. A tool-using agent can replay a fixed task pack against recorded or sandboxed API behavior. If the workflow has human consequences, log the review decision and the evidence behind it.
Current and candidate memory should face the same fixed evaluation. Stochastic outputs need multiple probes. The audit record should include the acceptance threshold, probe count, random seeds where possible, and both scores. Stored memory also needs re-evaluation when the environment changes because a previously helpful instruction can become stale. The paper calls this re-anchoring under piecewise-stationary environments.
CooperBench measured a coordination tax, and CCBench showed how quickly a time budget can disappear. Memory verification adds calls, so its test pack must be cheap enough to run repeatedly and strong enough to catch meaningful regressions. The adaptive-probe result offers a practical compromise: spend more verification only when the comparison is uncertain.
The paper assumes bounded coupling, finite action sets, calibrated verification, and a continuing supply of corrective proposals. It does not prove that every agent memory can converge safely. When the same words can be right in one environment and wrong in another, however, rereading them cannot reveal which environment the agent occupies. Durable memory needs an external test.
