Ecdysis raised reported harness accuracy 18.56% by grouping failures before patching
An agent fails an airline refund task. Does its runtime harness need a patch, or did the model make one bad decision?
Self-editing systems tend to skip that question. A failed trajectory becomes a bug report, a coding model writes a patch, and the next failure produces another patch. Code churn can then reward a model's quirks instead of fixing a defect shared across tasks.
Ecdysis, a new research release from a team led by Yu Cui, groups failures across tasks before proposing a harness change. The authors report that the full method raised average accuracy from 58.67% with serial self-evolution to 69.56% across five models and three datasets. That is an 18.56% relative gain. On the harder airline subset, harness training was 1.84 times faster.
One failure records one event. Several failures with the same shape offer better evidence that the interface, tool contract, memory policy, or control loop needs repair.
The harness changes a frozen model's behavior
A runtime harness prepares observations, exposes tools, executes actions, interprets feedback, manages context, and decides when a trajectory continues or stops. Altering that layer can change how a frozen model behaves without updating its weights.
Ecdysis builds on Life-Harness, which learns reusable runtime interventions from training trajectories. Every evolution method in the new comparison starts from the same human-augmented Life-Harness base. The model weights and evaluation environments remain fixed.
The serial self-evolution baseline processes failures one by one. Each failed instance calls a coding agent, and those code changes accumulate into the next candidate harness. Ecdysis combines evidence from several tasks first. Its simpler form creates one round-level modification plan from the batch, allowing repeated failures to share one diagnosis and one coding pass.
The full system adds Failure-Driven Collaborative Refinement, or FDCR. Analyst, critic, engineer, and moderator roles examine the grouped evidence and produce a structured modification specification. A separate coding agent applies it. OpenCode with DeepSeek-V4-Pro handled code modification in the experiments, and the four FDCR roles also used DeepSeek-V4-Pro.
Diagnosis and implementation happen in different stages. The coding agent does not have to infer the meaning of a single failure while editing the system around it.
What the experiment covers
The researchers tested Qwen3-8B, Qwen3-14B, Qwen3-32B, MiniMax-M2.7, and Llama-3.1-8B. The tasks came from the Airline and Retail subsets of tau2-bench and from AgentBench. The tau2 tasks require conversational agents to call tools that alter environment state.
Each tau2 subset contributed 20 training tasks and 20 test tasks. Every held-out task ran three times, producing 60 test trajectories for each model, subset, and method combination. Ecdysis evolved the harness with Qwen3-8B for at most three candidate-generation rounds, froze it, and then tested it with all five models.
Across the ten combinations of model and tau2 domain, full Ecdysis averaged 59.33% accuracy. Serial self-evolution averaged 46.67%, while the fixed human-augmented harness averaged 51.67%. Pass@3 moved from 63.5% under self-evolution to 71.5%. The share of tasks solved on all three trials moved from 29% to 45%.
When AgentBench is included, the paper reports the broader 18.56% relative improvement: 69.56% average accuracy for full Ecdysis versus 58.67% for self-evolution across five models and three datasets.
The aggregate hides several mixed cells. On tau2 Retail with MiniMax-M2.7, Ecdysis without FDCR scored 100%, while the full version scored 96.67%. On AgentBench with Qwen3-8B, full Ecdysis and self-evolution both scored 90%. Llama-3.1-8B remained weak on tau2 Retail, reaching 11.67% with the full system. The result is an average improvement, not an unbroken sweep.
Aggregation did most of the speed work
Serial self-evolution took 1,831.4 seconds on Retail and 8,120.6 seconds on Airline. Full Ecdysis took 1,405.9 and 4,403.0 seconds, equal to 1.30 and 1.84 times speedups. The aggregated version without FDCR was faster: 1,292.4 seconds on Retail and 2,510.8 on Airline, or 1.42 and 3.23 times speedups.
Grouping failures removes repeated coding calls. Collaborative diagnosis spends part of that saving on a stronger modification plan. The paper describes aggregation as the primary efficiency mechanism and FDCR as an accuracy-oriented component that adds evolution cost.
API costs followed the same pattern. On Airline, self-evolution cost $6.382, aggregation without FDCR cost $2.136, and full Ecdysis cost $2.609. Those are measurements from the authors' models, infrastructure, and provider prices, not a reusable price estimate.
The public release is not a complete reproduction package
The Ecdysis repository publishes the main implementation, tests, experiment scripts, failure processing, artifact handling, patch application, replay checks, and multi-agent analysis modules. Its README says local experiment configurations, raw benchmark data, generated traces, private run artifacts, and credentials are excluded.
Researchers can inspect the mechanism, but reproducing the paper requires reconstructing part of the setup. Experiment scripts expect user-supplied YAML files, and the exact run files are absent. The top-level repository files currently declare no software license. Public code without a license is visible, but that does not grant permission to reuse it.
The paper names the collaborative stage FDCR. The repository README calls it Multi-Agent Debate, or MAD. The roles and structure appear aligned, but a replication should pin a commit and record which terminology and configuration it follows.
A cautious harness-improvement loop
Start by saving full trajectories, including tool inputs, outputs, state changes, and the active harness version. Group failures by behavior or violated contract rather than error string alone. A general harness patch should answer evidence from several tasks.
Write the modification specification before touching code. Apply the patch in an isolated environment, replay affected tasks, and run a held-out set. Keep the previous harness available. Accept the candidate only against a score chosen before the test.
The evidence has to reflect the agent's real environment. Code Assistant updating its edit history after a formatter changes a file is one example of a harness repairing stale state before the next model action. COBRA-Skills' selective evaluation loop tackles a neighboring budget problem: which candidate deserves another costly test.
A single failure may call for a retry, a better task definition, or no change. Repeated failures across distinct tasks are stronger evidence that the harness needs work.
