Badger Code prices one fully solved task at 1.12 million tokens
Coding-agent benchmarks usually put success and efficiency in separate columns. Badger Code combines them. Its formula subtracts 0.01 from a Terminal-Bench score for every million input and output tokens used across the run.
The penalty looks small until it is converted into tasks. On an 89-task suite, one fully rewarded task adds about 0.01124 to the mean score. Dividing that gain by the competition's 0.01-per-million-token penalty gives an exchange rate: one additional perfect task offsets roughly 1.12 million extra tokens.
This is the incentive embedded in one competition, not a universal price for correctness. It tells entrants where the organizers want them to work: on agent scaffolds that solve more without letting retries, context, and verification expand without limit.
An open-weight contest with a real constraint
Badger Code is a community hackathon hosted by ML+X at the University of Wisconsin-Madison. Entrants build an autonomous coding agent and evaluate it on all 89 tasks in Terminal-Bench 2.1, with one attempt per task. The submitted system must use publicly downloadable model weights and fit within a total reported VRAM budget of 96 GB. Weights may be quantized to 4-bit or higher.
The memory limit applies to the whole model system, not each component. A planner and coder spend the sum of their reported VRAM. The competition's accounting convention includes checkpoint size, a 16,000-token KV cache, and roughly 2 GB of headroom per model. Actual peak memory may differ, but every entrant uses the same reporting rule.
There is no cash prize and no subjective judging of the writeup. Teams publish a repository at an exact tag or commit, disclose every model and quantization, report the hardware and total tokens, and submit a short project report. Organizers say they will spot-check entries, rerun the top five, verify token counts and checkpoints, and inspect code for task-specific hardcoding. The current rules require submission repositories to use the MIT or Apache 2.0 license.
The harness becomes the interesting variable. Model choice still matters, but entrants cannot hide a closed model behind a small open planner or stretch across a giant cluster.
The 1.12 million-token exchange rate
Badger Code calculates its leaderboard score as:
leaderboard_score = TB_score - 0.01 * (total_tokens / 1,000,000)
TB_score is the mean reward across 89 tasks. If one task moves from zero reward to full reward while everything else stays fixed, the mean increases by 1 / 89, or about 0.011236. A million additional tokens subtracts 0.01. The break-even calculation is therefore:
(1 / 89) / 0.01 * 1,000,000 = 1,123,596 tokens
The formula does not recommend spending 1.12 million tokens on every stubborn task. It says an extra full reward is still worthwhile at that token cost if the change does not harm other tasks. The competition notes that a typical complete run uses one to three million tokens, so a single recovered task can outweigh a large fraction of a normal run's token bill.
Cutting 500,000 tokens improves the leaderboard score by 0.005, less than half the value of one full task. A shorter transcript does not automatically redeem a small accuracy loss. The metric favors capability first, then uses efficiency to separate systems with similar results.
Partial task rewards complicate the picture. The exact value of a fix depends on the reward gained, and language-model sampling means a one-run improvement may not repeat. The 1.12 million figure is a conversion implied by the formula, not an empirical claim about how much any task should cost.
What the formula encourages
The obvious optimization is fewer wasted loops, not merely shorter answers. Trim command output and retrieve only relevant files. Detect repeated failures. Stop once the verifier evidence is sufficient. These changes can reduce cost without lowering the agent's chance of finishing.
Verification needs more care. A second test run consumes tokens only when its output returns to the model, and it may prevent a false finish. Removing verification can make a trace look efficient while losing full-task rewards. Under Badger Code's exchange rate, a broad token cut that causes one additional failure may need to save more than 1.12 million tokens just to break even.
Context compression presents the same tradeoff. Summarizing a long trajectory can lower future input usage, but an overaggressive summary may discard the error, constraint, or file path needed to finish. Entrants should compare changes at the job level: reward gained or lost, input tokens, output tokens, wall time, and failure category.
Our context-budget guide covers retrieval and compaction choices. The Boundary-Bench analysis shows another source of expensive loops: agents may keep working after a sandbox policy blocks their preferred action. Both problems are scaffold problems before they are model problems.
The benchmark version matters
Badger Code currently uses Terminal-Bench 2.1, not the newer version displayed on the general Terminal-Bench homepage. The competition says 2.1 retains the same 89 tasks as 2.0 but fixes 26 of them, including bugs, timeouts, resource limits, and reward-hacking defenses. It warns entrants not to compare 2.0 and 2.1 scores directly.
Runs use Harbor, the container-based framework that supplies the agent, task, environment, and reward interfaces. Harbor writes per-task result.json files containing the reward and token counts used for the submission card. The starter repository includes a minimal ReAct agent, a walkthrough, and a VRAM estimator, although the live Kaggle page is the authority for current rules.
Benchmark tooling and competition rules change. Pin the dataset identifier, harness commit, model checkpoint, quantization, system prompt, and scoring script with every reported result.
Badger Code's formula does not settle the best way to value agent tokens. It makes the tradeoff inspectable: did the longer trace purchase enough additional verified work?
