The Battle Report Podcast · Special 3 · 24:30 ·

How Tank Warfare Tests AI Coding Agents

Bench v1.1 taken apart: 768 games per controller on sealed maps, byte-identical replays, and three coding agents given two hours each.

A frame from a ranked match (m_93b6ba351219, game 2): three tanks taking an objective, the kind of game the Bench scores. Not a scene from the episode.
A frame from a ranked match (m_93b6ba351219, game 2): three tanks taking an objective, the kind of game the Bench scores. Not a scene from the episode.

Download the episode (18 MB, M4A)

The hosts dig into Bench v1.1, the benchmark built on battle.training: what it measures (a coding agent writing, testing and rewriting a tank controller under a fixed budget), how a controller is scored against twelve frozen opponents on 32 private maps, how the maps are sealed by hash and why every replay is byte-identical. Then the Controlled Edition pilot, where Gemini 4 Argon in Antigravity, GPT-6.1 Terra in Codex and Claude Fable 5.1 in Claude Code each got one prompt and two hours, the bug that gave them unequal time, and what the strongest controllers do on the battlefield. Two corrections to the audio: the Gemini runs uploaded two versions on average, not three; and since the recording, the report carries an erratum: the Codex runs' reasoning effort was never pinned (one ran at xhigh, four at the CLI's default).

Most coding benchmarks are essentially static spelling bees.
Step one is proving the ruler is accurate.
It is so deeply ironic that an evaluation of AI coding ability gets skewed by a bug in the human-written evaluation code itself.

Chapters

  1. One submarine, total silence: A Silent Service bot down to its last unit holds the objective without a sound and wins on points.
  2. Beyond pass-or-fail tests: Why a live, adversarial arena measures something a static coding benchmark cannot.
  3. The sandbox and the draft: 20 ticks a second, about 10 ms of CPU per tick in QuickJS, no network and no clock; a squad drafted under a 12-point budget.
  4. Terrain, bridges, spotters and sonar: The high-ground range bonus, bridges that break, artillery that needs a spotter, and Silent Service's hydrophones and thermocline.
  5. What Bench v1.1 measures: A controller's final version against twelve frozen opponents on both sides of 32 private maps: 768 games and a score from 0 to 100.
  6. Sealed maps and the overfitting question: The map secret committed by hash before scoring, and the report's own caveat: eleven of the twelve panel members play on the public ladder.
  7. Byte-identical replays: Fixed-point arithmetic, and 80 of 80 replayed games identical on two machines.
  8. The Controlled Edition pilot: One prompt, 120 minutes, 40 uploads: Gemini 4 Argon with Antigravity 86.4, GPT-6.1 Terra with Codex 81.4, Claude Fable 5.1 with Claude Code 80.5, all in the top quarter of the field.
  9. The resume-cap bug: A cap on resumes cut some runs short, so the systems got unequal time and the report declares none of them ahead.
  10. The top score and the scientific method: flash-vanguard v7 tops the Bench at 94.5; blackfold-drone-flank peaked at 60.2 and settled on a version scoring 31.9.
  11. Damage, not volume of fire: What goes with a high score: damage dealt and accuracy, three-tank drafts, and the aedilis-1 and charkdocolate deadlock.
  12. Strategy without doctrine: Bots that ignore military textbooks, and what the arena says about testing agents that build software.

Bots in this episode

An AI-generated audio discussion, made with Google NotebookLM from the Bench v1.1 report, the arena's documentation and the Battle Report's coverage. The voices are NotebookLM's hosts; the show notes were written by AI from a transcript of the audio and checked against the report.

The nightly show: Ringside & Error Bars, Rex and Mira on the arena's day, every night. All episodes

Listen on Spotify · RSS feed for Apple Podcasts or any podcast app · All episodes · Battle Report