In late July I entered a one-week bot competition for Generals, an imperfect-information real-time strategy game, run with a $3,000 first prize. I wrote my own self-play training stack in JAX on top of a C/CUDA engine, rented GPUs by the hour on a student’s budget, and finished 9th of 114. This is what I tried, in the order I tried it, with the numbers attached — including the experiments that were wrong, the two bugs that mattered more than any architectural idea I had, and the reason I now distrust most of my own ablation results.

FINAL RANK9/ 114
TRAINING1.764B steps
POLICY2.72M params
INFERENCE20.7 ms / 150

The through-line arrived about four days in, and it is arithmetic rather than insight. On a fixed compute budget you are not optimising learning. You are optimising learning per second, and almost every intervention that buys you one costs you the other.

Before any of that, the thing itself. This is the exact submitted policy, exported from Equinox to ONNX and running in your browser.

FIGURE 1 · LIVE POLICY Play the submitted agent—or let it play itself.
G08 · 2.72M PARAMETERS The exact exported policy, running locally in your browser. Open full screen ↗

The brief is four numbers

Generals is played on a small rectangular grid under fog of war. You own a general that spawns army every other turn; you move stacks of army into adjacent tiles to claim territory, and you win by walking a bigger stack onto the enemy general. You can only see tiles adjacent to something you own, so most of the board is a memory problem rather than a perception problem.

The 2026 competition ruleset differs from public Generals in ways that change training rather than flavour. Maps are rectangular with each axis drawn independently in 18–21, so the network cannot assume a shape. There are no neutral castles on the map: castles are built, not captured, at a cost of 35 army plus a crowding surcharge of max(0, 14 − 2d) against your nearest existing castle, and building consumes your turn. Generals spawn at least 17 BFS steps apart. From turn 800 mutual contact is deathtouch — both generals die, scored as a draw — and the game truncates to a draw at turn 1200.

Those rules are the interesting part of the problem. The four numbers that actually constrained every decision are on the other side of the submission boundary:

Figure 2  ·  The submission budget, and how much of it I used

TIME PER MOVE 20.7 ms used 150 ms limit 86% of the latency budget unused MEMORY 10.5 MB of weights 2048 MB cap HARDWARE AT MATCH TIME 1 CPU core no GPU no network 50 faults forfeits the game CONSEQUENCE Whatever you train must run as a numpy program, single-core, in under 150 ms. The 2.72 M CNN was sized for that limit and undershot it by 7×.
Figure 2Match-time limits (hatched) against what the final submission actually consumed (solid). Latency was measured on the CPU numpy bot over 663 turns with zero action mismatches against the JAX reference and a maximum logit difference of 8.2 × 10−5. The unused 86% is the clearest thing I got wrong: I sized the network against a limit I never approached.

I sized the policy at 2.7 million parameters early, on the assumption that inference would be tight, and never revisited it. It was not tight. The final bot spent 20.7 ms of a 150 ms allowance and 10 MB of a 2 GB allowance. There was room for a network several times larger, or for a shallow search on top of the policy, and I spent the week improving the training of a model that was smaller than the rules required. That is a planning error, not a research finding, and it is the first thing I would change.

One equation

Fix a strength target — say, beating a frozen reference checkpoint by a statistically significant margin. Let \(N\) be the number of agent-steps needed to get there and \(R\) the agent-steps per second the stack sustains. The thing you actually spend is

\(T = N / R\)time to a fixed strength target = steps needed ÷ steps per second

which is trivial, and which I nonetheless kept reasoning about only one term at a time. Nearly every change I made moved both terms, in opposite directions. A richer opponent mix makes each sample more informative and makes every rollout slower. Reusing more of each rollout in the PPO update raises the work per sample and lowers the samples per second. Reward shaping, auxiliary losses, larger minibatches, augmentation, a bigger network: all of them trade \(N\) against \(R\).

So the acceptance test for an intervention is not “does it learn in fewer samples”. Writing \(a = N’/N\) for the fraction of samples the new recipe needs and \(b = R’/R\) for the fraction of throughput it retains,

\(\dfrac{N'}{R'} < \dfrac{N}{R} \quad\Longleftrightarrow\quad \underbrace{1-a}_{\text{sample-efficiency gain}} > \underbrace{1-b}_{\text{throughput cost}}\)the fractional reduction in samples has to exceed the fractional loss in throughput

Everything above the diagonal in Figure 3 is worth doing; everything below it is a slower way to reach the same policy.

This is arithmetic a first-year student can do, and I still got it wrong repeatedly, for a structural reason worth naming: sample-efficiency gains are legible and throughput costs are not. A better learning curve shows up within an hour and feels like progress. A 40% throughput cost shows up at the end of the day as “I ran three experiments instead of five”, which does not feel like anything at all.

Figure 3  ·  The break-even plane

drag the point, or load a real ablation

worth doing time-to-target falls not worth doing time-to-target rises break-even 0 20 40 60 80% throughput cost 1 − R′/R 0 20 40 60 80% sample-efficiency gain 1 − N′/N adv_top_frac 0.50 −38% cost, −38% gain 20% frozen opponents −44% throughput 50% transformer opponent −53% throughput gain unmeasured for both — they are plotted at zero
time to the same strength 1.00×
exactly break-even
measured
Figure 3An intervention is worth taking only if it sits above the diagonal. Three of the four measured points are on or below it. adv_top_frac 0.50 landed on the line to two significant figures and duly produced a statistical tie in 2,048 games (§7). The two opponent-mix points have a measured throughput cost and an unmeasured sample-efficiency gain — which is precisely the asymmetry that makes this mistake easy to make. FP8 sits off the plane to the left: it bought throughput and destroyed the policy (§9).

The 11.7× I did not take

The training stack is a JAX/Equinox PPO trainer built on a C and CUDA game engine from PufferLib. The policy, cnn_global, is a 2,724,746-parameter residual CNN: a 45-channel observation at 21 × 21 (31 base channels plus history stacks, including scouting and deathtouch planes), a 1 × 1 stem, six 192-wide 3 × 3 residual blocks with a global-context vector injected into each, a 4,410-way masked policy head laid out as ten action planes over the board, and a 128-bin HL-Gauss value head. Trained with Muon throughout.

Early on I measured the two stacks against each other on one A100-40GB, and the result was not close. PufferLib’s native CUDA path ran 391,412 agent-steps per second; my JAX trainer ran 33,512. A factor of 11.7. The whole measurement cost twenty-five cents, and for about a day I treated switching as obviously correct.

It was not, and finding out why produced the single most useful process change of the week. When I audited what the native backend had actually instantiated, it was not the CNN. The custom CUDA Generals CNN was shape-gated for an older 18-channel, 4,609-action layout; the competition environment presents 19 channels and 5,185 actions, so the gate silently fell through to a generic flat encoder with a MinGRU — 4,325,888 parameters, a completely different model. It was 11.7× faster because it was doing something much easier.

I trained it anyway, to be sure. It reached 100,139,008 agent-steps in six minutes and thirty-five seconds at 252.5k SPS, entropy fell from 8.55 to 2.08, and every training counter moved. Then I evaluated it across stacks:

Figure 4  ·  Throughput of a model you cannot use

TRAINING THROUGHPUT · one A100-SXM4-40GB, identical box PufferLib native CUDA flat encoder + MinGRU · 4,325,888 params 391,412 SPS JAX / Equinox history transformer · 5,128,274 params 33,512 SPS 11.7× slower WHAT THE FAST MODEL LEARNED · after 100 M steps vs 7 scripted bots · 448 games 0 wins 29 losses 419 draws vs my champion · 128 games 0 W · 112 L · 16 D mean game length 1,145–1,200 turns — it had learned to stall
Figure 4The 11.7× throughput advantage was real and the model behind it was worthless: no scripted opponent conceded a single win in 448 games, and games ran to the 1,200-turn truncation. Throughput is only a denominator when the numerator is the model you intend to ship.

Out of that came a rule I now apply before renting anything, and which I would defend as a general practice in RL engineering: the runtime architecture identity gate. Before a run is allowed to spend money, it must log and assert, from the live objects rather than from the configuration file, the resolved encoder and decoder implementations, the observation shape, the action shape, the parameter count, the recurrent-state shape, and a fingerprint of the parameter layout. A config name is a request. A backend is free to ignore it and frequently does.

A learning smoke test proves that the resolved model learns. It never proves that the model you asked for was the one selected.

Two bugs worth more than any architecture change

The two largest single improvements of the whole week were both corrections to things I had already implemented and believed were working. Neither was a new idea. Both had the same shape: the arithmetic was locally defensible and the trajectory distribution was wrong.

4.1 · The draw penalty that never reached the loss

Draws are the failure mode of this ruleset. A policy that accumulates a large economy but cannot walk into an enemy general before turn 1200 scores nothing, and on a competition ladder that is indistinguishable from being bad. So I added a penalty on truncation: a draw is worth −0.5, later −1.5 in the shaped return. Draw rates did not move. I assumed the coefficient was too small and turned it up. They still did not move.

The penalty was never reaching the gradient. Truncation was being handled as a reset rather than as a terminal, in three separate places that were each individually reasonable:

  1. the value bootstrap read \(V(s_{t+1})\) from the state after the environment had reset, i.e. from a fresh board;
  2. the GAE carry was cut at the episode boundary, so nothing propagated backwards past it;
  3. the final row of the rollout — the only row carrying the −1.5 target — was masked out of the loss as an incomplete transition.

Each of those is a thing you do somewhere in a PPO implementation. Together they made the penalty exactly invisible: the one row that contained the signal was the one row excluded from the update. The fix is to treat truncation as a true terminal in the advantage computation,

\(\delta_t = r_t + \gamma(1-d_t)V(s_{t+1}) - V(s_t), \qquad \hat{A}_t = \delta_t + \gamma\lambda(1-d_t)\hat{A}_{t+1}\)with \(d_t = 1\) on a 1200-turn truncation, and the row restored to the batch

Figure 5  ·  Where the draw penalty went

BEFORE — the penalty never reaches the gradient turn 1200 reset t · · · r = −1.5 new 1 · bootstraps V from the post-reset board 2 · advantage carry cut at the boundary MASKED 3 · the only row with the target is dropped net effect: ∂L/∂θ contains no draw term at all Each of the three is a reasonable thing to do somewhere in a PPO loop. Together they delete the reward term exactly. AFTER — truncation is a true terminal turn 1200 reset t · · · r = −1.5 new 1 · d = 1, so (1−d)·V(s′) = 0 — no bootstrap 2 · the −1.5 propagates back through GAE KEPT 3 · the row stays in the loss held stable to iteration 1,700 · scripted 97–100% W · self-play draws 6–9% Draw rate responded to the coefficient for the first time. One-line change; the largest single gain of the week.
Figure 5The three defensible decisions that jointly deleted a reward term. The fixed run stayed healthy through 1,700 iterations. It also exonerated a feature I was about to remove: the no-forced-build control had collapsed before the fix, so the collapse I had blamed on forced-build exploration was the missing draw penalty all along.
THE LESSON

A reward you cannot find in the gradient is a comment. I now trace the full path — config, rollout, return, loss, metric — before changing a coefficient, because turning up a number that reaches nothing produces a very convincing null result.

4.2 · Symmetry augmentation applied one step too late

A Generals board has the eight symmetries of the square: four rotations and four reflections. That is free data augmentation, and the first implementation looked like this: collect a transition in the canonical orientation; inside each PPO epoch, draw a random \(g \in D_4\) and transform the stored observation, action and mask; then recompute the behaviour log-probability \(\log \pi_{\theta_{\text{old}}}(g \cdot a \mid g \cdot s)\) in the transformed frame.

That last step is the one that looks careful and is the reason it is wrong. PPO’s importance ratio

\(\rho_t(\theta) = \dfrac{\pi_\theta(a_t \mid s_t)}{\pi_{\theta_{\text{old}}}(a_t \mid s_t)}\)

is only an importance weight if \(a_t\) was sampled from \(\pi_{\theta_{\text{old}}}\) in state \(s_t\). The transformed action \(g \cdot a_t\) was never sampled by a policy looking at \(g \cdot s_t\); it was sampled by a policy looking at \(s_t\) and then rotated. Recomputing the denominator makes the arithmetic self-consistent and the estimator biased. As a bonus, a given sample could receive a different frame on each PPO pass, which adds gradient variance for nothing.

Figure 6  ·  Where the group action belongs

BEFORE — g applied to stored samples env π samples a | s buffer PPO epoch apply g · recompute log π_old(g·a | g·s) g·a was never sampled in state g·s — the ratio stops being an importance weight and a sample can draw a different frame on every pass AFTER — g drawn per episode, before inference env apply g to obs + legal mask one g per episode π samples a′ | g·s buffer stored in frame g g⁻¹ applied only to the action sent to env.step RESULT · 2,048 seat-balanced games each corrected D4 vs no D4 1083 – 949 – 7 margin +134 bar ≈ 74 corrected D4 vs frozen parent 1180 – 854 – 3 +326 no D4 vs frozen parent 1076 – 966 – 1 +110 the control also beat the parent — which is why only the first row is evidence
Figure 6The correction moves one operation from the update into the rollout. Drawing the frame before inference restores PPO's on-policy identity and gives the policy a temporally consistent coordinate system for the whole episode. Note the third row: the no-augmentation control gained +110 on its parent from continued training alone, which is the confound that §6 is about.

In on-policy RL a symmetry transform has to preserve the entire trajectory distribution, not merely the tensor shapes.

How I decided anything

Self-play win rate against yourself is always 50%, so every decision has to come from outside the run. Candidates were compared in seat-balanced games against a fixed opponent, and promoted only when

\(W - L \geq 1.65\sqrt{W+L}\)the normal approximation to a fair-coin binomial over the decisive games; 1.65 is the one-sided 95% point

Under the null the standard deviation of \(W-L\) is \(\sqrt{W+L}\). Draws are excluded from the test and reported separately, because in this ruleset a draw is a distinct failure mode rather than half a win.

Two things about that gate cost me real time to learn.

First, small evaluations are not cheap, they are free of information. I spent the first several days making calls from a 32-game greedy match against a random mover, because it ran in twenty seconds. The same checkpoint scored 97, 97 and 88 on three different boxes. At \(n=32\) the gate cannot resolve anything smaller than about 13 percentage points, which is larger than every effect I was trying to measure. It is a smoke test, and I had been reading it as a result.

Second, the eval scales sublinearly, so the cheap option was also the imprecise one for no reason. A fixed 50–75 second compile dominates the small sizes:

Figure 7  ·  Why the gate runs at 2,048 games

RESOLVING POWER · margin as a multiple of the significance bar 3.6× 6.0× 9.5× 512 1024 2048 games measured on one ≈68%-win matchup WALL CLOCK · seconds, WSL2 RTX 4070 158 s 228 s 338 s 512 1024 2048 hatched: the fixed 50–75 s compile, paid at every size
Figure 7Four times the games costs 2.1× the wall clock and halves the error bar. Once the compile is amortised there is no reason to run the imprecise version — which meant that for several days I had been buying uncertainty at nearly full price.

The ladder, and what it does not prove

With roughly 48 hours left I stopped redesigning the network. The 2.72 M CNN was frozen, training was pure self-play with Muon, and each experiment was a matched two-hour arm on 2× RTX 5090. Every candidate then played its sibling and its frozen parent at up to 2,048 seat-balanced games. Six generations were promoted:

Figure 8  ·  Six promotions, two causal claims

treatment vs frozen parent control vs frozen parent treatment vs control — the causal statistic promotion bar, ≈74
0 +100 +200 +300 margin W − L (2,048 games) promotion bar g03 corrected D4 augmentation +134 g04 minibatch 1024 over 2048 promoted; no direct treatment-vs-control record survives g05 two PPO passes +36 g06 GAE λ = 0.9 over 0.7 +172 g07 entropy floor 0.003 over 0.001 +83 g08 advantage top-50% over top-25% −21
Figure 8Every promoted arm cleared the bar against its frozen parent — and in three of the five generations with a recorded control, so did the control (g03, g06, g08). Beating your parent certifies the checkpoint; it does not identify the cause. Read down the diamonds instead: only g06 and g03 have a direct treatment-vs-control margin that clears the bar on its own.

Six promotions look like six successful ablations. They are not, and Figure 8 is the reason. In three of the five generations where I ran a matched control, the control also beat the frozen parent by a significant margin. Two more hours of self-play is itself a powerful treatment — the lineage had not plateaued — so beating the parent proves that the new checkpoint is better and says nothing about why. The only statistic that isolates the change is the direct treatment-versus-control match, and those margins are much smaller:

ChangeDirect W–L–DMarginEvidence
GAE λ 0.9 over 0.71102 – 930 – 13+172strong
Corrected D4 augmentation1083 – 949 – 7+134strong
Entropy floor 0.0031055 – 972 – 14+83moderate
Two PPO passes1033 – 997 – 7+36tie — promoted on the parent gate
Advantage top-50%1009 – 1030 – 2−21tie — promoted on a tie-break
Minibatch 1024not recorded—suggestive
EMA decay 0.9995 vs 0.999no successor—no signal
FP8 training0 – 2044 – 4−2044catastrophic

Two strong results, one moderate, three that do not separate from their control, one rejection. That is a fair summary of a week of ablations, and it is a much smaller claim than “six improvements”.

The ablation that predicted itself

The last generation is the cleanest illustration of the equation above, and I did not see it until I wrote the numbers next to each other.

adv_top_frac = 0.50 trains PPO on the top half of the advantage distribution by magnitude instead of the top quarter — twice as many selected samples from each rollout. The three measurements were: throughput fell from 26.9k to 16.6k agent-steps per second, a 38% cut; it matched the faster control while consuming roughly 38% fewer environment interactions; and head to head over 2,048 games it went 1009 – 1030 – 2, a tie.

Put those in the equation. The fraction of samples still needed is \(a = N’/N \approx 0.62\), and the fraction of throughput retained is \(b = R’/R = 16.6/26.9 = 0.617\). The acceptance condition \(a < b\) is not satisfied — it holds with equality, to two significant figures. The sample-efficiency gain paid for the throughput loss exactly and bought nothing on top, so \(T’/T \approx 1\). The tie in the head-to-head is what the arithmetic predicted before the games were played.

I promoted 0.50 regardless, using a tie-break I had declared in advance: the larger margin against the frozen parent, +111 against +80. In a competition with a deadline that is a reasonable way to settle a coin flip. As a claim about advantage filtering it is worth nothing, and I want to be explicit that g08 is my operational champion, not a replicated result.

The narrowing

The most interesting thing that happened all week is a failure I did not solve. Across three independent runs, the policy became monotonically stronger against its training distribution and monotonically worse at finishing games.

In the league run, the seven scripted opponents saturated — six of seven at 100% wins and 0% draws by iteration 725, and across the full evaluation panel the champion lineage lost zero games to scripted bots at any checkpoint I measured. Over the same window, greedy games against a random mover went from 1 draw in 32 at iteration 200 to 18 draws in 32 at iteration 1400. Mean owned cities rose from 3.28 to 4.87.

My first explanation was overfitting to the scripted pool, and it was wrong. The run that killed it had no scripted opponents at all — pure self-play against frozen copies of itself — and its draws against random still went from 1/32 to 13/32 by iteration 800, with cities rising 4.5 → 5.99 and entropy falling.

Figure 9  ·  Stronger against its own distribution, worse at closing

two measured endpoints per series — the segments are not interpolated curves

SCRIPTED OPPONENTS win rate, 7 bots 100% 0 100% · 0 losses iter 200 1400 DRAWS vs A RANDOM MOVER out of 32 greedy games 20 10 0 18 / 32 1 / 32 13 / 32 no scripted pool iter 200 1400 MEAN OWNED CITIES castles held at sampled timesteps 6 3 0 3.28 4.87 4.5 → 5.99 iter 200 1400 Build rate stayed flat near 2% throughout — this is accumulation over long games, not a build-rate runaway.
Figure 9Each series is drawn between the two checkpoints I actually measured; the connecting segments are not interpolated learning curves. The dashed blue series is the run with no scripted opponents, which is what falsified the scripted-overfitting explanation.

The third occurrence is the sharpest. In a round-robin between three arms of an identical recipe and seed that differed only in GPU count, the 2× arm at 45.9 million agent-steps beat the 8× arm at 104.9 million, 268 – 230 – 11. More than twice the samples, worse policy.

The mechanism I believe, and did not get to falsify: self-play rewards building — both seats build, castles convert into army, the economy compounds — but it never prices close-out speed, because a symmetric opponent is equally slow to finish. Build rate stays flat around 2% throughout, so this is accumulation over long games rather than a build-rate runaway. The resulting city-heavy policy cannot finish a game against a weak or erratic opponent, and “weak or erratic opponent” is exactly what the vs-random draw metric measures.

The cheapest falsification, which I ran out of time to run: anneal the build cost upward from its flat 35, or cap city count, and check whether the vs-random draw rate stops climbing while frozen-anchor win rates hold. A low-probability random-mover opponent in the training mix would also suppress the symptom, but it treats the symptom rather than the cause.

THE OPEN PROBLEM

Self-play generates a distribution that is stationary in a way the real evaluation is not. The metric that revealed it — draw rate against a deliberately weak opponent — is one I only started logging because a champion had already fooled me.

The rejection ledger

Most of what I tried did not work, which is the normal ratio and worth writing down explicitly. Three of the rejections share a structure that I think is the most transferable methodological lesson here: a microbenchmark measured a speed-up that the end-to-end system did not deliver.

Figure 10  ·  What the microbenchmark promised, what the run delivered

SPEED-UP MEASURED IN ISOLATION → SPEED-UP REALISED IN TRAINING microbenchmark end-to-end training run no change 0 +10 +20 +30 +40 +50% HL-Gauss value lookup 3,050 → 4,560 samples/s, synthetic Muon update +49.5% 0.0% both arms 74.7k SPS · the update was never the bottleneck FP8 training the one that kept its speed-up +21%, delivered → and then lost 0 – 2044 – 4 to the seed checkpoint post-step observation elision concurrent 4×4070S runs, then a sequential same-GPU rerun +0.94% → −0.34% 96,526 vs 96,855 agent-steps/s · below measurement noise Every one of these was measured honestly. The microbenchmark was simply not the system.
Figure 10Three optimisations whose isolated speed-up did not survive contact with the training loop. The HL-Gauss lookup was a 49.5% win on a synthetic update and zero on matched 2×5090 runs, because the update was not the bottleneck. I now reject a synthetic update benchmark as a proxy for training throughput at all.

The remaining rejections, briefly:

WhatResultRead
Exact CUDA port of the CNN forward pass 16.73 → 7.57 ms per batch-128 forward after channels-last GroupNorm, fused residuals, CUDA graphs and single-pass norm statistics — a 2.21× optimisation. JAX still did it in 6.27 ms. rejected Six 3×3 192-channel convolutions were 68.4% of kernel time; XLA already had them. Writing my own kernels lost to the compiler by 17–20%.
Conv+MinGRU student distilled from the champion 16.4 M teacher-labelled steps. Greedy action agreement rose 1.6% → 10.6% against an 80% gate; policy KL stuck above 4 against a 0.20 gate. rejected A 332k-parameter student at 10% agreement is not a compression of the teacher, it is a different policy.
Model soup across 2×/4×/8× arms Soup beat the 8× arm by +44 and tied the other two. rejected All three arms shared one recipe and one seed, so they sat in the same basin. Soups earn their gains from independent initialisations; there was no diversity to average.
Reverse-KL "castle magnet" Inert — build probability did not move. rejected Forward KL did raise build probability, and flattened the policy everywhere else. Forced-build exploration was the cleaner mechanism.
Counterfactual paired build rollouts The auxiliary objective fit its replay: preference accuracy 0.33 → 0.82 over 726 iterations. Natural build rate went 0.324% → 0.264%. rejected It learned to classify a sparse replay buffer without changing behaviour. Worse, a path bug meant the checkpoint mirror watched the wrong directory, so no weights survived the instance being destroyed.

The bottleneck was not the algorithm

I logged every GPU I rented. Over eight days that is 67 rental records across 54 instances, and exactly one of them is marked success in the failure column. The other 66 carry 41 distinct failure modes: no-ssh-in-budget ten times, image pulls wedged for 90 minutes, broken CDI GPU mappings, a host whose advertised $2.809/hr billed at $4.79, two spot preemptions, and one instance that kept billing for two hours after two destroy calls silently aborted.

The clearest picture of what that costs is a race I ran deliberately: five machines rented in the same minute, same image, same script.

Figure 11  ·  Five hosts, same minute, same image

minutes from rental to the first training step

0 1 2 3 4 5 6 min HOST CONTAINER → SSH → RSYNC → JAX SEES GPU → FIRST TRAINING STEP Finland $1.085/hr JAX 1.52 training 3.10 N. Carolina $0.946/hr 2.04 3.69 Washington $1.139/hr 2.27 4.50 Vietnam $1.074/hr 4.28 5.60 California $0.647/hr wedged 13 minutes, destroyed, never usable N. Carolina $0.959/hr · warm 0.64 — a cached image cuts time-to-GPU 3× 3.36 · 77,536 SPS
Figure 11Identical request, five providers, a 2.8× spread in time-to-GPU and one total loss. The 3.57 GB container image is the dominant term, which is why the warm-cache row is the interesting one: the same host reached JAX in 0.64 minutes instead of 2.04.

Money behaved the same way. The accountable spend across both providers is about $63 — $49 on Vast from the durations I actually wrote down and $14 on Prime Intellect from paired create/terminate records — and that is a lower bound, because roughly half the rentals have no recorded duration. One overnight lineage died mid-run when the balance hit zero. An 8×5090 at $4.50/hr was 61% of a $7.42/hr three-box burn against a $23 credit balance, which is a sentence I could have written before renting it rather than after.

The useful measurement to come out of all this is that the smallest box won on throughput per dollar. In the production league workload, 2×5090 delivered 40,038 SPS per dollar-hour against 33,020 for 8×5090; in near-pure self-play, 48,916 against 37,819. Multi-GPU scaling was 3.4–3.6× across a 4× device count, and the interconnect on rented consumer hardware is where the rest went. Dropping the scripted opponent pool bought another 15–22%.

THE HONEST ACCOUNTING

None of this is research. All of it is the reason the research budget was the size it was. The honest accounting of a compute-constrained week is that a large fraction of it went into making the compute exist.

What I believe, and what I do not

Three conclusions I would defend:

  1. Symmetry augmentation works when it is part of the behaviour policy. Applied to stored samples it is a biased estimator wearing a helpful costume. +134 over the matched control in 2,048 games.
  2. Longer-horizon credit assignment materially helped. GAE λ = 0.9 over 0.7, +172 direct and +247 against the frozen parent — the cleanest single optimisation result of the week.
  3. The 2.72 M CNN had not reached a capacity ceiling. Every generation still improved on its parent through 1.76 billion cumulative steps, and I shipped a model that used 14% of its latency budget.

And the things this evidence cannot support, which I think matter more:

  • One seed per arm. Everything above is a single training run per condition. Three independent 100 M-step seeds would have told me more about any one hyperparameter than one more 300 M-step continuation did.
  • Immediate-parent comparison hides non-transitivity. I never evaluated every generation against one permanent external panel. Self-play lineages can specialise into rock-paper-scissors relationships that a parent-only ladder cannot see, and I have three arms that beat each other inconsistently as evidence that this is not hypothetical.
  • 1.76 billion self-play steps is not 1.76 billion samples. Opponents, maps, advantages and value targets all come from adjacent versions of one network. The effective sample size is far smaller and I do not know by how much.
  • A leaderboard placing is one draw from a distribution. 9th of 114 is one submission on one ladder, and I would not read a rank difference of three places as a strength difference at all.

The version of this sweep I would run with a supervisor and a real compute allocation is not more steps. It is three seeds per arm, matched on both agent-steps and wall-clock; intermediate checkpoints rather than terminal ones; one permanent frozen anchor panel including the architectures I abandoned; and direct treatment-versus-control as the reported statistic, with the parent comparison demoted to a sanity check.

Ablation-maxxing is an excellent way to ship a stronger agent and a poor way to explain one. In self-play RL, continued training is itself a powerful treatment, and every ablation has to prove it contributed more than another two hours of self-play would have.

The competition result was 9th of 114, on a policy that moved in 20 ms against a 150 ms budget, trained for about sixty dollars. The part I would actually defend in a seminar is none of that. It is the observation in One equation and its one clean instance in The ablation that predicted itself: on a constrained budget the two levers are steps-to-target and steps-per-second, they almost always pull against each other, and the second one is much easier to lose without noticing. That is a question I would like to answer properly — across many environments, with seeds, and with the throughput term measured rather than assumed.

Notes on the numbers

All head-to-head records are seat-balanced and quoted as W–L–D from the first named agent's perspective. "Margin" is \(W-L\); the promotion bar is \(1.65\sqrt{W+L}\), which is ≈74 at 2,048 games. Agent-steps are counted as \(2\times\) environment ticks in a 1v1 game. Throughput figures are median train/sps over post-compilation iterations and are total across devices. The submission bundle for the final champion exports to numpy and matches the JAX reference at 0 action mismatches over 663 turns with a maximum logit difference of 8.2 × 10−5.

Figures 9 and 10 plot only measured endpoints; the connecting segments carry no claim about the path between them. The adv_top_frac sample-efficiency figure of ≈38% is derived from matched-wall-clock environment interactions, not from a separate steps-to-gate measurement, and is the weakest number in the section it appears in.