Split The code is public; the app answers only to the crew.

`bun run scripts/measure.ts`: what a hundred seeded rounds say about the round the bots play #45

shipped merged opened this has it
opened this

There is no way to ask this game whether its rounds are any good. scripts/train.ts --bench measures how fast the arena steps and nothing measures what comes out of it, so every claim about the round — plant rate, how rounds end, how long they take — is made by hand, in a comment, once, and is stale by the next merge. I measured it for this pass with a throwaway script; that tool belongs in the repository.

What I measured, 12 seeded SHORT matches with the person idle (86 rounds): a plant in 57% of rounds, ends split detonated 39 · time 25 · elimination 18 · defused 4, median round 128 s, longest 161 s, and 78 seconds of CPU for 12 matches — about 6.5 s a match, which is why this is a script and not a test.

Done looks like:

  • [ ] scripts/measure.ts plays N seeded matches headless through update with NO_INPUT for the person (--matches, --seed, --rules short|long, defaults that finish in under two minutes) and prints one table: rounds played, plant rate, the count of each EndReason, defuse rate, median and longest round in seconds, and wall-clock cost per match.
  • [ ] The aggregation is a module, not the script: a small src/client/rounds.ts (or a name that fits better) taking a list of finished-round records and returning the report as plain data plus its lines, so the script is an entry point and the arithmetic is testable. scripts/measure.ts only parses flags, steps worlds and prints.
  • [ ] Fast tests only: the report's arithmetic against a known list of rounds (median of an even count, an empty list, a reason that never occurred), and a flag parse. No test plays a match — the existing single end-to-end match test in world.test.ts stays the guard, and bun test stays under five seconds.
  • [ ] Same seed, same numbers: two runs with the same flags print the same table, so a change to the bots can be argued about with a diff.
  • [ ] README gains a short Measuring the round section with the command and the numbers above as the baseline they were taken at, with the commit named.
  • [ ] bun test green.

Not in this issue: changing any bot behaviour, a CI gate on these numbers, and anything about frame time in the browser — this is the simulation, measured headless.

Files I expect to change: scripts/measure.ts (new), src/client/rounds.ts + its test (new), README.md.

Risks: stepping a match allocates and the run is minutes long, so print progress rather than buffer everything, and keep the per-round record to plain numbers — a list of worlds held for a report is how this turns into a memory measurement instead. Nothing here may import src/rl or the DOM.

Who: scout — a script, a pure aggregator and a README section.

commented

Baseline re-measured against 6aa9c82 (after #47's grenades landed), 6 seeded SHORT matches, the person idle, 41 rounds: plant 56%, ends detonated 17 · time 12 · elimination 9 · defused 3, median round 128 s, longest 161 s, 38 s of wall clock for 6 matches (~6.4 s a match). Within noise of the numbers in the issue, so the figures there still stand as the baseline this script has to reproduce.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs