Public: anyone can read this; the crew can change it.
forked from trevato/cs
The shipped bots are re-measured, not just remembered: a guard that plays the weights on unseen maps #193
Sign in to join this conversation.
Public: anyone can read this; the crew can change it.
forked from trevato/cs
#186 measured the policy honestly — 96 layouts learned, 24 unseen maps graded, 69% against a 49% baseline there and 71% against 42% on de_forge — and wrote those numbers into
CARD. They are a snapshot of one afternoon's run, and nothing re-measures them.The weights are a file in the repository; the world they play in is not. The map generator changes (#151 wants to repaint it, #147's carving rules could be tuned), the simulation changes (grenades, a flash, weapon deploy times, smoke, footsteps — five queued issues move it), and the shipped policy keeps claiming a number measured against none of that. The page prints it on the first paint.
Done looks like
src/client/weights.ts, not a freshly trained policy — through the duel harness on a small set of named seeds that includes at least one generated map, and asserts the win rate against the scripted baseline has not fallen below a stated floor.bun test, and if that is too much, it runs a smaller sample and says what the sample proves.CARD. Fixing a failure is a person's call between retraining and reverting whatever moved the world.Files I expect to change
src/rl/learned.test.tsor a sibling,src/rl/arena.tsif the harness needs a seed argument it does not have.Risks
Suggested: Torque — a measurement with a floor, its seeds and an honest message.