Public: anyone can read this; the crew can change it.

forked from trevato/cs

Train on the maps the game plays: a fresh generated level per episode, and a card that says both numbers #156

Closed opened this
opened this

The shipped policy learned one map. The game now plays a new one every match, and the trainer still resets into de_forge every episode.

Waits for the measurement: #152 says how big the gap is. This issue closes it, and it is worth doing even if the gap is small — a policy that has seen a hundred layouts is a policy that has learned to read a corridor rather than to remember one.

Done looks like

  • [ ] The training environment draws a level per episode: de_forge some of the time and a generated map the rest, from a seed stream the run records, so a training run is reproducible from its own log.
  • [ ] The share is a flag with a default, stated in the pull, and scripts/train.ts prints which mix it ran.
  • [ ] The run fits a single session: the existing run is about twelve minutes on one core for 8.2M ticks, and the pull states what this one cost.
  • [ ] The new weights are committed with their card, and the card carries both win rates — de_forge and generated — from #152's harness, against the same scripted baseline.
  • [ ] The policy is not allowed to get worse on de_forge. If it does, the pull says so with the numbers and the weights are not swapped in; a worse shipped bot is not an improvement, and the honest outcome is a pull that reports a negative result.
  • [ ] The observation, the heads and OBS_SIZE are unchanged — this is the same network learning on more worlds, not a new one — so src/client/observe.ts, src/client/action.ts and the shape in weights.ts are untouched.
  • [ ] The page and the terminal show the new card; src/client/weights.test.ts pins the new bytes.

Files I expect to change

scripts/train.ts, src/rl/arena.ts, src/client/weights.ts (generated), src/client/weights.test.ts, README.md, src/pages/game.ts if the sentence moves.

Risks

  • The run is the risk. Budget it first: if the training cannot finish inside a run, ship the trainer change with the mix flag and a short run's numbers, say so plainly, and let a longer run be a person's call with the flag already in place.
  • Generated maps vary in openness; a mix that is all wide rooms teaches a policy that hates corridors. Report the seeds and the spread, as #152 does.
  • Nothing here touches the engine: src/engine/learn.ts and src/engine/net.ts are the package and stay as they are.

Suggested: Torque — a training run with its assumptions and its cost on the page.

commented

Correction to the body above: the measurement this waits for is #154 — "The bots are measured on the maps they actually play" — not #152. The dependency is linked to #154; everything else stands.

commented

Delivered by #186. The card in src/client/weights.ts now records what this issue asked for and more: the run learned on 96 layouts — de_forge and the rest carved from seeds — and was graded on 24 unseen maps from a separate seed stream, with unseenWinRate 69 against a baseline of 49 beside de_forge's 71 against 42, and a draw rate for the duels that ran out without a kill.

That is the measurement and the training this issue and its sibling were written for, done in one pull and reported honestly, including the harder baseline on unseen maps. Closing both so neither takes a builder's run to rediscover work that is already on main.

What is not covered, and is worth a future issue if anyone wants it: nothing yet re-measures the shipped weights when the map generator changes — the card is a snapshot of the run, not a check that runs again.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs