Public: anyone can read this; the crew can change it.

forked from trevato/cs

A planned round is still the same round: a match id keeps its promise when a deployment has a planner key #397

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

On a deployment with a planner key, a match's bot plans are now saved with the match (the issue's option 2). The first time a round of a match is asked about, the server keeps its answer, and every later playing of that id in the same situation gets that plan back without asking the model again. `bun run test` passes (typecheck plus 1663 tests). The app boots and `/` answers in 22 ms.

shipped production is serving it deploy ae07c2d

opened this

On a deployment with a planner key, a match's bot plans are now saved with the match (the issue's option 2). The first time a round of a match is asked about, the server keeps its answer, and every later playing of that id in the same situation gets that plan back without asking the model again. bun run test passes (typecheck plus 1663 tests). The app boots and / answers in 22 ms.

Server

  • src/plan/served.ts (new) decides what a round gets. The key is the match id (checked with matchFrom) plus the round's situation (keyOf). If a plan is already kept under that key, it returns it. If not, it asks the planner, keeps whatever it got and returns it — including the local plan if the model timed out or the rate limit said no. So a round that was played to the local plan is replayed to it too.
  • Keyed by situation as well as round, because a person rarely plays a match the same way twice: round three at 2–0 and round three at 0–2 are different questions, each answered once per match.
  • src/db.ts has a new plan table and two functions, recordedPlan and recordPlan. The first answer saved is the one that stays, and a saved row is checked with parsePlan again when it is read back. It is created with CREATE TABLE IF NOT EXISTS, so older databases pick it up when they open; the existing old-database test now covers that.
  • Rate limits are only charged when a question actually goes to the model; reading a saved plan costs nothing.
  • src/plan/typesafe.ts gives the model 5 seconds in total, retries and Retry-After included, out of an 8-second freeze. After that the round gets the local plan. An answer that lands later is still cached in memory for the next match that asks the same thing.

Page

  • src/client/main.ts sends the match id with the situation and hands the answer to offerPlan instead of switching to it immediately.
  • offerPlan (in bots.ts, with a new world.planned field) holds the plan. updateWith in world.ts switches to it on the exact tick the freeze ends, so when the answer arrived never changes the round. A plan offered after that tick is dropped.
  • I also removed the planning guard in main.ts. A request still in flight from the previous match used to stop the next match from asking for its first round at all, which would break reproducibility too.
  • When a planner is configured, the Match id hint on / now says what an id keeps of the plan (PLANNED_ID). The planner paragraph and the README say the same.

Tests

  • served.test.ts: a stub planner that answers A, then B, gives one match id the same plan twice and is only asked once.
  • replay.test.ts: a new pinned run, PLANNED_ROUND, for a round a model planned. It shows:

- the same round whether the answer arrives at tick 1, 300 or 511 of the freeze; - an answer arriving at tick 513 or 900 leaves the round exactly as the local plan played it; - the same id served twice through the real database gives the same round, even though the two raw answers produce different rounds.

  • main.test.ts: a plan is held through the freeze and played from the tick it ends; a late answer is ignored.
  • server.test.ts: with no key, the endpoint answers exactly as before whatever match is sent.
  • Also covered: the saved plan survives reopening the database, and the planner's 5-second cutoff has its own test.
  • One existing test changed deliberately: the page's plan request now includes match, so the check on exactly which fields are sent was updated. It still checks that the playing key and anything about the person are not sent.

Not done

  • One remaining gap: if the first playing's answer reaches the browser after its freeze has ended (the server has already given up at 5 seconds, so this needs over 3 seconds of network delay), that playing ran the local plan while the saved record holds the model's plan. Later playings then get the model's plan. The page and README both say so.
  • "A pasted id plans locally" was rejected. It would never use the model for the matches people share, and a fresh match's link would replay as a different match.
  • Existing quirk left alone: bots buy for a new round before that round's plan is applied, so a plan's save decision only shows up in the next round's buying. The existing pinned runs are unchanged.

To try it: without a key the preview behaves exactly as before. With TYPESAFE_API_KEY set, play a match, copy its link and play it again: the plan line on the HUD says the same thing each round, and the plan table under FORGE_APP_DATA has one row per round.

🤖 Generated with Claude Code

Closes #238

commented

A match keeps the plans its rounds were given (#238). The first time a round of a match is asked about, the plan it gets is recorded against the match id and the round's situation (src/plan/served.ts, stored with bun:sqlite under FORGE_APP_DATA). Every later playing of that id reads the record and never asks again. A round waits at most five seconds for the planner's answer; after that it gets the local plan. That local plan is what gets recorded for the match, and the answer that lands late is only kept in memory for the next asker.

This branch now has main merged in, including the planner readout from #402. How the three conflicts were resolved:

  • src/plan/typesafe.ts: keeps main's counts (hits, joined, answered, failures by kind, requests sent, tokens) and this branch's five-second deadline. A joined request is also limited by the deadline. Failures are counted instead of logged, following main; the old warning line is gone. The success cache is now filled in the promise chain, so an answer that arrives after the deadline is still remembered.
  • src/server.ts: /api/plan goes through servePlan with the rate limit and spend ceiling, and also bumps unkeyed and readoutMoved() as main does. The limiters are only charged when a question actually goes upstream, so a round that is already recorded is neither charged nor counted as refused.
  • src/plan/typesafe.test.ts: both sides' tests are kept.

Two test stubs from this branch (src/plan/served.test.ts, src/client/replay.test.ts) were wrong after the merge: they lacked the new tally and model members of Planner. I added those members and changed no assertions.

Not done: a late answer is counted as answered when it lands, even though the round that asked got the local plan. The readout has no separate "late" count, so for those rounds it will say "answered" when the local plan was served.

To try it: bun test passes (1690 tests), and tsc --noEmit is clean. On a deployment with TYPESAFE_API_KEY set, play a match, open its id again and compare the rounds. Previews have no key, so they always plan locally.

🤖 Generated with Claude Code

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs