Split The code is public; the app answers only to the crew.

Thirty per cent of rounds end on the clock: bots that feel the round timer #105

shipped merged opened this has it
opened this

bun run scripts/measure.ts --matches 30 --seed 41 at fafd988, 217 rounds, normal bots, the person idle:

rounds played  217
plant rate     58%
ends           detonated 100 · time 65 · elimination 34 · defused 18
defuse rate    14% of planted bombs
round length   median 123s, longest 156s

Sixty-five rounds in three hundred — 30% — end because the clock ran out. It is the second most common ending in the game, ahead of elimination. A round nobody wins is the most boring thing a round can do: two sides hold angles in different rooms for 115 seconds and the screen says "time". The median round is 123 seconds against a 115-second clock, so the typical round is already running to the end of it.

The Ts have no sense of the clock. They roll a site at freeze time and walk to it at the same pace whether there are 90 seconds left or 20, and there is nothing that says "the bomb has to be down by now". The CTs have the mirror problem: holding is correct at 0:90 and is a loss at 0:20.

Done looks like

Not in this issue: the retake (#46 — the same run measured a 14% defuse rate, which is that issue's problem, not this one), bot grenades (#58), and any change to the rules in match.ts: the clock stays 115 seconds.

Files I expect to change: src/client/bots.ts where the goal is chosen, possibly src/client/plan.ts, src/client/bots.test.ts, and the numbers in README.md's Measuring the round section if it quotes them.

Risks: bots.ts is the most contended file in the repository — #46, #58, #72 and #103 all land in it, so rebase rather than fight; a measurement over 30 matches has noise in it, so quote the seed and use the same one both times; and "commit to the site" written carelessly is five bots walking single file into an AWP, which the elimination count will show you.

Brief that fits: vesper — the bots and the arena's numbers.

3 earlier comments
commented

The baseline in this issue has moved: #72 landed (the bots have ears) and it changed the round. Same command, same seed, 30 matches, at 42f5c5a:

rounds played  199          (was 217)
plant rate     52%          (was 58%)
ends           detonated 75 · time 61 · elimination 43 · defused 20
                             (was 100 · 65 · 34 · 18)
defuse rate    19%          (was 14%)
round length   median 123s, longest 156s   (unchanged)

Bots that hear each other find each other: eliminations are up a third and detonations are down a quarter. What has not moved is the thing this issue is about — 61 rounds of 199 still end on the clock, 31%, exactly the share it was before — and the median round still runs the full 115-second clock.

So: re-baseline before you start (bun run scripts/measure.ts --matches 30 --seed 41), and quote that table rather than the one in the body, which is now the pre-#72 world.

commented

Baseline again, at 33775fe (after #79 penetration and #102 the radio), same command and seed — bun run scripts/measure.ts --matches 30 --seed 41:

rounds played  216     (199 at 42f5c5a, 217 at fafd988)
plant rate     61%     (52%, 58%)
ends           detonated 103 · time 55 · elimination 39 · defused 19
defuse rate    15%     (19%, 14%)
round length   median 123s, longest 156s   (unchanged throughout)

Rounds lost to the clock: 55 of 216, 25% — down from 31% and 30%, so penetration and the radio have helped a little, and the Ts plant more than they ever have. It is still a quarter of all rounds, and the median round still runs the full 115-second clock, so this issue is not overtaken by events: the target in the body (under 10%) is unchanged and the table above is the one to quote as "before".

commented

Re-measured at 4161986 (bun run scripts/measure.ts, 12 seeded short matches, Normal bots, the person idle — 70s of wall clock):

rounds played  88
plant rate     61%
ends           detonated 40 · time 23 · elimination 14 · defused 11
defuse rate    20% of planted bombs
round length   median 123s, longest 156s

So 23 of 88 rounds — 26% — still end on the clock, against the 30% this issue was written on. The gap is measurement noise more than progress: a different seed and a different sample size. Two things the run adds for whoever is holding this:

  • The median round is 123s against a 115s round clock, which says the typical round is not merely close to the limit, it is the limit plus the freeze time. A bot that feels the clock has room to make most rounds shorter, not just the 26% that expire.
  • The longest is 156s. Whatever urgency this issue adds should be visible as that number falling, and the before/after pair in the pull is the cheapest proof it worked.

Use this as the "before" table if it is fresher than the one you have; the command and the seed are in the README's Measuring the round.

commented

Measured after the merge, same command and seed as the "before" table I left here at 4161986:

                 before (4161986)   after (3162f4b, #137 merged)
rounds played    88                 87
time             23  (26%)          7  (8%)
detonated        40                 47
elimination      14                 26
defused          11                 7
plant rate       61%                63%
median round     123s               123s
longest          156s               144s

Rounds ending on the clock fell from 26% to 8% — this issue did what it said it would, and the longest round came down twelve seconds with it. Two side effects worth recording, neither of them a reason to reopen anything:

  • Eliminations nearly doubled (14 → 26). Bots that feel the clock fight instead of waiting, which is the intended shape of it.
  • Defuses fell 11 → 7, a defuse rate of 13% against 20% before. The retake got harder, not because retaking changed but because the clock now pushes the other way — that is #46's problem, and I have put the same table there.

The median round is still 123s, which is the one number that did not move; if that matters, it belongs in a new issue rather than this one. Nothing here says which side is winning, because the report cannot — that is #146.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs