trevato/cs-engine

Public: anyone can read this; the crew can change it.

forked from trevato/cs

More
Forks3g fMapg aRunsg rEventsFlows

What a room costs, measured before anything is built on it: the world stepped headless in this process, at a ceiling a test holds #346

Merged torquecrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/cs-engine:torque/302-what-a-room-costs-measured-before-anythi → main 1 commit fork

  • Merged. torque/302-what-a-room-costs-measured-before-anythi was merged into main as 84de85b.

Review

Sign in to review this change.
5 files changed +831 −18
README.md +54
@@ -27,6 +27,8 @@ bun run scripts/surface.ts # is the door still the
2727 bun run scripts/surface.ts --write 0.2.0 # record today's door as that version
2828 bun run scripts/cost.ts # what each group behind the door adds to a bundle
2929 bun run scripts/cost.ts --guide # the same, as the lines GUIDE.md carries
30 bun run scripts/room.ts # what one room costs: CPU a world-second, snapshot bytes, rooms a core
31 bun run scripts/room.ts --seconds 30 --margin 0.25 # a longer window, and a quarter of a core for rooms
3032 bun test # the check every pull runs
3133 bun run scripts/mutate-result.ts # break each rule in src/result.ts and src/gate.ts, see a test go red
3234 FORGE_APP_DATA=./data bun run src/tty.ts # the terminal interface
@@ -1592,3 +1594,55 @@ What is real and what is not, today:
15921594 effects recorded for one screen and a snapshot goes to everybody); and a room's
15931595 result is not posted to the leaderboard. Rooms live in this process only — a
15941596 deploy ends them.
1597
1598 ### What a room costs, measured
1599
1600 The caps above are arithmetic on two numbers, and `bun run scripts/room.ts`
1601 measures both rather than asserting them: what a **world-second** of the
1602 simulation costs this process, and what a **snapshot** of that world weighs on
1603 a wire. It steps a real `createRoom` with ten bodies on de_forge, with the
1604 shipped policy thinking, and encodes the round that leaves. It excludes the
1605 socket: no connection is opened, so the kernel's share of the bytes is in no
1606 number here.
1607
1608 Measured on the forge's own box — an Intel Xeon Gold 6230 at 2.1 GHz, 80 shared
1609 cores at a load average of about 20, bun 1.3.14 — over an eight-second window of
1610 a live round, median of seven batches, across four runs. The spread between runs
1611 is the box and not the code, and the dearest second is one 64-tick window, so it
1612 carries most of that noise:
1613
1614 | per simulated world-second | average | dearest second |
1615 | --- | --- | --- |
1616 | ten bots thinking, the shipped policy | 5.4–8.1 ms | 10–24 ms |
1617 | ten seats driven by members, nothing thinking | 2.5–3.0 ms | 2.8–5.5 ms |
1618 | of which the input frames off the wire | 0.35–0.54 ms | |
1619
1620 So the ten policies are about **60–70% of a thinking room**: a room with ten
1621 people in every seat is less than half the CPU of a room full of bots. There is
1622 no third case — a bot without a policy does not think, it throws — so "without
1623 the trained policy" is measured as the room whose seats are all held.
1624
1625 The snapshot, over 128 beats of that round at 16 Hz:
1626
1627 | a snapshot of ten bodies | median | widest | 8 Hz | 16 Hz | 32 Hz |
1628 | --- | --- | --- | --- | --- | --- |
1629 | packed binary (what the wire sends) | 514 B | 764 B | 4.1 kB/s | 8.2 kB/s | 16.4 kB/s |
1630 | JSON | 3.8 kB | 5.1 kB | 30.7 kB/s | 61.4 kB/s | 122.7 kB/s |
1631 | JSON gzipped (level 6) | 684 B | 1.1 kB | 5.5 kB/s | 10.9 kB/s | 21.9 kB/s |
1632
1633 and what each costs this process to encode: 0.006 ms a beat packed, 0.019 ms
1634 JSON, 0.066 ms JSON gzipped — which for ten members at 16 Hz is 1.0 ms, 3.1 ms
1635 and 10.7 ms of CPU per world-second. Gzip buys 170 bytes a beat over the packed
1636 frame and costs ten times its CPU, which is the whole comparison in a sentence.
1637
1638 **The answer:** a whole room — the step plus ten packed snapshots at 16 Hz —
1639 costs **6.4–9.0 ms of CPU a second**, so one core holds **55–78 rooms** with
1640 half of it left over, and 19–45 if every room has its dearest second at the same
1641 moment. `ROOM_CAP` is 20, which is a sixth of a core: the cap is a choice about
1642 a box that also serves pages, not a ceiling the simulation put there. Raising it
1643 is a decision these numbers can now be read before.
1644
1645 `src/budget.test.ts` holds the ceiling that keeps it true: a room-second must
1646 cost under 40 ms in this process, and the same file proves the assertion can
1647 fail. The bench prints the machine it ran on because a number from a laptop is
1648 not a number from production.
scripts/room.test.ts added +135
@@ -0,0 +1,135 @@
1 /**
2 * The room bench, held to the two things a bench can be wrong about that a
3 * reader would never see: its **arithmetic** and its **subject**.
4 *
5 * The timings themselves are not asserted here and must not be — a number from
6 * a shared box is not a number from production, which is why `scripts/room.ts`
7 * prints the machine it ran on and why the suite's only assertion about speed
8 * is the ceiling in `src/budget.test.ts`. What is asserted is everything
9 * around them:
10 *
11 * - The flags are read or refused with a sentence, so nobody measures a window
12 * of `NaN` seconds and writes the answer in a pull.
13 * - Rooms per core is the division it claims to be, floored, and a margin
14 * narrows it.
15 * - The JSON candidate carries the **same round** the packed one does — ten
16 * bodies with their health and their guns, the grenades, the smoke, the bomb
17 * and the clock — because a comparison between a packed snapshot and a
18 * smaller idea of one would flatter the winner and settle #304 on a lie.
19 * - And the three encodings are measured over a real stepped room, packed
20 * smallest, JSON largest.
21 */
22
23 import { expect, test } from "bun:test";
24 import { GZIP_LEVEL } from "../src/compress.ts";
25 import { useShipped } from "../src/client/shipped.ts";
26 import { createRoom, stepRoom, SNAPSHOT_HZ } from "../src/rooms/room.ts";
27 import { SNAPSHOT_MAX } from "../src/rooms/wire.ts";
28 import {
29 DEFAULTS, RATES, encodings, jsonSnapshot, machine, parseOptions, perMemberPerSecond, roomsPerCore,
30 } from "./room.ts";
31 import type { Player } from "../src/client/world.ts";
32
33 // --- the flags ------------------------------------------------------------
34
35 test("the flags are read, and anything that is not a run is a sentence", () => {
36 expect(parseOptions([])).toEqual(DEFAULTS);
37 expect(parseOptions(["--seconds", "2.5"]).seconds).toBe(2.5);
38 expect(parseOptions(["--members", "4"]).members).toBe(4);
39 expect(parseOptions(["--margin", "0.25"]).margin).toBe(0.25);
40
41 expect(() => parseOptions(["--fast"])).toThrow(/not a flag this script knows/);
42 expect(() => parseOptions(["seconds", "2"])).toThrow(/is not a flag/);
43 expect(() => parseOptions(["--seconds"])).toThrow(/needs a value/);
44 expect(() => parseOptions(["--seconds", "--members"])).toThrow(/needs a value/);
45 expect(() => parseOptions(["--seconds", "soon"])).toThrow(/takes a number/);
46 expect(() => parseOptions(["--seconds", "0"])).toThrow(/above zero/);
47 expect(() => parseOptions(["--members", "11"])).toThrow(/1…10/);
48 expect(() => parseOptions(["--members", "2.5"])).toThrow(/1…10/);
49 expect(() => parseOptions(["--margin", "2"])).toThrow(/share of one core/);
50 expect(() => parseOptions(["--margin", "0"])).toThrow(/share of one core/);
51 });
52
53 // --- the arithmetic -------------------------------------------------------
54
55 test("rooms per core is the division it says it is, floored, and the margin narrows it", () => {
56 // A second of one core is 1000 ms of CPU: a room costing 10 ms of it a second
57 // is a hundredth of a core, so a hundred of them is the core and fifty is
58 // half of it.
59 expect(roomsPerCore(10, 1)).toBe(100);
60 expect(roomsPerCore(10, 0.5)).toBe(50);
61 // Half a room is no room.
62 expect(roomsPerCore(30, 1)).toBe(33);
63 // And a cost nobody measured is not an infinite number of rooms.
64 expect(roomsPerCore(0, 1)).toBe(0);
65 expect(roomsPerCore(-1, 1)).toBe(0);
66 });
67
68 test("bytes a member a second is the frame at the rate, and the rates include the shipped one", () => {
69 expect(perMemberPerSecond(544, 16)).toBe(8704);
70 expect(perMemberPerSecond(544, 32)).toBe(17408);
71 expect(RATES).toEqual([8, SNAPSHOT_HZ, 32]);
72 expect(RATES).toContain(SNAPSHOT_HZ);
73 });
74
75 test("the machine is named, because a number from a laptop is not a number from production", () => {
76 expect(machine()).toContain(`bun ${Bun.version}`);
77 expect(machine()).toContain(process.platform);
78 });
79
80 // --- the subject ----------------------------------------------------------
81
82 /** A room with a few seconds of a live round played into it. */
83 function played(seconds: number) {
84 useShipped();
85 const room = createRoom("TESTRM", { map: "de_forge", seed: 9, length: "short", now: 0 });
86 room.world.match.phase = "live";
87 room.world.match.timer = room.rules.roundTime;
88 for (let i = 0; i < 64 * seconds; i++) stepRoom(room);
89 return room;
90 }
91
92 test("the JSON candidate carries the same round the packed one does", () => {
93 const room = played(3);
94 room.world.bomb.state = "planted";
95 room.world.bomb.site = "A";
96 room.world.clouds.push({ pos: [500, 500, 40], born: room.world.time - 1, owner: 3 });
97 const snapshot = jsonSnapshot(room.world, room.tick, 0) as Record<string, unknown>;
98
99 expect(snapshot.t).toBe(room.tick);
100 expect(snapshot.phase).toBe("live");
101 const bomb = snapshot.bomb as Record<string, unknown>;
102 expect(bomb.state).toBe("planted");
103 expect(bomb.site).toBe("A");
104 expect((snapshot.clouds as unknown[]).length).toBe(1);
105
106 const bodies = snapshot.players as Record<string, unknown>[];
107 expect(bodies.length).toBe(room.world.players.length);
108 bodies.forEach((body, i) => {
109 const p = room.world.players[i] as Player;
110 // Every body, to the hundredth of a unit a JSON wire would round to.
111 expect((body.pos as number[])[0]).toBeCloseTo(p.pos[0], 1);
112 expect(body.health).toBe(Math.round(p.health));
113 expect(body.weapon).toBe(p.weapon);
114 expect(body.alive).toBe(p.alive);
115 expect(body.money).toBe(Math.round(p.money));
116 });
117 });
118
119 test("the three encodings are measured over a real round, and the packed one is the smallest", () => {
120 const room = played(3);
121 const frame = encodings(room.world, room.tick, 0, new Uint8Array(SNAPSHOT_MAX));
122
123 expect(frame.packed).toBeGreaterThan(0);
124 expect(frame.packed).toBeLessThan(frame.gzip);
125 expect(frame.gzip).toBeLessThan(frame.json);
126 // The packed frame is the one the wire sends, so it is the one pinned by
127 // `src/rooms/wire.test.ts`; this only holds the ordering the pull reports.
128 expect(frame.json).toBeGreaterThan(2000);
129
130 // And the gzip is really gzip at the level this server compresses at: a
131 // different level would be a different number in the table.
132 const text = JSON.stringify(jsonSnapshot(room.world, room.tick, 0));
133 expect(frame.gzip).toBe(Bun.gzipSync(Buffer.from(text, "utf8"), { level: GZIP_LEVEL }).byteLength);
134 expect(frame.json).toBe(Buffer.byteLength(text, "utf8"));
135 });
scripts/room.ts added +506
@@ -0,0 +1,506 @@
1 /**
2 * What one room costs, measured:
3 *
4 * bun run scripts/room.ts # the table
5 * bun run scripts/room.ts --seconds 30 # a longer window of play
6 * bun run scripts/room.ts --members 4 # snapshots for four people, not ten
7 * bun run scripts/room.ts --margin 0.25 # a quarter of a core for rooms
8 *
9 * `src/rooms/hub.ts` holds `ROOM_CAP` rooms in this process and the argument for
10 * where that cap is is arithmetic on two numbers: what a world-second of the
11 * simulation costs this process, and what a snapshot of that world weighs on a
12 * wire. This script measures both rather than asserting them, because every
13 * question multiplayer asks after it — how many rooms a deployment holds, how
14 * often a snapshot can go out, whether this process can be the truth at all —
15 * is those two numbers and nothing else.
16 *
17 * It measures a **real room**: `createRoom` from `src/rooms/room.ts`, ten
18 * bodies on de_forge, `stepRoom` at the game's own 64 Hz with the shipped
19 * policy thinking, and `writeSnapshot` on the round that leaves. Nothing here
20 * is a stub and nothing is a model of the game; the only thing it does that a
21 * deployment does not is skip the freeze at the top of the round, because eight
22 * seconds of ten bodies standing still is eight seconds of a tick that is not
23 * the tick anybody is asking about.
24 *
25 * **What it excludes**, said plainly because a bench that hides its edges is
26 * worse than none: the socket. No WebSocket is opened, nothing is written to a
27 * file descriptor, and the kernel's share of sending 8.7 kB a second to ten
28 * members is not in any number below. What is in them is everything this
29 * process computes — the simulation, the input frames off the wire, and the
30 * encoding of every snapshot.
31 *
32 * **It is not a ceiling.** `src/budget.test.ts` holds the ceiling and fails
33 * when a room-second gets dearer; this prints numbers for a person reading
34 * them, and the two are kept apart for the reason that file's header gives.
35 *
36 * It ships in no bundle and the server never imports it: this is a script a
37 * person runs and `room.test.ts` runs the pure parts of. It writes nothing,
38 * anywhere.
39 */
40
41 import { cpus } from "node:os";
42 import { GZIP_LEVEL } from "../src/compress.ts";
43 import { useShipped } from "../src/client/shipped.ts";
44 import { createRoom, drive, join, stepRoom, TICK, SNAPSHOT_HZ, type Room } from "../src/rooms/room.ts";
45 import { writeInput, writeSnapshot, SNAPSHOT_MAX } from "../src/rooms/wire.ts";
46 import { NO_INPUT, type Effect, type Player, type World } from "../src/client/world.ts";
47
48 // --- the options ----------------------------------------------------------
49
50 export type Options = {
51 /** How many simulated seconds one batch of the step is. */
52 seconds: number;
53 /** How many members' snapshots a full room is costed for. */
54 members: number;
55 /** What share of one core rooms are allowed, for the "how many" arithmetic. */
56 margin: number;
57 };
58
59 export const DEFAULTS: Options = { seconds: 10, members: 10, margin: 0.5 };
60
61 const KNOWN = ["--seconds", "--members", "--margin"];
62
63 export const USAGE = `usage: bun run scripts/room.ts [${KNOWN.join("] [")}]`;
64
65 /**
66 * argv (what comes after the script name) in, options out. Anything that is not
67 * a run — an unknown flag, a value that is not a number, a window of zero
68 * seconds, a margin outside 0…1 — throws a sentence naming the flag and what it
69 * got, because a table of numbers measured over `NaN` seconds is worse than an
70 * error.
71 */
72 export function parseOptions(argv: readonly string[]): Options {
73 const options: Options = { ...DEFAULTS };
74 for (let i = 0; i < argv.length; i++) {
75 const flag = argv[i] as string;
76 if (!flag.startsWith("--")) throw new Error(`${JSON.stringify(flag)} is not a flag; ${USAGE}`);
77 if (!KNOWN.includes(flag)) throw new Error(`${flag} is not a flag this script knows; ${USAGE}`);
78 const text = argv[++i];
79 if (text === undefined || text.startsWith("--")) throw new Error(`${flag} needs a value`);
80 const n = Number(text);
81 if (!Number.isFinite(n)) throw new Error(`${flag} takes a number, and got ${JSON.stringify(text)}`);
82 if (flag === "--seconds") {
83 if (n <= 0) throw new Error(`--seconds is a window of play, so it is above zero, and got ${n}`);
84 options.seconds = n;
85 } else if (flag === "--members") {
86 if (!Number.isInteger(n) || n < 1 || n > 10) throw new Error(`--members is 1…10, and got ${JSON.stringify(text)}`);
87 options.members = n;
88 } else {
89 if (n <= 0 || n > 1) throw new Error(`--margin is a share of one core, above 0 and at most 1, and got ${n}`);
90 options.margin = n;
91 }
92 }
93 return options;
94 }
95
96 // --- the arithmetic the numbers are read through --------------------------
97
98 /**
99 * How many rooms one core holds, given what a room-second of CPU costs and the
100 * share of that core rooms may have. A thousand milliseconds of CPU is a second
101 * of one core; a room that costs `each` milliseconds a simulated second takes
102 * that much of it, and the floor is taken because half a room is no room.
103 *
104 * The margin is not conservatism for its own sake: this process also serves
105 * pages, encodes snapshots for everybody, and runs a garbage collector, and a
106 * room that is late does not drop a frame — it judders for ten people.
107 */
108 export const roomsPerCore = (each: number, margin: number): number =>
109 each <= 0 ? 0 : Math.floor((1000 * margin) / each);
110
111 /** What one member is sent a second, in bytes, at a rate of `hz` snapshots. */
112 export const perMemberPerSecond = (bytes: number, hz: number): number => bytes * hz;
113
114 /** The rates this script costs every encoding at. */
115 export const RATES = [8, SNAPSHOT_HZ, 32] as const;
116
117 // --- the two clocks -------------------------------------------------------
118
119 /**
120 * Milliseconds of CPU this process has burned, user and system together.
121 *
122 * Every batch below is timed by this **and** by the wall clock, and the
123 * measurement is the smaller of the two: interference only ever pushes one of
124 * them up — a descheduled process reads a long wall clock, a GC thread on
125 * another core reads a long CPU clock — so the smaller is the tightest bound
126 * the pair honestly supports. It is the same argument `src/budget.test.ts`
127 * makes at length, and the fifteen lines are here rather than imported from
128 * there because a script that imports a test file to run is a worse trade than
129 * fifteen lines that say the same thing.
130 */
131 const cpuMs = (): number => {
132 const used = process.cpuUsage();
133 return (used.user + used.system) / 1000;
134 };
135
136 const median = (xs: number[]): number =>
137 [...xs].sort((a, b) => a - b)[(xs.length - 1) >> 1] as number;
138
139 /** Batches thrown away before any are kept, so nothing measured is cold. */
140 const WARM = 2;
141 /** Batches kept, of which the median is the answer. Odd, so there is one. */
142 const BATCHES = 7;
143
144 export type Measured = { label: string; each: number; wall: number; cpu: number };
145
146 /** One batch's worth of world-seconds, timed one at a time. */
147 const secondsOf = (ticks: number, step: () => void): number[] => {
148 const out: number[] = [];
149 for (let i = 0; i < ticks; i += 64) {
150 const wall = performance.now();
151 const cpu = cpuMs();
152 for (let j = 0; j < Math.min(64, ticks - i); j++) step();
153 out.push(Math.min(performance.now() - wall, cpuMs() - cpu));
154 }
155 return out;
156 };
157
158 /**
159 * The cost of a world-second across a round rather than averaged over one: a
160 * room's first second is ten bots walking out of spawn and its fifth is a
161 * firefight, and a deployment is sized by the dear second and not the cheap
162 * one. `make` hands back a fresh room per batch; `step` takes one tick of it.
163 */
164 export function profile(ticks: number, make: () => () => void): { mean: number; peak: number; at: number } {
165 const means: number[] = [];
166 const peaks: number[] = [];
167 const ats: number[] = [];
168 for (let b = 0; b < WARM + BATCHES; b++) {
169 const each = secondsOf(ticks, make());
170 if (b < WARM) continue;
171 means.push(each.reduce((a, c) => a + c, 0) / each.length);
172 const peak = Math.max(...each);
173 peaks.push(peak);
174 ats.push(each.indexOf(peak) + 1);
175 }
176 return { mean: median(means), peak: median(peaks), at: median(ats) };
177 }
178
179 /**
180 * `runs` of something, warmed and repeated, in milliseconds each. `batch` does
181 * `runs` operations and is called once per batch; the answer is the median of
182 * `BATCHES` kept batches on the tighter of the two clocks.
183 */
184 function measure(label: string, runs: number, batch: () => void): Measured {
185 const walls: number[] = [];
186 const cpus_: number[] = [];
187 for (let i = 0; i < WARM + BATCHES; i++) {
188 const wall = performance.now();
189 const cpu = cpuMs();
190 batch();
191 if (i < WARM) continue;
192 walls.push((performance.now() - wall) / runs);
193 cpus_.push((cpuMs() - cpu) / runs);
194 }
195 const wall = median(walls);
196 const cpu = median(cpus_);
197 return { label, each: Math.min(wall, cpu), wall, cpu };
198 }
199
200 // --- the room being measured ----------------------------------------------
201
202 /** How many bodies a room has. Ten, like the game. */
203 const BODIES = 10;
204
205 /**
206 * A room with its round already under way: the same fixed map and seed every
207 * time, and the freeze set aside for the reason in the header. Fresh per batch,
208 * because a world played for thousands of ticks is a world where people have
209 * died and a dead body does not think — the measurement would drift downwards
210 * into a number about nothing.
211 */
212 function liveRoom(seed = 9): Room {
213 const room = createRoom("BENCH1", { map: "de_forge", seed, length: "short", now: 0 });
214 room.world.match.phase = "live";
215 room.world.match.timer = room.rules.roundTime;
216 return room;
217 }
218
219 /** The same room with every seat held by a member, so no body is left to think. */
220 function heldRoom(seed = 9): { room: Room; ids: string[] } {
221 const room = liveRoom(seed);
222 const ids: string[] = [];
223 for (let i = 0; i < BODIES; i++) {
224 const id = `member-${i}`;
225 if (join(room, id, id, 0) !== null) ids.push(id);
226 }
227 return { room, ids };
228 }
229
230 /** One input frame per member per tick, the way a page sends them: keys down. */
231 function driveAll(room: Room, ids: readonly string[], frames: Uint8Array<ArrayBuffer>[]): void {
232 ids.forEach((id, i) => {
233 writeInput(
234 { tick: room.tick, input: { ...NO_INPUT, forward: 1 }, yaw: 90, pitch: 0, switchCode: 0, buy: null },
235 frames[i] as Uint8Array<ArrayBuffer>,
236 );
237 drive(room, id, frames[i] as Uint8Array);
238 });
239 }
240
241 // --- the three encodings a wire could use ---------------------------------
242
243 /**
244 * The same snapshot as plain data: every field `writeSnapshot` packs, in the
245 * shape a JSON wire would send it. It exists so the comparison below is between
246 * two encodings of *one* frame rather than between a packed snapshot and a
247 * smaller idea of one — the ten bodies, the grenades in the air, the smoke
248 * standing, the bomb, the clock, and the events since the member's last beat.
249 *
250 * Nothing chooses an encoding here. This measures three candidates; the choice
251 * and its reasons are `src/rooms/wire.ts`'s, and a reader comparing the numbers
252 * should read that file's header for what the packed one buys.
253 *
254 * Numbers are rounded the way an honest JSON wire would round them — positions
255 * to a hundredth of a unit, angles to a tenth of a degree — because
256 * `JSON.stringify` of an f32 writes seventeen significant digits and charging
257 * JSON for those would be measuring a strawman.
258 */
259 export function jsonSnapshot(world: World, tick: number, since: number): unknown {
260 const r2 = (v: number): number => Math.round(v * 100) / 100;
261 const r1 = (v: number): number => Math.round(v * 10) / 10;
262 const m = world.match;
263 return {
264 t: tick,
265 time: r2(world.time),
266 phase: m.phase,
267 round: m.round,
268 score: [m.score.t, m.score.ct],
269 timer: r2(m.timer),
270 winner: m.matchWinner,
271 bomb: {
272 state: world.bomb.state,
273 pos: world.bomb.pos.map(r2),
274 plant: r2(world.bomb.plantProgress),
275 defuse: r2(world.bomb.defuseProgress),
276 site: world.bomb.site,
277 },
278 players: world.players.map((p: Player) => ({
279 pos: p.pos.map(r2),
280 vel: p.vel.map(r2),
281 yaw: r1(p.yaw),
282 pitch: r1(p.pitch),
283 health: Math.round(p.health),
284 armor: Math.round(p.armor),
285 alive: p.alive,
286 team: p.team,
287 onGround: p.onGround,
288 ducking: p.ducking,
289 duck: r2(p.duckAmount),
290 bomb: p.hasBomb,
291 helmet: p.helmet,
292 kit: p.defuseKit,
293 scoped: p.scoped,
294 weapon: p.weapon,
295 primary: p.primary,
296 secondary: p.secondary,
297 mag: p.ammo[p.weapon]?.mag ?? 0,
298 reserve: p.ammo[p.weapon]?.reserve ?? 0,
299 money: Math.round(p.money),
300 blind: r2(p.blind),
301 pocket: [p.pocket.he, p.pocket.flash, p.pocket.smoke],
302 kills: p.kills,
303 deaths: p.deaths,
304 damage: Math.round(p.damage),
305 })),
306 grenades: world.grenades.map((g) => ({
307 pos: g.pos.map(r2), weapon: g.weapon, owner: g.owner, team: g.team, fuse: r2(g.fuse),
308 })),
309 clouds: world.clouds.map((s) => ({ pos: s.pos.map(r2), age: r2(world.time - s.born), owner: s.owner })),
310 events: eventsSince(world, since).map((e) => ({ ...e, at: r2(e.at) })),
311 };
312 }
313
314 /** Everything in the world's effect ring younger than `since`, oldest first. */
315 function eventsSince(world: World, since: number): Effect[] {
316 const out: Effect[] = [];
317 const ring = world.effects;
318 for (let i = 0; i < ring.count; i++) {
319 const e = ring.events[i];
320 if (e !== undefined && e.at > since) out.push(e);
321 }
322 return out;
323 }
324
325 export type Frame = { packed: number; json: number; gzip: number };
326
327 /** What one beat weighs in each of the three encodings, in bytes. */
328 export function encodings(world: World, tick: number, since: number, into: Uint8Array): Frame {
329 const packed = writeSnapshot(world, tick, since, into).byteLength;
330 const text = JSON.stringify(jsonSnapshot(world, tick, since));
331 const bytes = Buffer.byteLength(text, "utf8");
332 return { packed, json: bytes, gzip: Bun.gzipSync(Buffer.from(text, "utf8"), { level: GZIP_LEVEL }).byteLength };
333 }
334
335 // --- printing -------------------------------------------------------------
336
337 const ms = (v: number): string => `${v.toFixed(v < 1 ? 3 : 2)} ms`;
338 const kb = (v: number): string => (v < 1000 ? `${Math.round(v)} B` : `${(v / 1000).toFixed(1)} kB`);
339 const pad = (text: string, width: number): string => text.padEnd(width);
340 const rpad = (text: string, width: number): string => text.padStart(width);
341
342 /** Which machine this ran on, because a number from a laptop is not production. */
343 export const machine = (): string =>
344 `${cpus()[0]?.model.trim() ?? "unknown CPU"} × ${cpus().length}` +
345 `, bun ${Bun.version}, ${process.platform}/${process.arch}`;
346
347 async function main(options: Options): Promise<void> {
348 useShipped(); // a bot with no policy does not think, and would not be measured
349 const ticks = Math.round(options.seconds * 64);
350 const seconds = ticks * TICK;
351 const buffer = new Uint8Array(SNAPSHOT_MAX);
352
353 console.log(`what a room costs — ten bodies on de_forge at 64 Hz, ${seconds.toFixed(2)} simulated seconds a batch`);
354 console.log(` ${machine()}`);
355 console.log(` median of ${BATCHES} batches after ${WARM}, on the tighter of the wall and CPU clocks\n`);
356
357 // --- the step ---
358 // The rooms of every batch are kept so the round each measurement really
359 // played can be asked afterwards how many bodies were still standing in it.
360 const played: Room[] = [];
361 const thinking = profile(ticks, () => {
362 const room = liveRoom();
363 played.push(room);
364 return () => stepRoom(room);
365 });
366 const aliveAfter = (played[played.length - 1] as Room).world.players.filter((p) => p.alive).length;
367
368 // The other half of the pair, and the only way this game has of stepping a
369 // world with no policy in the loop: a bot without one does not think, it
370 // throws (`policy()` in `src/client/bots.ts`), so "the policy unloaded" is a
371 // room whose every seat is held by a member sending keys.
372 const frames = Array.from({ length: BODIES }, () => new Uint8Array(14));
373 const idle = profile(ticks, () => {
374 const { room, ids } = heldRoom();
375 return () => {
376 driveAll(room, ids, frames);
377 stepRoom(room);
378 };
379 });
380
381 // What the wire itself costs inside that number: ten input frames written and
382 // read every tick, which the thinking room above does not pay at all.
383 const wire = measure(" of which the input frames off the wire", seconds, () => {
384 const { room, ids } = heldRoom();
385 for (let i = 0; i < ticks; i++) driveAll(room, ids, frames);
386 });
387
388 console.log("CPU per simulated world-second");
389 console.log(` ${pad("", 40)}${rpad("average", 10)}${rpad("dearest second", 18)}`);
390 const step = (label: string, m: { mean: number; peak: number; at: number }): void =>
391 console.log(` ${pad(label, 40)}${rpad(ms(m.mean), 10)}${rpad(`${ms(m.peak)} (no. ${m.at})`, 18)}`);
392 step("ten bots thinking, the shipped policy", thinking);
393 step("ten seats driven, nothing thinking", idle);
394 console.log(` ${pad(wire.label, 40)}${rpad(ms(wire.each), 10)}`);
395 // Ten bodies are stepped either way; what the first row has and the second
396 // does not is ten policies thinking, and what the second has and the first
397 // does not is the wire. So the policy's share is the difference, with the
398 // wire handed back — which is an estimate and is called one: bots that think
399 // also shoot, throw and die, and a body walking forward into a wall does not.
400 const share = thinking.mean - (idle.mean - wire.each);
401 console.log(
402 ` ${pad("so the ten policies cost about", 40)}${rpad(ms(share), 10)}` +
403 ` (${((share / thinking.mean) * 100).toFixed(0)}% of a thinking room)`,
404 );
405 console.log(
406 ` ${aliveAfter} of ${BODIES} bodies were still alive after ${seconds.toFixed(1)}s` +
407 ` of the thinking round, which is what the numbers above are seconds of\n`,
408 );
409
410 // --- the snapshot, over a real round ---
411 const beats: Frame[] = [];
412 {
413 const room = liveRoom();
414 const every = Math.max(1, Math.round(64 / SNAPSHOT_HZ));
415 let since = 0;
416 for (let i = 0; i < ticks; i++) {
417 stepRoom(room);
418 if (i % every !== 0) continue;
419 beats.push(encodings(room.world, room.tick, since, buffer));
420 since = room.world.time;
421 }
422 }
423 const pick = (of: (f: Frame) => number): { med: number; max: number } => ({
424 med: median(beats.map(of)),
425 max: Math.max(...beats.map(of)),
426 });
427 const sizes = {
428 "packed binary": pick((f) => f.packed),
429 "JSON": pick((f) => f.json),
430 [`JSON gzipped (level ${GZIP_LEVEL})`]: pick((f) => f.gzip),
431 };
432
433 console.log(`a snapshot of this round, over ${beats.length} beats at ${SNAPSHOT_HZ} Hz`);
434 for (const [name, size] of Object.entries(sizes))
435 console.log(` ${pad(name, 30)} median ${rpad(kb(size.med), 8)} widest ${rpad(kb(size.max), 8)}`);
436 console.log();
437
438 console.log(`bytes a member a second, at the median beat`);
439 console.log(` ${pad("", 30)}${RATES.map((hz) => rpad(`${hz} Hz`, 12)).join("")}`);
440 for (const [name, size] of Object.entries(sizes))
441 console.log(
442 ` ${pad(name, 30)}` +
443 RATES.map((hz) => rpad(`${kb(perMemberPerSecond(size.med, hz))}/s`, 12)).join(""),
444 );
445 console.log();
446
447 // --- what encoding a beat costs this process ---
448 const busy = liveRoom();
449 for (let i = 0; i < ticks; i++) stepRoom(busy);
450 // One beat's worth of "since", not the whole ring: what a member is sent
451 // sixteen times a second is the round plus what happened in the last
452 // sixteenth of it, and costing them the whole ring would be costing a frame
453 // nobody is ever sent.
454 const since = busy.world.time - 1 / SNAPSHOT_HZ;
455 const runs = 200;
456 const encode = [
457 measure("packed binary", runs, () => {
458 for (let i = 0; i < runs; i++) writeSnapshot(busy.world, busy.tick, since, buffer);
459 }),
460 measure("JSON", runs, () => {
461 for (let i = 0; i < runs; i++) JSON.stringify(jsonSnapshot(busy.world, busy.tick, since));
462 }),
463 measure(`JSON gzipped (level ${GZIP_LEVEL})`, runs, () => {
464 for (let i = 0; i < runs; i++)
465 Bun.gzipSync(Buffer.from(JSON.stringify(jsonSnapshot(busy.world, busy.tick, since)), "utf8"), { level: GZIP_LEVEL });
466 }),
467 ];
468 console.log("CPU to encode one beat, and what that is a world-second for a full room");
469 for (const m of encode)
470 console.log(
471 ` ${pad(m.label, 30)} ${rpad(ms(m.each), 9)} a beat` +
472 ` ${rpad(ms(m.each * SNAPSHOT_HZ * options.members), 9)} a world-second` +
473 ` (${options.members} members at ${SNAPSHOT_HZ} Hz)`,
474 );
475 console.log();
476
477 // --- the answer ---
478 const packedCost = (encode[0] as Measured).each * SNAPSHOT_HZ * options.members;
479 const whole = thinking.mean + packedCost;
480 const dearest = thinking.peak + packedCost;
481 const row = (label: string, each: number): void =>
482 console.log(
483 ` ${pad(label, 46)} ${rpad(String(roomsPerCore(each, options.margin)), 4)} rooms` +
484 ` (${roomsPerCore(each, 1)} flat out, at ${ms(each)} a room-second)`,
485 );
486 console.log(`rooms one core holds at 64 Hz, with ${((1 - options.margin) * 100).toFixed(0)}% of it left for everything else`);
487 row("the step alone, bots thinking", thinking.mean);
488 row("the step alone, nobody thinking", idle.mean);
489 row(`a whole room: step and ${options.members} packed snapshots`, whole);
490 row("the same, priced at the round's dearest second", dearest);
491 console.log(
492 `\none room costs ${ms(whole)} of CPU a second — the step ${ms(thinking.mean)}` +
493 ` and ${options.members} snapshots ${ms(packedCost)} — so one core holds ${roomsPerCore(whole, options.margin)}` +
494 ` of them at ${SNAPSHOT_HZ} Hz with ${((1 - options.margin) * 100).toFixed(0)}% of the core to spare,` +
495 ` or ${roomsPerCore(dearest, options.margin)} if every one of them is having its dearest second at once.`,
496 );
497 }
498
499 if (import.meta.main) {
500 try {
501 await main(parseOptions(process.argv.slice(2)));
502 } catch (error) {
503 console.error(error instanceof Error ? error.message : String(error));
504 process.exit(1);
505 }
506 }
src/budget.test.ts +119 −6
@@ -8,16 +8,23 @@
88 * for the same reason: both claims are about the two games at once, and neither
99 * game may import the other.
1010 *
11 * **This is not a bench.** #20 is a script a person runs to read numbers. This
12 * is a ceiling the suite enforces on every push, and nothing here is meant to
13 * be read as "how fast is it": for that, run the bench. Merging the two would
14 * lose the only thing this file is for — failing.
11 * **This is not a bench.** #20 is a script a person runs to read numbers, and
12 * `bun run scripts/room.ts` is the one for the room below. This is a ceiling
13 * the suite enforces on every push, and nothing here is meant to be read as
14 * "how fast is it": for that, run the bench. Merging the two would lose the
15 * only thing this file is for — failing.
1516 *
1617 * ## What is measured
1718 *
1819 * - One tick of the shooter at 64 Hz on a live round: ten bodies on de_forge,
1920 * nine of them bots running the shipped policy, plus the grenades, the bomb
2021 * and the rules clock. This is the whole simulation a playing frame steps.
22 * - **A world-second of a room**, which is the same simulation stepped by this
23 * process instead of by a browser: `createRoom` and sixty-four `stepRoom`s,
24 * ten bots thinking, which is what a deployment pays per room per second of
25 * play. A frame's budget is a person's patience; a room-second's budget is a
26 * core, and `ROOM_CAP` in `src/rooms/hub.ts` is one divided by the other, so
27 * this is the number that decides how many rooms one box holds.
2128 * - `buildScene` on the world that tick left, with a firefight's worth of
2229 * effects in the ring — bodies, the bomb, tracers, sparks and blood — which
2330 * is a hungry frame rather than a quiet one.
@@ -42,6 +49,20 @@
4249 * 0.022 ms and 0.0010 ms — still under their ceilings by four, seven and twenty
4350 * times over.
4451 *
52 * The room-second is the same argument at a different scale, and it is worth
53 * saying exactly which second it is: the **first** of a live round, which is
54 * ten bots walking out of spawn, and which reads about **4.6 ms** here. A round
55 * gets dearer as it meets itself — `bun run scripts/room.ts` measures an
56 * eight-second window on the forge's box, eighty cores at a load average of
57 * twenty, and reads 5.4–8.1 ms an average second with 10–24 ms in the dearest
58 * one — so the ceiling is set against the bench's numbers and not against this
59 * test's own. It is **40 ms**: roughly five times an average second of play,
60 * four per cent of the second a room has to be stepped inside, and still above
61 * the dearest second this box has produced. What it catches is a change that
62 * makes the simulation *several* times dearer, not one that makes it a fifth
63 * dearer — a tighter ceiling on a shared box is a red build about somebody
64 * else's job, which nobody would read twice.
65 *
4566 * The multiple is deliberately larger where the number is smaller. Clock
4667 * resolution and scheduler jitter are absolute, not proportional: ten times
4768 * 0.3 µs is inside the noise of any machine, so the range's ceiling is sixty
@@ -83,6 +104,7 @@ import { useShipped } from "./client/shipped.ts";
83104 import { NO_INPUT, createWorld, update, type Effect, type World } from "./client/world.ts";
84105 import { buildRangeScene, createRangeScene } from "./range/scene.ts";
85106 import { EYE, createRun, start, step } from "./range/targets.ts";
107 import { createRoom, stepRoom, type Room } from "./rooms/room.ts";
86108
87109 /** The fixed simulation step, and the wall clock a frame of it has to fit in. */
88110 const TICK = 1 / 64;
@@ -94,6 +116,13 @@ const TICK_CEILING = 1.0;
94116 const SCENE_CEILING = 0.15;
95117 /** The range's scene. Today: 0.0003 ms — see the note on small numbers above. */
96118 const RANGE_SCENE_CEILING = 0.02;
119 /**
120 * One second of play in one room, stepped here. Today: 4.6 ms for the second
121 * this test takes, 5.4–8.1 ms for an average one on the forge's box under load
122 * — see the note above, and `bun run scripts/room.ts` for where the numbers
123 * come from and what a deployment does with them.
124 */
125 const ROOM_SECOND_CEILING = 40;
97126
98127 /** Batches thrown away before any are kept, so nothing measured is cold. */
99128 const WARM = 2;
@@ -131,13 +160,21 @@ function timed(runs: number, work: () => void): Batch {
131160 */
132161 const PATIENCE = 60_000;
133162
163 /**
164 * What a ceiling here is a slice of. A frame has 15.625 ms of a person's
165 * patience; a room-second has a second of a core, which is the same sentence
166 * about a different clock, and the printed line says which one it meant.
167 */
168 const FRAME = { ms: FRAME_BUDGET, name: "frame" };
169 const ROOM_SECOND = { ms: 1000, name: "second a room has" };
170
134171 /**
135172 * Run `batch` — which times some number of operations and hands back the
136173 * milliseconds one of them cost — until it is warm, then seven more times, and
137174 * return the median. The number is printed, because a green run should still
138175 * tell a person what a frame costs today.
139176 */
140 function measure(label: string, ceiling: number, batch: () => Batch): number {
177 function measure(label: string, ceiling: number, batch: () => Batch, within = FRAME): number {
141178 for (let i = 0; i < WARM; i++) batch();
142179 const kept: Batch[] = [];
143180 for (let i = 0; i < BATCHES; i++) kept.push(batch());
@@ -147,7 +184,7 @@ function measure(label: string, ceiling: number, batch: () => Batch): number {
147184 console.log(
148185 ` ${label}: ${cost.toFixed(4)} ms` +
149186 ` (median of ${BATCHES}: wall ${wall.toFixed(4)}, cpu ${cpu.toFixed(4)};` +
150 ` ceiling ${ceiling} ms of a ${FRAME_BUDGET.toFixed(3)} ms frame)`,
187 ` ceiling ${ceiling} ms of a ${within.ms.toFixed(3)} ms ${within.name})`,
151188 );
152189 return cost;
153190 }
@@ -225,6 +262,82 @@ test("a tick with ten clouds of smoke standing is still well inside the budget",
225262 expect(clouds).toBe(10);
226263 }, PATIENCE);
227264
265 /**
266 * A room's own round, whose freeze is set aside the way `liveRound` sets it
267 * aside and for the same reason. Every one of the ten bodies is a bot here —
268 * `createRoom` makes `world.me` one too, because a room has seats and not a
269 * person — so this is ten policies thinking and not nine.
270 */
271 function liveRoom(seed: number): Room {
272 const room = createRoom("BUDGET", { map: "de_forge", seed, length: "short", now: 0 });
273 room.world.match.phase = "live";
274 room.world.match.timer = room.rules.roundTime;
275 return room;
276 }
277
278 /**
279 * What this process pays to *be* one room for one second of play. It is the
280 * number `ROOM_CAP` is divided out of, so the day the simulation gets several
281 * times dearer this is where it is read — before twenty rooms full of people
282 * find out for themselves.
283 *
284 * The snapshots are not in here: what they weigh is pinned in
285 * `src/rooms/wire.test.ts` and what they cost is printed by
286 * `bun run scripts/room.ts`, and this holds the one number that is the room
287 * whether anybody is watching it or not.
288 */
289 test("a second of play in one room costs a small fraction of the second it had", () => {
290 useShipped(); // a bot with no policy does not think, and would not be measured
291 let alive = 0;
292 const each = measure(
293 "a room-second: 64 steps of ten bodies, all ten thinking",
294 ROOM_SECOND_CEILING,
295 () => {
296 // A fresh round per batch, for the reason the tick above takes one.
297 const room = liveRoom(7);
298 const batch = timed(1, () => {
299 for (let i = 0; i < TICKS; i++) stepRoom(room);
300 });
301 alive = room.world.players.filter((p) => p.alive).length;
302 return batch;
303 },
304 ROOM_SECOND,
305 );
306
307 expect(each).toBeLessThan(ROOM_SECOND_CEILING);
308 // And the second that was measured really was ten bodies thinking: a room
309 // that had emptied out would be a cheap number about nothing.
310 expect(alive).toBe(10);
311 }, PATIENCE);
312
313 /**
314 * And the room's ceiling bites too, for the reason the frame's does. Sixteen
315 * seconds of play charged as one is a simulation sixteen times dearer — nothing
316 * about the room changed, it is simply asked for sixteen times the work inside
317 * the second it had — and the measurement notices and the ceiling refuses it.
318 * Sixteen and not two, for the same reason the frame's proof is written at
319 * sixty-four substeps and not at ten: the margin between the ceiling and
320 * today's cost is what keeps a busy box from turning this file red, and a proof
321 * has to clear that margin on a machine slower than this one as well.
322 */
323 test("a room-second that got several times dearer is past the ceiling", () => {
324 useShipped();
325 const dearer = 16;
326 const each = measure(
327 `a room-second asked to step ${dearer} of them, to prove the ceiling bites`,
328 ROOM_SECOND_CEILING,
329 () => {
330 const room = liveRoom(7);
331 return timed(1, () => {
332 for (let i = 0; i < TICKS * dearer; i++) stepRoom(room);
333 });
334 },
335 ROOM_SECOND,
336 );
337
338 expect(each).toBeGreaterThan(ROOM_SECOND_CEILING);
339 }, PATIENCE);
340
228341 /**
229342 * The guard's own guard. A ceiling nothing can reach is a green light wired to
230343 * nothing: a loop the JIT lifted out, a clock read twice in a row, a batch that
src/rooms/hub.ts +17 −12
@@ -1,13 +1,18 @@
11 /**
22 * Every room this process is holding, and the one clock that steps them.
33 *
4 * A room is CPU on a shared box, so this file is mostly numbers with reasons.
5 * One room is one world stepped 64 times a second: measured by
6 * `src/budget.test.ts` a tick of ten bodies with nine of them thinking costs
7 * about 0.08 ms, so a room costs about **5 ms of CPU a second** — half a per
8 * cent of one core — and `ROOM_CAP` rooms at once is a tenth of a core plus the
9 * snapshots. That is the whole argument for the cap being where it is, and it is
10 * the number to revisit before raising it.
4 * A room is CPU on a shared box, so this file is mostly numbers with reasons,
5 * and the numbers are measured rather than assumed: `bun run scripts/room.ts`
6 * steps a real room and prints them. On the forge's own box a room-second of
7 * ten bots thinking costs **5.4–8.1 ms** — 10–24 ms in the round's dearest
8 * second — and ten packed snapshots a second on top of it about **1 ms**, so a
9 * whole room is about **6.4–9.0 ms of CPU a second**, under one per cent of a
10 * core. `ROOM_CAP` rooms at once is therefore about a sixth of a core, and one
11 * core would hold fifty to eighty of them. That is the whole argument for the
12 * cap being where it is: it is a choice about a box that also serves pages, not
13 * a ceiling the simulation put there. `src/budget.test.ts` holds a room-second
14 * under 40 ms so the day the simulation gets several times dearer this file's
15 * arithmetic goes red rather than silently wrong.
1116 *
1217 * Nothing here is a socket either. The hub is handed a `Wire` — deliver these
1318 * bytes to that member, close that member — and the server is the only file that
@@ -29,11 +34,11 @@ import {
2934 import type { PlayedMap } from "../mapname.ts";
3035
3136 /**
32 * How many rooms one deployment holds at once. Twenty rooms is about 0.1 s of
33 * CPU a second — a tenth of a core — against a box that also serves pages, and
34 * two hundred people in them at ten a room. Past it a create is refused with a
35 * sentence, which is the honest answer: a twenty-first room would make the
36 * other twenty judder.
37 * How many rooms one deployment holds at once. Twenty rooms is a measured
38 * 0.13–0.18 s of CPU a second — about a sixth of a core — against a box that
39 * also serves pages, and two hundred people in them at ten a room. Past it a
40 * create is refused with a sentence, which is the honest answer: a twenty-first
41 * room would make the other twenty judder.
3742 */
3843 export const ROOM_CAP = 20;
3944

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs