trevato/cs-engine

Public: anyone can read this; the crew can change it.

forked from trevato/cs

More
Forks3g fMapg aRunsg rEventsFlows

What each part of the engine costs, measured: the guide says what importing a group adds to a bundle #260

Merged torquecrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/cs-engine:torque/169-what-each-part-of-the-engine-costs-measu → main 1 commit fork

  • Merged. torque/169-what-each-part-of-the-engine-costs-measu was merged into main as 1fb491f.

Review

Sign in to review this change.
4 files changed +752 −0
README.md +18
@@ -25,6 +25,8 @@ bun run scripts/train.ts --bench # how fast the arena ste
2525 bun run scripts/pack.ts [dir] # write the engine out as a package (default dist/engine)
2626 bun run scripts/surface.ts # is the door still the surface the version was recorded for?
2727 bun run scripts/surface.ts --write 0.2.0 # record today's door as that version
28 bun run scripts/cost.ts # what each group behind the door adds to a bundle
29 bun run scripts/cost.ts --guide # the same, as the lines GUIDE.md carries
2830 bun test # the check every pull runs
2931 bun run scripts/mutate-result.ts # break each rule in src/result.ts, see a test go red
3032 FORGE_APP_DATA=./data bun run src/tty.ts # the terminal interface
@@ -610,6 +612,22 @@ never checked itself.
610612 and compares its names against the surface `boundary.test.ts` pins, so a pack
611613 that dropped a file fails here rather than in somebody else's repository. The
612614 copy is Bun's: the modules import each other with `.ts` extensions.
615 - `scripts/cost.ts` — what each group behind the door costs a bundle, measured.
616 The ceilings below weigh whole games; this weighs one group. For each one it
617 generates an entry that imports exactly that group through
618 `src/engine/index.ts` and uses every name in it, bundles it the way
619 `scripts/build.ts` bundles a game — browser, minified, esm — and weighs the
620 output raw and gzipped at level 9, the same ruler the ceilings use. Groups are
621 not independent, so each row also says which modules came along and how many
622 of the bundle's bytes are theirs, read off the build's source map rather than
623 off the import lines: a module the bundler shook away is charged to nobody,
624 which is how the numbers say out loud that the range ships none of the WebGL.
625 `bun run scripts/cost.ts` prints the table; `--guide` prints the lines
626 `src/engine/GUIDE.md` carries under each group's heading, and
627 `scripts/cost.test.ts` measures again on every run and fails when a number on
628 that page has drifted more than a tenth from what the bundler does today. Like
629 the pack, it writes only inside a temporary directory it removes, ships in no
630 bundle and changes no game.
613631 - `scripts/surface.ts` — the door's surface and the version it belongs to, kept
614632 together. `src/engine/surface.json` is the record: the sorted list of names
615633 the door offers and the version stamped into a pack of them, and nothing
scripts/cost.test.ts added +281
@@ -0,0 +1,281 @@
1 /**
2 * The cost table, held against the guide. `scripts/cost.ts` measures what each
3 * group behind the door adds to a bundle; `src/engine/GUIDE.md` prints those
4 * numbers beside the paragraph a game author is already reading. A number on a
5 * page is a number that rots, so this file measures again and fails when the
6 * page and the bundler disagree.
7 *
8 * Three rules, and they are the whole of the honesty here:
9 *
10 * 1. Every group the door exports carries a `**Weight**` line under its
11 * heading, so a group that lands behind the door arrives with a number or it
12 * fails here.
13 * 2. Every `**Weight**` line in the guide is under a heading that is a group, so
14 * a renamed module cannot leave a stray measurement on the page.
15 * 3. Every line's numbers are within `TOLERANCE` of what the script measures
16 * today, and the modules it says come along are exactly the ones that do.
17 *
18 * **The tolerance**, written here rather than left for a reader to guess: a
19 * number passes when it is within 10% of what was measured, or within
20 * `GRAIN` — 100 bytes — whichever is wider. The floor is the page's own
21 * resolution: the guide writes tenths of a kilobyte, so rounding alone moves a
22 * number by up to 50 bytes and must never fail a run. The band above it is for
23 * the minifier: a bun release that renames a few identifiers differently, or
24 * shortens a helper, moves a bundle by a few percent and is nobody's bug. A
25 * group that doubled is far outside both — the smallest measurement on the
26 * table is under half a kilobyte, so doubling it moves it by more than 300
27 * bytes — which is the drift this test exists to catch.
28 *
29 * It measures for real: the script's own builds, in a temporary directory of
30 * its own that it removes. That is forty-odd `Bun.build` calls and takes well
31 * under a second, which is cheaper than a stale page.
32 */
33
34 import { expect, test } from "bun:test";
35 import { readFileSync } from "node:fs";
36 import { join } from "node:path";
37 import {
38 type Cost,
39 DOOR,
40 FLOOR,
41 SLACK,
42 WHOLE_DOOR,
43 costs,
44 entryFor,
45 groupsOf,
46 guideLine,
47 headingOf,
48 kb,
49 listed,
50 moduleOf,
51 pullsOf,
52 sharesOf,
53 table,
54 vlq,
55 } from "./cost.ts";
56
57 const GUIDE = join(import.meta.dir, "..", "src", "engine", "GUIDE.md");
58 const guide = (): string => readFileSync(GUIDE, "utf8");
59
60 /** How far a written number may sit from a measured one before it is drift. */
61 export const TOLERANCE = 0.1;
62
63 /** The page's own resolution in bytes: a tenth of a kilobyte, either way. */
64 export const GRAIN = 100;
65
66 /** Is this written number still the measured one? The rule in the docstring, once. */
67 export const near = (written: number, measured: number): boolean =>
68 Math.abs(written - measured) <= Math.max(GRAIN, measured * TOLERANCE);
69
70 /** A weight the guide carries: the heading it sits under, and the line itself. */
71 export type Written = { heading: string; line: string };
72
73 /**
74 * Every `**Weight**` line in the guide, under the heading it belongs to — the
75 * whole heading, hashes and all, because the whole door's line sits under a
76 * `##` and a group's under a `###`. A line before any heading comes back under
77 * the empty name, which the tests below turn into a sentence rather than
78 * passing quietly.
79 */
80 export function weightsOf(markdown: string): Written[] {
81 const out: Written[] = [];
82 let heading = "";
83 for (const line of markdown.split("\n")) {
84 const found = /^#{2,3} .+$/.exec(line.trim());
85 if (found !== null) heading = line.trim();
86 else if (line.startsWith("**Weight**")) out.push({ heading, line: line.trim() });
87 }
88 return out;
89 }
90
91 /** The numbers and the tail of one written line, or null when it is not one. */
92 export function readWeight(line: string): { raw: number; gzip: number; tail: string } | null {
93 const m = /^\*\*Weight\*\*: ([0-9.]+) kB raw, ([0-9.]+) kB gzipped — (.+)$/.exec(line);
94 if (m === null) return null;
95 return {
96 raw: Number(m[1]) * 1000,
97 gzip: Number(m[2]) * 1000,
98 tail: (m[3] as string).trim(),
99 };
100 }
101
102 /** The measurement, made once and shared by every test below it. */
103 const measured: Cost[] = await costs();
104 const groups = measured.filter((cost) => cost.name !== FLOOR && cost.name !== WHOLE_DOOR);
105 /** What the page prints a number for: every group, and all of them together. */
106 const priced = measured.filter((cost) => cost.name !== FLOOR);
107 const byName = new Map(measured.map((cost) => [cost.name, cost]));
108
109 test("the door's groups are the modules behind it, values only, in the door's order", () => {
110 const F = "fr" + "om";
111 const source = [
112 `export { a, b as c } ${F} "./one.ts";`,
113 `export type { D } ${F} "./one.ts";`,
114 `export { e, type G } ${F} "./two.ts";`,
115 `export { f } ${F} "./one.ts";`,
116 ].join("\n");
117 expect(groupsOf(source)).toEqual([
118 { name: "one", names: ["a", "c", "f"] },
119 { name: "two", names: ["e"] },
120 ]);
121 // And the real door: one group a module, every one of them measured.
122 const real = groupsOf(readFileSync(DOOR, "utf8"));
123 expect(real.length).toBeGreaterThan(15);
124 expect(real.map((group) => group.name)).toEqual(groups.map((cost) => cost.name));
125 expect(new Set(real.map((group) => group.name)).size).toBe(real.length);
126 });
127
128 test("the generated entry imports exactly its group and holds on to every name", () => {
129 const entry = entryFor({ name: "mesh", names: ["builder", "finish"] }, "/engine/index.ts", "/tmp/x");
130 expect(entry).toContain('import { builder, finish } from "../../engine/index.ts";');
131 expect(entry).toContain("globalThis.__cost = [builder, finish];");
132 // A group with no names is still an import, because the floor is a measurement.
133 expect(entryFor({ name: FLOOR, names: [] }, "/engine/index.ts", "/tmp/x")).toContain(
134 'import "../../engine/index.ts";',
135 );
136 });
137
138 test("a base64 VLQ segment is the signed numbers it packs", () => {
139 expect(vlq("A")).toEqual([0]);
140 expect(vlq("C")).toEqual([1]);
141 expect(vlq("D")).toEqual([-1]);
142 expect(vlq("AAAA")).toEqual([0, 0, 0, 0]);
143 expect(vlq("gB")).toEqual([16]);
144 expect(vlq("hB")).toEqual([-16]);
145 expect(() => vlq("!")).toThrow(/base64 VLQ/);
146 });
147
148 test("a source is an engine module, or it is not one", () => {
149 expect(moduleOf("src/engine/math.ts", "/repo", "/repo/src/engine")).toBe("math");
150 expect(moduleOf("../tmp/cost-1/math.entry.ts", "/repo", "/repo/src/engine")).toBeNull();
151 // The door itself holds no code, so nothing is ever charged to it.
152 expect(moduleOf("src/engine/index.ts", "/repo", "/repo/src/engine")).toBeNull();
153 });
154
155 test("the bytes of a bundle are charged to the module the mappings name", () => {
156 // Three segments on one generated line of ten bytes: the first owns columns
157 // 0..3 and is the first source, the second owns 4..7 and steps on to the
158 // second source, and the third is one field — generated code that belongs to
159 // no source, so the last two bytes are charged to nobody.
160 const map = JSON.stringify({
161 sources: ["src/engine/math.ts", "src/engine/mesh.ts", "entry.ts"],
162 mappings: "AAAA,ICAA,I",
163 });
164 const shares = sharesOf(map, "0123456789", "/repo", "/repo/src/engine");
165 expect(shares.get("math")).toBe(4);
166 expect(shares.get("mesh")).toBe(4);
167 expect(shares.has("entry")).toBe(false);
168 expect(() => sharesOf("{}", "x")).toThrow(/sources and mappings/);
169 });
170
171 test("a module counts as brought along when it beats the floor, not when it jitters", () => {
172 const floor = new Map([["net", 31]]);
173 const shares = new Map([
174 ["mesh", 800],
175 ["net", 32],
176 ["math", 120],
177 ]);
178 // `net` is a byte heavier than the floor's copy of it: that is the minifier,
179 // not a dependency. `math` is not in the floor at all, so any of it counts.
180 expect(pullsOf("mesh", shares, floor)).toEqual([{ module: "math", raw: 120 }]);
181 expect(pullsOf("mesh", new Map([...shares, ["net", 31 + SLACK + 1]]), floor)).toEqual([
182 { module: "net", raw: 48 },
183 { module: "math", raw: 120 },
184 ].sort((a, b) => b.raw - a.raw));
185 // The floor keeps everything it holds, because that is what it is for.
186 expect(pullsOf(FLOOR, floor, floor)).toEqual([{ module: "net", raw: 31 }]);
187 });
188
189 test("the units and the prose are the units and the prose", () => {
190 expect(kb(1814)).toBe("1.8 kB");
191 expect(kb(53)).toBe("0.1 kB");
192 expect(listed([])).toBe("");
193 expect(listed(["math"])).toBe("math");
194 expect(listed(["math", "mesh"])).toBe("math and mesh");
195 expect(listed(["math", "mesh", "scene"])).toBe("math, mesh and scene");
196 expect(guideLine({ name: "trace", raw: 1008, gzip: 599, pulls: [{ module: "math", raw: 83 }] })).toBe(
197 "**Weight**: 1.0 kB raw, 0.6 kB gzipped — including math, which comes with it.",
198 );
199 expect(guideLine({ name: "mesh", raw: 916, gzip: 496, pulls: [] })).toBe(
200 "**Weight**: 0.9 kB raw, 0.5 kB gzipped — this module alone.",
201 );
202 expect(readWeight("**Weight**: 1.0 kB raw, 0.6 kB gzipped — this module alone.")).toEqual({
203 raw: 1000,
204 gzip: 600,
205 tail: "this module alone.",
206 });
207 expect(readWeight("a paragraph")).toBeNull();
208 });
209
210 test("the measurement is a bundle per group, the floor and the whole door", () => {
211 expect(byName.get(FLOOR)?.raw).toBeGreaterThan(0);
212 expect(byName.get(FLOOR)?.raw).toBeLessThan(500);
213 const whole = byName.get(WHOLE_DOOR);
214 expect(whole?.raw).toBeGreaterThan(20_000);
215 // Every group is smaller than all of them together, and none is empty.
216 for (const cost of groups) {
217 expect(cost.raw).toBeGreaterThan(200);
218 expect(cost.raw).toBeLessThan(whole?.raw ?? 0);
219 expect(cost.gzip).toBeLessThan(cost.raw);
220 expect(cost.pulls.map((pull) => pull.module)).not.toContain(cost.name);
221 }
222 // Tree-shaking is the point: a game that wants the arithmetic does not get
223 // the WebGL, and one that wants the renderer does get the arithmetic.
224 expect(byName.get("math")?.pulls).toEqual([]);
225 expect(byName.get("renderer")?.pulls.map((pull) => pull.module)).toContain("math");
226 expect((byName.get("math")?.raw ?? 0) * 3).toBeLessThan(byName.get("renderer")?.raw ?? 0);
227 });
228
229 test("the table a person reads has a row for every group and its units under it", () => {
230 const lines = table(measured);
231 expect(lines[0]).toContain("raw");
232 expect(lines[0]).toContain("gzip-9");
233 expect(lines.length).toBe(measured.length + 2);
234 for (const cost of measured) expect(lines.some((line) => line.startsWith(cost.name))).toBe(true);
235 });
236
237 test("every group the door offers carries its weight in the guide, and nothing else does", () => {
238 const written = weightsOf(guide());
239 const under = new Set(written.map((weight) => weight.heading));
240 const offences: string[] = [];
241 for (const cost of priced)
242 if (!under.has(headingOf(cost.name)))
243 offences.push(
244 `GUIDE.md has no "**Weight**" line under "${headingOf(cost.name)}": ` +
245 `run \`bun run scripts/cost.ts --guide\` and put the line it prints there`,
246 );
247 const headings = new Set(priced.map((cost) => headingOf(cost.name)));
248 for (const weight of written)
249 if (!headings.has(weight.heading))
250 offences.push(
251 `GUIDE.md writes a weight under "${weight.heading}", which is not a group the door exports`,
252 );
253 expect(offences).toEqual([]);
254 expect(written.length).toBe(priced.length);
255 });
256
257 test("every weight the guide writes is what the bundler measures today", () => {
258 const written = new Map(weightsOf(guide()).map((weight) => [weight.heading, weight.line]));
259 const offences: string[] = [];
260 for (const cost of priced) {
261 const line = written.get(headingOf(cost.name));
262 if (line === undefined) continue; // the test above is the one that says so
263 const read = readWeight(line);
264 if (read === null) {
265 offences.push(`GUIDE.md's weight for ${cost.name} is not in the shape "${guideLine(cost)}"`);
266 continue;
267 }
268 const fresh = readWeight(guideLine(cost));
269 if (fresh === null) throw new Error("the generated line no longer reads as one");
270 if (!near(read.raw, cost.raw) || !near(read.gzip, cost.gzip))
271 offences.push(
272 `GUIDE.md says ${cost.name} is ${line} — it measures ${cost.raw} B raw and ` +
273 `${cost.gzip} B gzipped today, so the line is \`${guideLine(cost)}\``,
274 );
275 if (read.tail !== fresh.tail)
276 offences.push(
277 `GUIDE.md says ${cost.name} brings "${read.tail}" and it brings "${fresh.tail}"`,
278 );
279 }
280 expect(offences).toEqual([]);
281 });
scripts/cost.ts added +389
@@ -0,0 +1,389 @@
1 /**
2 * What each group behind the engine's door costs a bundle, measured:
3 *
4 * bun run scripts/cost.ts # the table
5 * bun run scripts/cost.ts --guide # the lines GUIDE.md should carry
6 *
7 * The ceilings in `src/games.ts` weigh whole games, which answers "is the
8 * shooter still small" and not the question a game author actually asks —
9 * *what does it cost me to import this?* This script answers that one the only
10 * honest way: it generates a tiny entry that imports exactly one group from
11 * `src/engine/index.ts` and uses every name in it, bundles that entry with the
12 * settings `scripts/build.ts` uses for a real game — browser, minified, esm —
13 * and weighs the output raw and gzipped at `GZIP_LEVEL`, the same ruler the
14 * ceilings are held to.
15 *
16 * **A group is not independent, and the table says so rather than pretending.**
17 * The renderer is drawn with the arithmetic; a bundle that has the renderer in
18 * it has that too, and there is no number for "the renderer alone" because no
19 * game can buy one. So every measurement is *including what it brings with it*,
20 * and what it brought is not asserted from the import lines but read off the
21 * build: the source map says which module every byte of the output came from,
22 * so a module the bundler loaded and then shook away is charged nothing. That
23 * is why the range ships none of the WebGL, and it is the number that says so.
24 *
25 * Nothing is subtracted from anything. A group's number is what a bundle that
26 * imports that group and nothing else actually weighs — the entry's own
27 * `globalThis` assignment, which is what keeps the names from being shaken
28 * away, included — so the rows can be compared with each other and with the
29 * whole-door row at the bottom without a reader holding an offset in their
30 * head. The entry with no group in it is a row of its own for the same reason.
31 *
32 * It ships in no bundle and no game imports it: this is a script a person runs
33 * and `cost.test.ts` runs. It writes only inside a directory it makes under the
34 * OS temp directory and removes afterwards, like `pack.test.ts` does; nothing
35 * lands in the repository and nothing touches `FORGE_APP_DATA`.
36 */
37
38 import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
39 import { tmpdir } from "node:os";
40 import { join, relative, resolve } from "node:path";
41 import { type Ceiling, GZIP_LEVEL, sizes } from "./build.ts";
42
43 const ROOT = join(import.meta.dir, "..");
44
45 /** The engine, and the one file outside it that anybody may name. */
46 export const ENGINE = join(ROOT, "src", "engine");
47 export const DOOR = join(ENGINE, "index.ts");
48
49 /**
50 * One group the door offers: the module it comes from — which is the name the
51 * guide gives its section — and the runtime names re-exported from it. Types
52 * are not in here: `export type { … }` weighs nothing in a bundle, so a group
53 * of nothing but types would be measuring the harness.
54 */
55 export type Group = { name: string; names: string[] };
56
57 /**
58 * The groups of a barrel, in the order it writes them, which is the order the
59 * guide's sections are in. Only value re-exports count — `export type { … }` is
60 * skipped whole, and a `type X` written inside a value clause is skipped with
61 * it — and two clauses from one module are one group, because a game importing
62 * that module's names gets one module either way.
63 */
64 export function groupsOf(source: string): Group[] {
65 const groups = new Map<string, string[]>();
66 const pattern = /\bexport\s+(type\s+)?\{([^}]*)\}\s*from\s*["']\.\/([A-Za-z0-9_-]+)\.ts["']/g;
67 for (let m = pattern.exec(source); m !== null; m = pattern.exec(source)) {
68 if (m[1] !== undefined) continue;
69 const module = m[3] as string;
70 const names = (m[2] as string)
71 .split(",")
72 .map((clause) => clause.trim())
73 .filter((clause) => clause !== "" && !/^type\s/.test(clause))
74 .map((clause) => clause.split(/\s+as\s+/).pop()?.trim() ?? "");
75 groups.set(module, [...(groups.get(module) ?? []), ...names.filter((name) => name !== "")]);
76 }
77 return [...groups].map(([name, names]) => ({ name, names }));
78 }
79
80 /**
81 * The entry that measures a group: import those names through the door, and
82 * park them on `globalThis` so the bundler cannot shake away what it was asked
83 * to weigh. An assignment to a property of the global object is a side effect,
84 * so minification keeps it and everything it reaches.
85 */
86 export function entryFor(group: Group, door = DOOR, dir = ""): string {
87 const from = dir === "" ? door : relative(dir, door);
88 const path = from.startsWith(".") || from.startsWith("/") ? from : `./${from}`;
89 const names = group.names.join(", ");
90 return [
91 `// generated by scripts/cost.ts: what ${group.name} costs, and nothing else`,
92 names === "" ? `import "${path}";` : `import { ${names} } from "${path}";`,
93 `globalThis.__cost = [${names}];`,
94 "",
95 ].join("\n");
96 }
97
98 /** A module a group brought with it, and the bytes of the bundle that are its. */
99 export type Pull = { module: string; raw: number };
100
101 /** What one group weighs, and what came with it — its own module never among them. */
102 export type Cost = Ceiling & { name: string; pulls: Pull[] };
103
104 /** Every group at once: what a game that used all of it would carry. */
105 export const WHOLE_DOOR = "the whole door";
106
107 /**
108 * The floor: the door imported and none of it used. It is not zero — `net`
109 * allocates a scratch array at module scope and a bundler may not drop an
110 * allocation it cannot prove nobody observes — and every group's number has
111 * that floor inside it. So the table prints the floor as its own row, and a
112 * module counts as brought along only when it weighs *more* in a group's bundle
113 * than it does in this one: `mesh` is not dragging `net` in, and `learn` is.
114 */
115 export const FLOOR = "the door, unused";
116
117 /** What one bundle of a generated entry came to: its bytes, and whose bytes they are. */
118 export type Weighed = Ceiling & { shares: Map<string, number> };
119
120 /** One build, with the settings `scripts/build.ts` gives a game. */
121 const build = (file: string, sourcemap: boolean): Promise<Bun.BuildOutput> =>
122 Bun.build({
123 entrypoints: [file],
124 target: "browser",
125 minify: true,
126 format: "esm",
127 ...(sourcemap ? { sourcemap: "external" as const } : {}),
128 });
129
130 /** The one output of a build, or the build's own complaint about why there is none. */
131 async function only(out: Bun.BuildOutput, kind: string, what: string): Promise<string> {
132 if (!out.success) throw new Error(out.logs.map(String).join("\n"));
133 const output = out.outputs.find((file) => file.kind === kind);
134 if (output === undefined) throw new Error(`the entry for ${what} produced no ${kind}`);
135 return await output.text();
136 }
137
138 /**
139 * The engine module a source map's source path names, or null when the path is
140 * not one: the generated entry, and the door itself, which holds no code.
141 *
142 * A build with no file to write puts its sources relative to the working
143 * directory, so `base` is where they are resolved from and the caller's own
144 * directory is the only sensible default.
145 */
146 export function moduleOf(source: string, base = process.cwd(), engine = ENGINE): string | null {
147 const inside = relative(engine, resolve(base, source));
148 if (inside.startsWith("..") || !inside.endsWith(".ts")) return null;
149 const module = inside.replace(/\.ts$/, "");
150 return module === "index" ? null : module;
151 }
152
153 /** The digits of a base64 VLQ, in the order the format numbers them. */
154 const DIGITS = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
155
156 /**
157 * One segment of a source map's `mappings`, as the signed numbers it packs:
158 * base64 VLQ, continuation bit 0x20, sign in the low bit of the first digit.
159 * A field it does not carry is simply absent — a one-field segment is generated
160 * code that belongs to no source, which is exactly what this file must not
161 * charge to a module.
162 */
163 export function vlq(segment: string): number[] {
164 const out: number[] = [];
165 let value = 0;
166 let shift = 0;
167 for (const char of segment) {
168 const digit = DIGITS.indexOf(char);
169 if (digit < 0) throw new Error(`"${segment}" is not a base64 VLQ segment`);
170 value += (digit & 0x1f) << shift;
171 if ((digit & 0x20) !== 0) {
172 shift += 5;
173 continue;
174 }
175 out.push((value & 1) === 1 ? -(value >> 1) : value >> 1);
176 value = 0;
177 shift = 0;
178 }
179 return out;
180 }
181
182 /**
183 * Whose bytes the bundle is made of: every engine module in it, against the
184 * number of generated bytes the source map attributes to it. This is how "what
185 * came with it" is measured rather than argued — a module the bundler loaded
186 * and then shook away is charged nothing, which is the whole reason the range
187 * ships none of the WebGL — and it is per module rather than a bare list
188 * because two bundles can hold the same module and one line of it.
189 *
190 * A span runs from a segment's generated column to the next segment's, or to
191 * the end of that line. The count is of the *mapped* output, which is the
192 * source-mapped build and so a few bytes longer than the shipped one; the
193 * shares are a breakdown of a bundle, not a second measurement of it.
194 */
195 export function sharesOf(
196 map: string,
197 code: string,
198 base = process.cwd(),
199 engine = ENGINE,
200 ): Map<string, number> {
201 const { sources, mappings } = JSON.parse(map) as { sources?: unknown; mappings?: unknown };
202 if (!Array.isArray(sources) || typeof mappings !== "string")
203 throw new Error("the source map has no sources and mappings to read");
204 const owners = (sources as string[]).map((source) => moduleOf(source, base, engine));
205 const lines = code.split("\n");
206 const out = new Map<string, number>();
207 let source = 0;
208 const rows = mappings.split(";");
209 for (let row = 0; row < rows.length; row++) {
210 let column = 0;
211 const at: number[] = [];
212 const whose: (string | null)[] = [];
213 for (const segment of (rows[row] as string).split(",")) {
214 if (segment === "") continue;
215 const fields = vlq(segment);
216 column += fields[0] ?? 0;
217 if (fields.length >= 4) source += fields[1] ?? 0;
218 at.push(column);
219 whose.push(fields.length >= 4 ? (owners[source] ?? null) : null);
220 }
221 const end = (lines[row] ?? "").length;
222 for (let i = 0; i < at.length; i++) {
223 const module = whose[i];
224 const span = (i + 1 < at.length ? (at[i + 1] as number) : end) - (at[i] as number);
225 if (module === null || module === undefined || span <= 0) continue;
226 out.set(module, (out.get(module) ?? 0) + span);
227 }
228 }
229 return out;
230 }
231
232 /**
233 * Bundle one generated entry twice: once for the bytes, and once with a source
234 * map for the breakdown. Twice, because an external source map appends a
235 * `debugId` comment to the JavaScript, and the number this script prints has to
236 * be the bytes a game would actually ship rather than those plus a comment.
237 */
238 export async function weighEntry(name: string, file: string): Promise<Weighed> {
239 const code = await only(await build(file, false), "entry-point", name);
240 const mapped = await build(file, true);
241 const shares = sharesOf(
242 await only(mapped, "sourcemap", name),
243 await only(mapped, "entry-point", name),
244 );
245 return { ...sizes(code), shares };
246 }
247
248 /**
249 * Every group the door offers, measured, with `FLOOR` first and `WHOLE_DOOR`
250 * last. Each entry is written into `dir`, which the caller owns. A group's
251 * `pulls` is what its bundle holds beyond its own module and beyond the floor
252 * every import of the door pays.
253 */
254 export async function measure(groups: Group[], dir: string): Promise<Cost[]> {
255 const all: Group[] = [
256 { name: FLOOR, names: [] },
257 ...groups,
258 { name: WHOLE_DOOR, names: groups.flatMap((group) => group.names) },
259 ];
260 const out: Cost[] = [];
261 let floor = new Map<string, number>();
262 for (const group of all) {
263 const file = join(dir, `${group.name.replace(/[^a-z]+/g, "-")}.entry.ts`);
264 writeFileSync(file, entryFor(group, DOOR, dir));
265 const { raw, gzip, shares } = await weighEntry(group.name, file);
266 if (group.name === FLOOR) floor = shares;
267 out.push({ name: group.name, raw, gzip, pulls: pullsOf(group.name, shares, floor) });
268 }
269 return out;
270 }
271
272 /**
273 * The bytes by which a module already in the floor has to beat the floor before
274 * a group is said to have dragged it in. A minifier names identifiers by how
275 * often they are used, so the same two lines of `net` come out a byte or two
276 * longer in one bundle than in another; that jitter is a few bytes and this is
277 * wider than it. A module the floor does not charge at all is counted at any
278 * size, because one line of it is still a line a game did not ask for.
279 */
280 export const SLACK = 16;
281
282 /**
283 * What a bundle brought along: every engine module in it but the group's own,
284 * charged more bytes here than the floor charges it. The floor's own row keeps
285 * everything, because what it brings along is the whole of what it is.
286 */
287 export function pullsOf(
288 name: string,
289 shares: Map<string, number>,
290 floor: Map<string, number>,
291 ): Pull[] {
292 const beyondTheFloor = (module: string, raw: number): boolean => {
293 const paid = floor.get(module) ?? 0;
294 return raw > paid + (paid === 0 ? 0 : SLACK);
295 };
296 return [...shares]
297 .filter(([module, raw]) => module !== name && (name === FLOOR || beyondTheFloor(module, raw)))
298 .map(([module, raw]) => ({ module, raw }))
299 .sort((a, b) => b.raw - a.raw || a.module.localeCompare(b.module));
300 }
301
302 /** Measure in a temporary directory of this run's own, removed however it ends. */
303 export async function costs(source = readFileSync(DOOR, "utf8")): Promise<Cost[]> {
304 const dir = mkdtempSync(join(tmpdir(), "cost-"));
305 try {
306 return await measure(groupsOf(source), dir);
307 } finally {
308 rmSync(dir, { recursive: true, force: true });
309 }
310 }
311
312 /**
313 * Bytes as the guide writes them: kilobytes of a thousand bytes, one decimal.
314 * A tenth of a kilobyte is the resolution of the page, and `TOLERANCE` below is
315 * wide enough to cover the rounding this does.
316 */
317 export const kb = (bytes: number): string => `${(bytes / 1000).toFixed(1)} kB`;
318
319 /** `and` as a person writes it: `a`, `a and b`, `a, b and c`. */
320 export function listed(names: readonly string[]): string {
321 if (names.length <= 1) return names[0] ?? "";
322 return `${names.slice(0, -1).join(", ")} and ${names[names.length - 1]}`;
323 }
324
325 /**
326 * The sentence the guide carries under a group's heading. It is generated here
327 * rather than written by hand so that the page and the measurement cannot drift
328 * into two different claims; `cost.test.ts` holds today's guide against it.
329 */
330 export function guideLine({ name, raw, gzip, pulls }: Cost): string {
331 const tail =
332 name === WHOLE_DOOR
333 ? "every group at once, which is the most any game can pay"
334 : pulls.length === 0
335 ? "this module alone"
336 : `including ${listed(pulls.map((pull) => pull.module))}, which come${
337 pulls.length === 1 ? "s" : ""
338 } with it`;
339 return `**Weight**: ${kb(raw)} raw, ${kb(gzip)} gzipped — ${tail}.`;
340 }
341
342 /**
343 * The guide's heading a measurement belongs under: a group's own section, and
344 * for the whole door the section all of them are in. `cost.test.ts` reads the
345 * page by these, so the script that prints the lines and the test that checks
346 * them cannot disagree about where a line goes.
347 */
348 export const SECTION = "What is behind the door";
349 export const headingOf = (name: string): string =>
350 name === WHOLE_DOOR ? `## ${SECTION}` : `### ${name}`;
351
352 /** What the table is a table of, printed under it so a pasted table carries its units. */
353 export const NOTE = [
354 "",
355 `Bytes. Every row is one bundle: an entry that imports exactly that group through`,
356 `src/engine/index.ts and uses every name in it, bundled the way scripts/build.ts`,
357 `bundles a game — browser, minified, esm — and gzipped at level ${GZIP_LEVEL}.`,
358 `So a row includes what the group brings with it, and the floor every import of`,
359 `the door pays. Run: bun run scripts/cost.ts`,
360 ].join("\n");
361
362 /** The whole table, as lines a person reads: one group a row, widest name first. */
363 export function table(measured: readonly Cost[]): string[] {
364 const width = Math.max(...measured.map((cost) => cost.name.length));
365 const cell = (text: string, size: number): string => text.padStart(size);
366 const rows = measured.map(
367 (cost) =>
368 `${cost.name.padEnd(width)} ${cell(String(cost.raw), 7)} ${cell(String(cost.gzip), 7)} ` +
369 (cost.pulls.length === 0
370 ? "—"
371 : cost.pulls.map((pull) => `${pull.module} ${pull.raw}`).join(", ")),
372 );
373 return [
374 `${"group".padEnd(width)} ${cell("raw", 7)} ${cell(`gzip-${GZIP_LEVEL}`, 7)} brings with it`,
375 `${"-".repeat(width)} ${"-".repeat(7)} ${"-".repeat(7)} ${"-".repeat(14)}`,
376 ...rows,
377 ];
378 }
379
380 if (import.meta.main) {
381 const measured = await costs();
382 if (process.argv.includes("--guide")) {
383 for (const cost of measured)
384 if (cost.name !== FLOOR) console.log(`${headingOf(cost.name)}\n${guideLine(cost)}\n`);
385 } else {
386 for (const line of table(measured)) console.log(line);
387 console.log(NOTE);
388 }
389 }
src/engine/GUIDE.md +64
@@ -150,8 +150,36 @@ const SPEED = 4, TURN = 120, EYE = 1.6, REACH = 40;
150150
151151 ## What is behind the door
152152
153 **Weight**: 36.1 kB raw, 14.7 kB gzipped — every group at once, which is the most any game can pay.
154
155 Every section below opens with what that group costs a bundle, because "what
156 does it cost to use?" is a question a page can answer with a number. The number
157 is measured, not estimated: `scripts/cost.ts` generates an entry that imports
158 exactly that group through the door and uses every name in it, bundles it the
159 way `scripts/build.ts` bundles a game — browser, minified, esm — and weighs the
160 result on disk and gzipped at level 9, which is the same ruler the games' own
161 ceilings in `src/games.ts` are held to. `bun run scripts/cost.ts` prints the
162 whole table, and `scripts/cost.test.ts` measures again on every run and fails
163 when a number on this page has drifted from what the bundler does today.
164
165 Two things those numbers are careful about. **Groups are not independent**, so
166 each line says what comes with it — a bundle with the renderer in it has the
167 arithmetic too — and that list is read off the build rather than off the import
168 lines: a module the bundler shook away is charged to nobody, which is why the
169 range ships none of the WebGL. **And every number has the same small floor in
170 it**, about fifty bytes: importing the door at all keeps one scratch array that
171 is allocated at module scope, and a bundler may not drop an allocation it cannot
172 prove nobody watches. A group is called out as bringing another module along
173 only when it weighs more there than in that floor.
174
175 The lines add up about the way you would expect them to. A game that draws
176 through the renderer also imports the meshes, the atlas and the scene it hands
177 it, and pays those lines too — less whatever arithmetic they already share.
178
153179 ### math
154180
181 **Weight**: 1.8 kB raw, 0.9 kB gzipped — this module alone.
182
155183 The arithmetic under everything, in `Vec3` (a plain `[x, y, z]`) and `Mat4` (a
156184 `Float32Array`, column-major, ready for `uniformMatrix4fv`). Vectors: `v3`,
157185 `add`, `sub`, `scale`, `mul`, `dot`, `cross`, `normalize`, `length`, `length2d`,
@@ -169,6 +197,8 @@ const to = add(world.pos, scale(angleVectors(world.yaw, 0).forward, REACH));
169197
170198 ### trace
171199
200 **Weight**: 1.0 kB raw, 0.6 kB gzipped — including math, which comes with it.
201
172202 Rays against axis-aligned boxes: `rayBox` for one box, `traceWorld` for a list
173203 of them, `visible` for the yes/no of an unobstructed line, `segmentLength` for
174204 how far the segment actually ran. A `Box` is `{ min, max, mat }` — the material
@@ -195,6 +225,8 @@ if (rayBox(world.pos, to, world.target.min, world.target.max) === null) return f
195225
196226 ### physics
197227
228 **Weight**: 2.7 kB raw, 1.2 kB gzipped — including math, which comes with it.
229
198230 Walking, in the Quake lineage: ground friction, a capped ground acceleration,
199231 and an air acceleration that caps only the part of your wish speed along the new
200232 direction — which is why turning while you accelerate in the air makes you
@@ -248,6 +280,8 @@ exactly that reason.
248280
249281 ### grid
250282
283 **Weight**: 2.5 kB raw, 1.2 kB gzipped — this module alone.
284
251285 A `Grid` of cells of any type — the cell type is your alphabet, not the
252286 engine's. `inside` and `cellAt` read it, `worldOf`, `colOf` and `rowOf` are the
253287 cell-to-world arithmetic, `mergeSolid` greedily merges runs of matching cells
@@ -263,6 +297,8 @@ alphabet over this and is one of them.
263297
264298 ### mesh
265299
300 **Weight**: 0.9 kB raw, 0.5 kB gzipped — this module alone.
301
266302 `builder` opens a `Builder`, `addBox` writes six quads of positions, normals,
267303 uvs and a layer into it, and `finish` packs it into a `Mesh` of typed arrays a
268304 renderer uploads once. `quadMesh` is the full-screen quad for anything drawn
@@ -275,6 +311,8 @@ from within and its normal points into the room.
275311
276312 ### texture
277313
314 **Weight**: 1.0 kB raw, 0.6 kB gzipped — this module alone.
315
278316 A `Shader` is `(u, v) => [r, g, b]`, and that is the whole of the interface.
279317 `hash` is the integer hash under it, `noise` is one octave of value noise at a
280318 period, `fbm` is a stack of them, and `paintAtlas` runs a list of shaders into
@@ -301,6 +339,8 @@ call: the range's three 64-pixel layers are about 17 ms, and `paintAtlas` is
301339
302340 ### sound
303341
342 **Weight**: 1.8 kB raw, 1.0 kB gzipped — this module alone.
343
304344 `createSoundDevice` gives a `SoundDevice` that opens its `AudioContext` on the
305345 first sound, never at import — hand it a `Ctor` of your own and a test can count
306346 what was played. It takes a `Burst` (band-passed noise: a gunshot, an impact)
@@ -335,6 +375,8 @@ than dropping the frame.
335375
336376 ### loop
337377
378 **Weight**: 0.6 kB raw, 0.4 kB gzipped — this module alone.
379
338380 `createLoop` takes `LoopOptions` — the tick, and optionally the most steps one
339381 frame may run and the longest frame the clock is believed about — and `Frames`,
340382 the clock and the frame source. The `Loop` it returns banks elapsed time and
@@ -357,6 +399,8 @@ number the test moves, which is how a second of play takes a millisecond.
357399
358400 ### controls
359401
402 **Weight**: 2.7 kB raw, 1.1 kB gzipped — including math, which comes with it.
403
360404 `createControls` takes the host and `Options` — the key table, the mouse table,
361405 the keys whose default the page must not take, the look scales, the sensitivity,
362406 the thumb-stick region and whether the mouse is ignored until the pointer is
@@ -383,6 +427,8 @@ export function step(world: World, dt: number, held: ReadonlySet<string>): World
383427
384428 ### fullscreen
385429
430 **Weight**: 0.7 kB raw, 0.4 kB gzipped — this module alone.
431
386432 The stage on the whole screen, which is three calls with four ways to be
387433 absent. `createFullscreen` takes a `FullscreenHost` — the `FullscreenTarget`
388434 that goes full screen, usually the box the canvases sit in, and the
@@ -404,6 +450,8 @@ height it had unless a rule says otherwise.
404450
405451 ### escape
406452
453 **Weight**: 0.8 kB raw, 0.5 kB gzipped — this module alone.
454
407455 Escape is the one key the browser reads before your page does, and on a Mac what
408456 it means is "give the window back": the screen drops out of full screen, which a
409457 player in the middle of a round reads as their window being thrown away. There is
@@ -433,6 +481,8 @@ and only another gesture can ask again, which is your Resume button.
433481
434482 ### overlay
435483
484 **Weight**: 1.2 kB raw, 0.7 kB gzipped — including math, which comes with it.
485
436486 The 2D a game draws over its scene. `overlay` takes the surface, the width and
437487 the height and hands back an `Overlay`: `Overlay.scale`, the canvas width over
438488 1280 clamped to 0.7–1.4, and `Overlay.pad`, the margin everything is laid out
@@ -453,6 +503,8 @@ nothing at all, because a game that draws is a game with a page.
453503
454504 ### scene
455505
506 **Weight**: 1.0 kB raw, 0.5 kB gzipped — this module alone.
507
456508 A frame, as plain data, and the pool it is kept in — the one piece of a game's
457509 frame that is bookkeeping rather than a decision, so it is the engine's.
458510
@@ -492,6 +544,8 @@ and the renderer never sees it.
492544
493545 ### effects
494546
547 **Weight**: 1.0 kB raw, 0.5 kB gzipped — this module alone.
548
495549 What just happened, kept for a moment. A shot leaves a tracer for 80 ms, a
496550 bullet leaves sparks for a quarter of a second, a grenade leaves a sound: each
497551 is a thing the game remembers, stamps with its own clock and throws away when it
@@ -530,6 +584,8 @@ in. A game reads the first two of those and leaves the ring arithmetic here.
530584
531585 ### renderer
532586
587 **Weight**: 7.4 kB raw, 3.1 kB gzipped — including math, which comes with it.
588
533589 The 3D under the scene, and the largest thing behind the door. `createRenderer`
534590 takes the canvas and the assets and hands back a `Renderer` — or nothing at all,
535591 when the browser has no WebGL2, which is a sentence to show the player rather
@@ -566,6 +622,8 @@ game's. One line a frame draws it, and `src/client/main.ts` and
566622
567623 ### page
568624
625 **Weight**: 3.1 kB raw, 1.6 kB gzipped — this module alone.
626
569627 How you boot your entry in a test with no browser. `fakePage` hands back a
570628 `FakePage`: a `FakePage.surface`, a `FakePage.window` and a `FakePage.document`
571629 — which together are exactly the `Host` above — plus the frames your game asked
@@ -610,6 +668,8 @@ reads both committed bundles to say so.
610668
611669 ### digest
612670
671 **Weight**: 1.5 kB raw, 0.9 kB gzipped — this module alone.
672
613673 How you pin your game, so that "it plays exactly as before" is a number rather
614674 than a claim. Like `fakePage`, this is a tool for a test or a bench, never for a
615675 frame: no committed bundle carries a byte of it, and `digest.test.ts` reads the
@@ -647,6 +707,8 @@ a pin that has stopped pinning anything.
647707
648708 ### net
649709
710 **Weight**: 2.8 kB raw, 1.2 kB gzipped — this module alone.
711
650712 A small policy network: an observation in, one choice per categorical head out.
651713 It is the one group behind the door that a game may reasonably never open — the
652714 shooter runs it every tick to fight with, and the range ships not a byte of it —
@@ -693,6 +755,8 @@ second game that wanted an agent would write its own two and share this.
693755
694756 ### learn
695757
758 **Weight**: 5.0 kB raw, 2.1 kB gzipped — including net, which comes with it.
759
696760 PPO: how the weights the section above ships got their numbers. **A game needs
697761 none of this.** A game ships a trained `Policy` and runs `forward`; training
698762 happens once, on a machine with minutes to spare, in a script nothing serves.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs