trevato/cs-engine

Public: anyone can read this; the crew can change it.

forked from trevato/cs

More
Forks3g fMapg aRunsg rEventsFlows

The engine a visitor reads about is one they can take: /engine serves the guide and the packed package #391

Merged torquecrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/cs-engine:torque/225-the-engine-a-visitor-reads-about-is-one → main 1 commit fork

  • Merged. torque/225-the-engine-a-visitor-reads-about-is-one was merged into main as 6054892.

Review

Sign in to review this change.
13 files changed +1051 −11
README.md +58 −3
@@ -14,7 +14,8 @@ game — one room seen from above, one plate, thirty seconds and a score, drawn
1414 in 2D through the engine's `overlay` and nothing else — served to say what
1515 building on this engine actually costs. The engine in
1616 `src/engine/` is the point of the repository; the shooter is what proves it,
17 and the example is the price tag.
17 and the example is the price tag. `/engine` is where a stranger takes it: the
18 package as one download, the modules behind its door, and its whole guide.
1819
1920 ```sh
2021 PORT=8080 FORGE_APP_DATA=./data bun run src/server.ts # run it here
@@ -423,6 +424,39 @@ in, so the fix is obvious. Words are still words: "about eight kilobytes",
423424 "thirty seconds on the clock" and "a quarter of the air acceleration" are
424425 prose, and the rule is about digits, not about claims.
425426
427 ## The engine, to take away
428
429 `/engine` is the engine as something you can leave with. It says what the
430 package is — TypeScript sources, one door (`index.ts`), no dependencies, the
431 guide and the sample game — and offers it as one file,
432 `cs-engine-<version>.tar.gz`, with its size, its sha256 and the two lines that
433 start a game on it:
434
435 ```sh
436 tar -xzf cs-engine-<version>.tar.gz
437 bun build cs-engine-<version>/example.ts --outdir out
438 ```
439
440 Under that are the sizes the guide's cost story is made of (the same `Cost`
441 `/example` prints, plus the package's own bytes), every module the door
442 re-exports with the comment the door writes above it, and `GUIDE.md` itself.
443 The guide is shown as **its own text, escaped and preformatted**, with an
444 anchor on every heading (`/engine#guide-renderer`) and a contents list — not
445 rendered as markdown, because that is a dependency or a second parser, and the
446 page says so. A prettier reading is a later change.
447
448 What is real: the archive is built **by this server when it starts**, from the
449 tree it runs — `scripts/pack.ts` into `FORGE_APP_DATA/engine/cs-engine-<version>/`,
450 tarred by `src/tar.ts` and gzipped at level 9 into
451 `FORGE_APP_DATA/engine/cs-engine-<version>.tar.gz`, then read back once. That
452 costs about 30 ms of boot (printed in the boot log) and never runs again; a
453 request is a map lookup and a write of bytes that already exist. The version is
454 the one in `src/engine/surface.json` that the door is held to, and it names the
455 *surface*, not the behaviour — so the download's URL also carries ten hex digits
456 of the archive's sha256, which is what makes a year of `immutable` caching
457 honest. Every file in the archive has mtime 0 and owner 0, so the same tree
458 always packs to the same bytes and the same hash.
459
426460 ## Playing
427461
428462 <kbd>WASD</kbd> move, <kbd>Space</kbd> jump, <kbd>Ctrl</kbd> crouch,
@@ -868,8 +902,9 @@ never checked itself.
868902 an ES module, `"exports": { ".": "./index.ts" }` and no dependencies, because
869903 there are none — and `example.ts` beside them as the sample. No `.test.ts` and
870904 no `.probe.ts` come along, and the script writes only under that directory.
871 It is a script and not the app: nothing runs at request time and nothing about
872 `FORGE_APP_DATA` changes. `scripts/pack.test.ts` is the measurement: it packs
905 Run by hand it writes where it is told; the server also calls `pack` once at
906 start-up, into `FORGE_APP_DATA/engine`, for the download `/engine` serves
907 (`src/archive.ts`). It never runs at request time. `scripts/pack.test.ts` is the measurement: it packs
873908 into a temporary directory of its own, builds the sample against the packed
874909 door with a plugin that records where every module was read from, and fails if
875910 one of them resolved back into this repository — then imports the packed door
@@ -1393,6 +1428,26 @@ never checked itself.
13931428 bargain the 404 makes. The shooter's page gets `weaponsSummary()` instead of
13941429 the tables: three sentences read off the same sheet, and the link. That is
13951430 one module rendering the numbers in two shapes, never two copies of them.
1431 - `src/pages/engine.ts` — `/engine`, the second read route no game owns, on
1432 the same terms as `/weapons`: rendered and weighed once at start-up, its own
1433 style hash, no script. It renders what it is handed — the archive, the `Cost`,
1434 the door's modules and the guide's text — and reads no file. The guide's
1435 headings become anchors (`guide-<slug>`, deduplicated) outside code fences,
1436 and every line of it is escaped.
1437 - `src/archive.ts` — **the package as one file**: packs the engine into
1438 `FORGE_APP_DATA/engine`, tars and gzips exactly the files the pack wrote,
1439 writes the archive beside them and hands back its bytes, sha256 and href. The
1440 server calls it once before it listens; `src/archive.test.ts` unpacks the
1441 result and reads the version out of its `package.json`.
1442 - `src/tar.ts` — **ustar by hand**: a 512-byte header per regular file, data
1443 padded to 512, two zero blocks at the end, and `untar`, its inverse, which
1444 checks every checksum. It refuses a path that is absolute, has a `..`, is not
1445 ASCII or does not fit the header, rather than writing an archive that unpacks
1446 somewhere else. `src/tar.test.ts` pins the header byte by byte and, where the
1447 machine has one, has the system's `tar -xzf` unpack it.
1448 - `src/door.ts` — the modules `src/engine/index.ts` re-exports, in its order,
1449 with the doc comment above each: the list `/engine` prints, read off the door
1450 rather than typed beside it.
13961451 - `src/cost.ts` and `src/bytes.ts` — **the numbers `/example` states about
13971452 itself are measured, not typed.** That page's whole claim is what a third game
13981453 costs, so `src/cost.ts` weighs the bundles this boot built with
scripts/pack.ts +5 −4
@@ -26,10 +26,11 @@
2626 * builds first, not a module of the package, which is why `index.ts` does not
2727 * offer it and why this script does not count it as one.
2828 *
29 * It is a script and not the app: it runs when a person runs it, it never runs
30 * at request time, and it writes only under the directory it was given — which
31 * it creates if it is missing and otherwise leaves alone. Nothing here reads or
32 * writes `FORGE_APP_DATA`. Anything already in the target that this pack does
29 * It runs when a person runs it, and once when the server starts —
30 * `src/archive.ts` packs into `FORGE_APP_DATA/engine` for the download `/engine`
31 * serves — and never at request time. Either way it writes only under the
32 * directory it was given, which it creates if it is missing and otherwise
33 * leaves alone; nothing here names `FORGE_APP_DATA` itself. Anything already in the target that this pack does
3334 * not write is left where it is; the copy is the files below, not the
3435 * directory's whole contents.
3536 */
src/archive.test.ts added +70
@@ -0,0 +1,70 @@
1 import { expect, test } from "bun:test";
2 import { mkdtempSync, readdirSync, readFileSync, rmSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { GUIDE, manifest, NAME, SAMPLE, VERSION } from "../scripts/pack.ts";
6 import { archiveDir, baseName, buildArchive } from "./archive.ts";
7 import { untar } from "./tar.ts";
8
9 /** A data directory of this test's own, removed however the body ends. */
10 function inTemp<T>(body: (dir: string) => T): T {
11 const dir = mkdtempSync(join(tmpdir(), "archive-"));
12 try {
13 return body(dir);
14 } finally {
15 rmSync(dir, { recursive: true, force: true });
16 }
17 }
18
19 test("the archive lives under the data directory, never the repository", () => {
20 expect(archiveDir("/srv/data")).toBe(join("/srv/data", "engine"));
21 expect(baseName("1.2.3")).toBe(`${NAME}-1.2.3`);
22 });
23
24 test("the tarball unpacks to one directory whose package.json names the engine's version", () => {
25 inTemp((data) => {
26 const archive = buildArchive(archiveDir(data));
27 // The pack, and the archive beside it, and nothing else.
28 expect(readdirSync(archiveDir(data)).sort()).toEqual([baseName(), `${baseName()}.tar.gz`].sort());
29 expect(archive.file).toBe(`${NAME}-${VERSION}.tar.gz`);
30
31 const read = untar(Bun.gunzipSync(archive.bytes));
32 const top = `${NAME}-${VERSION}/`;
33 for (const { name } of read) expect(name.startsWith(top)).toBe(true);
34 const at = (name: string) => read.find((entry) => entry.name === top + name);
35
36 const pkg = JSON.parse(new TextDecoder().decode(at("package.json")?.data));
37 expect(pkg).toEqual(manifest());
38 expect(pkg.version).toBe(VERSION);
39 expect(pkg.dependencies).toBeUndefined();
40
41 // The door, the guide and the sample are in it, and they are this tree's.
42 const engine = join(import.meta.dir, "engine");
43 for (const name of ["index.ts", GUIDE, SAMPLE])
44 expect(new TextDecoder().decode(at(name)?.data)).toBe(readFileSync(join(engine, name), "utf8"));
45 // Every module the pack counted, and no test.
46 for (const module of archive.modules) expect(at(module)).toBeDefined();
47 expect(read.some(({ name }) => name.endsWith(".test.ts"))).toBe(false);
48 expect(read.map(({ name }) => name)).toEqual(archive.files);
49 expect(archive.unpacked).toBe(read.reduce((sum, { size }) => sum + size, 0));
50 });
51 });
52
53 test("the bytes held are the file written, and the href carries their hash", () => {
54 inTemp((data) => {
55 const archive = buildArchive(archiveDir(data));
56 expect(readFileSync(join(archiveDir(data), archive.file))).toEqual(Buffer.from(archive.bytes));
57 const hex = new Bun.CryptoHasher("sha256").update(archive.bytes).digest("hex");
58 expect(archive.sha256).toBe(hex);
59 expect(archive.href).toBe(`/engine/${NAME}-${VERSION}.${hex.slice(0, 10)}.tar.gz`);
60 });
61 });
62
63 test("the same tree packs to the same bytes, so a hash in the URL is honest", () => {
64 inTemp((data) => {
65 const once = buildArchive(join(data, "a"));
66 const twice = buildArchive(join(data, "b"));
67 expect(twice.sha256).toBe(once.sha256);
68 expect(twice.bytes).toEqual(once.bytes);
69 });
70 });
src/archive.ts added +93
@@ -0,0 +1,93 @@
1 /**
2 * The engine as one file a stranger can take: `scripts/pack.ts` run once at
3 * start-up into this deployment's data directory, tarred by `./tar.ts`, gzipped
4 * by Bun, written beside it, and read back.
5 *
6 * FORGE_APP_DATA/engine/cs-engine-<version>/ ← the pack, as files
7 * FORGE_APP_DATA/engine/cs-engine-<version>.tar.gz ← the same files, as one
8 *
9 * Nothing here writes into the repository and nothing here runs per request:
10 * `src/server.ts` calls `buildArchive` once before it listens and serves the
11 * bytes it hands back. What those bytes are is the pack's own list — every
12 * shipping module, the guide, the sample and a generated `package.json` — under
13 * one top-level directory named for the version, which is what `tar -xzf`
14 * leaves behind.
15 *
16 * The version is `VERSION` from `scripts/pack.ts`, which is the one in
17 * `src/engine/surface.json` that `scripts/surface.test.ts` holds the door
18 * against (#191). There is no second copy of it here.
19 *
20 * The archive is a function of the files: every mtime is 0, owner and group
21 * are 0, and Bun's gzip writes no timestamp. So its sha256 changes exactly when
22 * a byte of the package does — which is what lets the URL carry a slice of that
23 * hash and be cached for a year, when the version alone would not: behaviour
24 * moves under a version that stands still (see the guide's "The version, and
25 * what it promises").
26 */
27
28 import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
29 import { join } from "node:path";
30 import { NAME, pack, VERSION } from "../scripts/pack.ts";
31 import { tar } from "./tar.ts";
32
33 /** The level the archive is gzipped at: it is built once, so it is built small. */
34 export const ARCHIVE_LEVEL = 9;
35
36 /** The directory the pack and its archive go in, under this deployment's data. */
37 export const archiveDir = (data = process.env.FORGE_APP_DATA ?? "./data"): string => join(data, "engine");
38
39 /** The name a download is saved under, and the directory it unpacks to. */
40 export const baseName = (version = VERSION): string => `${NAME}-${version}`;
41
42 /** What start-up made, as the page and the route need it. Read once, held. */
43 export type Archive = {
44 /** The package's name and version, as its `package.json` says them. */
45 name: string;
46 version: string;
47 /** What a browser saves it as: `cs-engine-0.3.0.tar.gz`. */
48 file: string;
49 /** Where it answers: the file name with ten hex digits of its sha256 in it. */
50 href: string;
51 /** The archive, exactly the bytes on disk. */
52 bytes: Uint8Array<ArrayBuffer>;
53 /** Hex sha256 of `bytes`: what a person checks a download against. */
54 sha256: string;
55 /** Modules of the package, as the pack counted them — `index.ts` included. */
56 modules: string[];
57 /** Every path in the archive, top directory and all, in archive order. */
58 files: string[];
59 /** Bytes of the files before tar and gzip. */
60 unpacked: number;
61 };
62
63 /**
64 * Pack into `dir`, tar and gzip what was packed, write the archive, and read it
65 * back. The entries are the files `pack` says it wrote, read from where it wrote
66 * them — so the archive is the pack on disk, not a second opinion of it.
67 */
68 export function buildArchive(dir = archiveDir()): Archive {
69 const base = baseName();
70 mkdirSync(dir, { recursive: true });
71 const packed = pack(join(dir, base));
72 const names = [...packed.files].sort();
73 const entries = names.map((name) => ({
74 name: `${base}/${name}`,
75 data: new Uint8Array(readFileSync(join(packed.dir, name))),
76 }));
77 const file = `${base}.tar.gz`;
78 const path = join(dir, file);
79 writeFileSync(path, Bun.gzipSync(tar(entries), { level: ARCHIVE_LEVEL }));
80 const bytes = new Uint8Array(readFileSync(path));
81 const sha256 = new Bun.CryptoHasher("sha256").update(bytes).digest("hex");
82 return {
83 name: NAME,
84 version: VERSION,
85 file,
86 href: `/engine/${base}.${sha256.slice(0, 10)}.tar.gz`,
87 bytes,
88 sha256,
89 modules: packed.modules,
90 files: entries.map(({ name }) => name),
91 unpacked: entries.reduce((sum, { data }) => sum + data.length, 0),
92 };
93 }
src/door.test.ts added +37
@@ -0,0 +1,37 @@
1 import { expect, test } from "bun:test";
2 import { readFileSync } from "node:fs";
3 import { join } from "node:path";
4 import { isModule, sources } from "../scripts/pack.ts";
5 import { doorModules } from "./door.ts";
6
7 const ENGINE = join(import.meta.dir, "engine");
8 const F = "fr" + "om";
9
10 test("each module once, in first-appearance order, with the comment right above it", () => {
11 const source = [
12 "/** The file's own comment, which is nobody's. */",
13 "",
14 "/** Arithmetic,",
15 " * over two lines. */",
16 `export { a } ${F} "./math.ts";`,
17 `export type { V } ${F} "./math.ts";`,
18 `export { b } ${F} "./trace.ts";`,
19 "/**",
20 " * Types only.",
21 " */",
22 `export type { T } ${F} "./shape.ts";`,
23 ].join("\n");
24 expect(doorModules(source)).toEqual([
25 { module: "math", says: "Arithmetic, over two lines." },
26 { module: "trace", says: "" },
27 { module: "shape", says: "Types only." },
28 ]);
29 });
30
31 test("the door's modules are exactly the modules the pack ships, less the door itself", () => {
32 const door = doorModules(readFileSync(join(ENGINE, "index.ts"), "utf8"));
33 const shipped = sources(ENGINE).filter(isModule).filter((file) => file !== "index.ts");
34 expect(door.map(({ module }) => `${module}.ts`).sort()).toEqual(shipped.sort());
35 // And every one of them is described: the door writes a comment per group.
36 for (const { module, says } of door) expect(`${module}: ${says}`.length).toBeGreaterThan(module.length + 12);
37 });
src/door.ts added +46
@@ -0,0 +1,46 @@
1 /**
2 * The engine's door, read as a list: which modules `src/engine/index.ts`
3 * re-exports, in its order, each with the comment the door writes above it.
4 *
5 * `/engine` shows a stranger what is in the package, and the honest list is the
6 * one the door already keeps — a module and the line or paragraph it says about
7 * itself. Typing that list into a page would make a second copy that goes stale
8 * the first time a module arrives; reading it off the door makes the page move
9 * with it. `boundary.test.ts` pins the *names* the door offers; this is the
10 * *modules* those names come from, which is what a reader scanning the package
11 * wants first.
12 *
13 * Plain text in, plain data out: it parses the source it is handed and reads no
14 * file itself. The server reads the door once at start-up and hands it here.
15 */
16
17 /** One module behind the door: its name (`math`, not `./math.ts`) and what the door says about it. */
18 export type Room = { module: string; says: string };
19
20 /** The text of a `/** … *\/` comment, its stars and line breaks taken out. */
21 const prose = (comment: string): string =>
22 comment
23 .replace(/^\/\*\*|\*\/$/g, "")
24 .split("\n")
25 .map((line) => line.replace(/^\s*\*\s?/, "").trim())
26 .filter((line) => line !== "")
27 .join(" ");
28
29 /**
30 * Every module the door re-exports from, once, in the order it first appears,
31 * with the doc comment immediately above its first `export`. A `type`-only
32 * re-export of a module that already has values is the same module; a module
33 * the door only offers types from is still a module of the package.
34 */
35 export function doorModules(source: string): Room[] {
36 const out: Room[] = [];
37 const seen = new Set<string>();
38 const pattern = /(\/\*\*(?:[^*]|\*(?!\/))*\*\/\s*)?export\s+(?:type\s+)?\{[^}]*\}\s*from\s*["']\.\/([a-z][a-z0-9-]*)\.ts["']/g;
39 for (let m = pattern.exec(source); m !== null; m = pattern.exec(source)) {
40 const module = m[2] as string;
41 if (seen.has(module)) continue;
42 seen.add(module);
43 out.push({ module, says: m[1] === undefined ? "" : prose(m[1].trim()) });
44 }
45 return out;
46 }
src/pages/engine.test.ts added +81
@@ -0,0 +1,81 @@
1 import { afterAll, expect, test } from "bun:test";
2 import { mkdtempSync, readFileSync, rmSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { VERSION } from "../../scripts/pack.ts";
6 import { buildArchive } from "../archive.ts";
7 import { bytes, kB } from "../bytes.ts";
8 import type { Cost } from "../cost.ts";
9 import { doorModules } from "../door.ts";
10 import { enginePage, guideHtml, headingsOf, slug } from "./engine.ts";
11
12 const ENGINE = join(import.meta.dir, "..", "engine");
13 const GUIDE = readFileSync(join(ENGINE, "GUIDE.md"), "utf8");
14 const DOOR = doorModules(readFileSync(join(ENGINE, "index.ts"), "utf8"));
15
16 const DATA = mkdtempSync(join(tmpdir(), "engine-page-"));
17 afterAll(() => rmSync(DATA, { recursive: true, force: true }));
18 const ARCHIVE = buildArchive(DATA);
19
20 /** Figures nobody measured, so a digit on the page is one this test handed it. */
21 const COST: Cost = { script: { raw: 12_345, gzip: 4_321 }, shooter: { raw: 98_765, gzip: 34_567 }, game: 432, wiring: 219 };
22
23 const render = (guide = GUIDE): string =>
24 enginePage({ env: "local", pull: undefined, commit: "abc1234", ssh: undefined, archive: ARCHIVE, cost: COST, door: DOOR, guide });
25
26 test("a heading becomes an anchor to itself; a # inside a code block does not", () => {
27 const guide = "# Title\n\ntext\n\n## Two & <three>\n\n```ts\n# not a heading\n```\n\n### math\n\n## Two & <three>\n";
28 const headings = headingsOf(guide);
29 expect(headings).toEqual([
30 { level: 1, text: "Title", id: "guide-title" },
31 { level: 2, text: "Two & <three>", id: "guide-two-three" },
32 { level: 3, text: "math", id: "guide-math" },
33 { level: 2, text: "Two & <three>", id: "guide-two-three-2" },
34 ]);
35 const html = guideHtml(guide);
36 expect(html).toContain(`<span class="h" id="guide-two-three"><a href="#guide-two-three">## Two &amp; &lt;three&gt;</a></span>`);
37 expect(html).toContain(`id="guide-two-three-2"`);
38 expect(html).toContain("\n# not a heading\n");
39 expect(slug("1. An entry")).toBe("1-an-entry");
40 expect(slug("???")).toBe("section");
41 });
42
43 test("the guide is on the page whole, escaped, with every heading an anchor in the contents", () => {
44 const body = render();
45 const headings = headingsOf(GUIDE);
46 expect(headings.length).toBeGreaterThan(30);
47 for (const { id, level } of headings) {
48 expect(body).toContain(`<span class="h" id="${id}">`);
49 if (level > 1) expect(body).toContain(`<a href="#${id}">`);
50 }
51 expect(body).toContain(`id="guide-what-a-game-brings"`);
52 // Escaped: a guide line with markup in it cannot become markup.
53 const hostile = render("# Hi\n\n<script>alert(1)</script> & \"q\"\n");
54 expect(hostile).not.toContain("<script>alert");
55 expect(hostile).toContain("&lt;script&gt;alert(1)&lt;/script&gt; &amp; &quot;q&quot;");
56 // And the page is honest that it is text, not a rendering.
57 expect(body).toContain("It is not rendered as markdown");
58 });
59
60 test("the page lists exactly the modules the door exports, each linked to its section of the guide", () => {
61 const body = render();
62 const listed = [...body.matchAll(/<li data-module="([^"]+)">/g)].map((m) => m[1]);
63 const exported = ARCHIVE.modules.filter((file) => file !== "index.ts").map((file) => file.replace(/\.ts$/, ""));
64 expect(listed.sort()).toEqual(exported.sort());
65 expect(body).toContain(`<li data-module="math"><a href="#guide-math"><code>math</code></a> — Vectors, matrices`);
66 });
67
68 test("the package is offered as it is: its version, its href, its size and hash, and the sizes handed in", () => {
69 const body = render();
70 expect(body).toContain(`<b>cs-engine ${VERSION}</b>`);
71 expect(body).toContain(`href="${ARCHIVE.href}" download="cs-engine-${VERSION}.tar.gz"`);
72 expect(body).toContain(`${bytes(ARCHIVE.bytes.byteLength)} bytes (${kB(ARCHIVE.bytes.byteLength)}) gzipped`);
73 expect(body).toContain(ARCHIVE.sha256);
74 expect(body).toContain(`${ARCHIVE.modules.length} TypeScript modules behind one door`);
75 expect(body).toContain(`tar -xzf cs-engine-${VERSION}.tar.gz\nbun build cs-engine-${VERSION}/example.ts`);
76 // The cost story is the `Cost` it was handed, formatted by `bytes.ts`.
77 for (const said of ["432 lines", "219 lines", "12,345 B", kB(4_321), "98,765 B", kB(34_567)]) expect(body).toContain(said);
78 expect(body).toContain("abc1234");
79 // Nothing runs on it.
80 expect(body).not.toContain("<script");
81 });
src/pages/engine.ts added +245
@@ -0,0 +1,245 @@
1 /**
2 * The engine's page: what the package is, the one file to take it as, what a
3 * game costs on it, the modules behind its door, and its guide — at `/engine`,
4 * for somebody who has not cloned this repository and should not have to.
5 *
6 * Nothing on it is typed that could be read instead. The version and the
7 * archive's size and hash are what `src/archive.ts` built at start-up; the
8 * sizes a game costs are `src/cost.ts`'s, the same `Cost` `/example` prints;
9 * the modules are `src/door.ts` reading `src/engine/index.ts`; and the guide is
10 * `src/engine/GUIDE.md`, the file that ships in the archive. The server reads
11 * all of those once, renders this once, and serves the bytes: nothing here
12 * touches the filesystem, and nothing here runs per request.
13 *
14 * The guide is shown as what it is — its own text, escaped and preformatted —
15 * with an anchor on every heading and a contents list over it. There is no
16 * markdown renderer, because that is a dependency or a second parser to own,
17 * and the page says so rather than pretending the text is a rendering.
18 */
19
20 import { ARCHIVE_LEVEL, type Archive } from "../archive.ts";
21 import { bytes, kB } from "../bytes.ts";
22 import { type Cost, GZIP_LEVEL } from "../cost.ts";
23 import type { Room } from "../door.ts";
24 import { navFor } from "./games.ts";
25 import { escape, page, type PageOptions as ShellOptions } from "./shell.ts";
26
27 /** Where the page answers. The example links it; `src/server.ts` routes it. */
28 export const ENGINE_HREF = "/engine";
29
30 /** Every heading's id starts with this, so no heading can take an id the shell uses. */
31 const ID_PREFIX = "guide-";
32
33 /**
34 * This page's own CSS. The guide is 80-column text and a phone is not, so the
35 * preformatted block wraps instead of scrolling the page sideways; the contents
36 * run in columns so forty links are not forty screens; and every link a thumb
37 * has to hit is 44px tall. On paper the chrome goes, the guide stays, and the
38 * colours become ink.
39 */
40 export const STYLE = `
41 #take .download { display:flex; flex-wrap:wrap; gap:.5rem 1rem; align-items:center; }
42 #take a.get { display:inline-flex; align-items:center; min-height:44px; padding:.6rem 1.1rem; border-radius:.45rem;
43 background:#2563eb; color:#fff; font-weight:600; text-decoration:none; }
44 #take a.get:focus-visible { outline:2px solid var(--ct); outline-offset:2px; }
45 #take .download span, #take .hash, #guide .said { color:var(--dim); font-size:.86rem; }
46 #take .hash code { overflow-wrap:anywhere; }
47 pre { background:var(--panel); border:1px solid var(--line); border-radius:.45rem; padding:.75rem;
48 white-space:pre-wrap; overflow-wrap:anywhere; font-size:.82rem; line-height:1.45; }
49 #engine-cost td.n, #engine-cost th.n { text-align:right; white-space:nowrap; }
50 ul.door { list-style:none; padding:0; margin:.5rem 0; font-size:.88rem; }
51 ul.door li { padding:.35rem 0; border-bottom:1px solid var(--line); }
52 ul.door li > a, ul.door li > code { display:inline-flex; align-items:center; min-height:44px; min-width:44px; }
53 ol.contents { columns:15rem; margin:.5rem 0; padding-left:1.2rem; font-size:.88rem; }
54 ol.contents li { break-inside:avoid; }
55 ol.contents li.l3 { margin-left:1rem; list-style:circle; }
56 ol.contents a { display:inline-flex; align-items:center; min-height:44px; }
57 a { color:var(--ct); }
58 pre.guide .h { color:var(--t); font-weight:700; }
59 pre.guide .h a { color:inherit; text-decoration:none; }
60 pre.guide .h a:hover, pre.guide .h a:focus-visible { text-decoration:underline; }
61 @media print {
62 header nav, header .env, header small.commit, footer, p.terminal, #take .download a.get { display:none; }
63 body { background:#fff; color:#000; }
64 pre { background:#fff; border:0; padding:0; font-size:8pt; }
65 pre.guide .h, a, #take .download span, #take .hash, #guide .said { color:#000; }
66 ol.contents a { min-height:0; }
67 }
68 `;
69
70 /** One heading of the guide: its depth, its words, and the id its anchor carries. */
71 export type Heading = { level: number; text: string; id: string };
72
73 /** A heading's words as an id: lower-case letters, digits and single dashes. */
74 export const slug = (text: string): string =>
75 text.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "") || "section";
76
77 /** A line that opens or closes a fenced code block, where a `#` is code, not a heading. */
78 const FENCE = /^\s*```/;
79
80 /** `## What a game brings` → level 2 and its words; anything else is not a heading. */
81 const HEADING = /^(#{1,6})\s+(.+?)\s*#*\s*$/;
82
83 /**
84 * Every heading of the guide, outside its code blocks, in order, each with an
85 * id nobody else on the page has. Two headings with the same words get `-2`,
86 * `-3` after the second and third, the way every renderer does it.
87 */
88 export function headingsOf(guide: string): Heading[] {
89 const out: Heading[] = [];
90 const used = new Map<string, number>();
91 let fenced = false;
92 for (const line of guide.split("\n")) {
93 if (FENCE.test(line)) fenced = !fenced;
94 const m = fenced ? null : HEADING.exec(line);
95 if (m === null) continue;
96 const text = m[2] as string;
97 const base = ID_PREFIX + slug(text);
98 const seen = (used.get(base) ?? 0) + 1;
99 used.set(base, seen);
100 out.push({ level: (m[1] as string).length, text, id: seen === 1 ? base : `${base}-${seen}` });
101 }
102 return out;
103 }
104
105 /**
106 * The guide as one preformatted block: every line its own text, escaped, and
107 * every heading line wrapped in an anchor to itself so a link to one section
108 * is a thing a person can copy.
109 */
110 export function guideHtml(guide: string, headings = headingsOf(guide)): string {
111 let next = 0;
112 let fenced = false;
113 const lines = guide.replace(/\n$/, "").split("\n").map((line) => {
114 if (FENCE.test(line)) fenced = !fenced;
115 const heading = !fenced && HEADING.test(line) ? headings[next++] : undefined;
116 if (heading === undefined) return escape(line);
117 return `<span class="h" id="${escape(heading.id)}"><a href="#${escape(heading.id)}">${escape(line)}</a></span>`;
118 });
119 return `<pre class="guide">${lines.join("\n")}</pre>`;
120 }
121
122 /** The contents: a link to every heading below the title, a sub-heading indented under its section. */
123 export const contentsHtml = (headings: readonly Heading[]): string =>
124 `<nav aria-label="The guide's contents"><ol class="contents">${headings
125 .filter(({ level }) => level > 1)
126 .map(({ level, text, id }) => `<li class="l${Math.min(level, 3)}"><a href="#${escape(id)}">${escape(text)}</a></li>`)
127 .join("")}</ol></nav>`;
128
129 /**
130 * The modules behind the door, each as the door describes it — and linked to
131 * the guide's section on it when the guide has a heading of that name, which
132 * today is every one of them.
133 */
134 export function doorHtml(door: readonly Room[], headings: readonly Heading[]): string {
135 const ids = new Map(headings.map(({ text, id }) => [text, id]));
136 return `<ul class="door">${door
137 .map(({ module, says }) => {
138 const id = ids.get(module);
139 const name = id === undefined
140 ? `<code>${escape(module)}</code>`
141 : `<a href="#${escape(id)}"><code>${escape(module)}</code></a>`;
142 return `<li data-module="${escape(module)}">${name}${says === "" ? "" : ` — ${escape(says)}`}</li>`;
143 })
144 .join("")}</ul>`;
145 }
146
147 /** The package, what it holds and the one file to take it as. */
148 export function takeHtml(archive: Archive, commit: string): string {
149 const { name, version, file, href, sha256, modules, files, unpacked } = archive;
150 const top = `${name}-${version}`;
151 return ` <section id="take">
152 <h2>Take the engine</h2>
153 <p><b>${escape(name)} ${escape(version)}</b> is the engine every game on this server is built on, as a
154 package: ${modules.length} TypeScript modules behind one door, <code>index.ts</code>, and no
155 dependencies — its <code>package.json</code> has no dependencies key at all, because there is
156 nothing to install. It ships as source: no build step, no compiled copy, no types to fetch.
157 Beside the modules are the guide, <code>GUIDE.md</code>, which is the whole of the text below, and
158 the guide's game, <code>example.ts</code>, the sample you build first.</p>
159 <p class="download"><a class="get" href="${escape(href)}" download="${escape(file)}">Download ${escape(file)}</a>
160 <span>${bytes(archive.bytes.byteLength)} bytes (${kB(archive.bytes.byteLength)}) gzipped ·
161 ${files.length} files, ${bytes(unpacked)} bytes unpacked</span></p>
162 <p class="hash">sha256 <code>${escape(sha256)}</code></p>
163 <p>Start here, with <a href="https://bun.sh">Bun</a> and nothing else:</p>
164 <pre>tar -xzf ${escape(file)}
165 bun build ${escape(top)}/example.ts --outdir out</pre>
166 <p>The first line leaves one directory, <code>${escape(top)}/</code>. The second builds the sample against
167 the door in it, and nothing else on your disk: this repository's own tests pack the engine the same
168 way and assert that every module the bundler read came out of the copy. The version is the engine's
169 surface — it moves when a name the door offers arrives, leaves or is renamed — and not its
170 behaviour, so the commit is the real identity of a pack${commit === "" ? "" : `; this one was packed
171 at <code>${escape(commit)}</code>`}. The archive is written by this server when it starts, from the
172 tree it runs, and the same tree always packs to the same bytes.</p>
173 </section>`;
174 }
175
176 /**
177 * The sizes the guide's cost story is made of, as a table: the lines a game
178 * brings, the lines of wiring the example needed, the bytes of its bundle
179 * against the shooter's, and the bytes of the package itself. Every figure is
180 * handed in; `../bytes.ts` writes each one.
181 */
182 export function costHtml(cost: Cost, archive: Archive): string {
183 const row = (what: string, raw: string, gzip: string): string =>
184 `<tr><td>${what}</td><td class="n">${raw}</td><td class="n">${gzip}</td></tr>`;
185 return ` <section id="engine-cost">
186 <h2>What a game costs on it</h2>
187 <p>Measured by this server when it started, from the bundles it built and the sources it runs — the
188 same figures <a href="/example">the example</a> writes its two paragraphs out of. The bundles are
189 gzipped at level ${GZIP_LEVEL} and the package at level ${ARCHIVE_LEVEL}, and a kB is 1,024 bytes.</p>
190 <table>
191 <thead><tr><th></th><th class="n">as written</th><th class="n">gzipped</th></tr></thead>
192 <tbody>
193 ${row("the guide's game, <code>src/engine/example.ts</code>", `${bytes(cost.game)} lines`, "—")}
194 ${row("the wiring that gave it a page", `${bytes(cost.wiring)} lines`, "—")}
195 ${row("the example's bundle", `${bytes(cost.script.raw)} B`, kB(cost.script.gzip))}
196 ${row("the shooter's bundle, for scale", `${bytes(cost.shooter.raw)} B`, kB(cost.shooter.gzip))}
197 ${row("this package, whole", `${bytes(archive.unpacked)} B`, kB(archive.bytes.byteLength))}
198 </tbody>
199 </table>
200 </section>`;
201 }
202
203 /** What the shell needs that this page cannot know, and everything it prints. */
204 export type PageOptions = Pick<ShellOptions, "env" | "pull" | "commit" | "ssh"> & {
205 archive: Archive;
206 cost: Cost;
207 door: readonly Room[];
208 guide: string;
209 };
210
211 /**
212 * The page, whole. Like `/weapons` it hands the shell no script — nothing on it
213 * runs — and its nav marks nothing current, because it is not a game.
214 */
215 export function enginePage({ archive, cost, door, guide, ...options }: PageOptions): string {
216 const headings = headingsOf(guide);
217 return page({
218 ...options,
219 title: `engine — ${archive.name} ${archive.version}, its guide and the package`,
220 description:
221 `The engine every game here runs on, as one download: ${archive.modules.length} TypeScript modules behind one door, no dependencies, its guide and a sample game.`,
222 heading: "engine",
223 nav: navFor(ENGINE_HREF),
224 terminalLead: "The games' boards in a terminal:",
225 footer: "no database, no cookie: this page is the package and its guide",
226 extraStyle: STYLE,
227 stageHtml: "",
228 sectionsHtml: `${takeHtml(archive, options.commit)}
229 ${costHtml(cost, archive)}
230 <section id="door">
231 <h2>Behind the door</h2>
232 <p>Every module <code>index.ts</code> re-exports, in its order, with what the door says about it. A game
233 imports from the door and from nowhere deeper.</p>
234 ${doorHtml(door, headings)}
235 </section>
236 <section id="guide">
237 <h2>The guide</h2>
238 <p class="said"><code>GUIDE.md</code> as it ships in the archive: its own text, escaped and
239 preformatted, with an anchor on every heading. It is not rendered as markdown — there is no
240 renderer here and no dependency for one; a prettier reading is a later change, not this one.</p>
241 ${contentsHtml(headings)}
242 ${guideHtml(guide, headings)}
243 </section>`,
244 });
245 }
src/pages/example.ts +3
@@ -13,6 +13,7 @@
1313 import { bytes, kB } from "../bytes.ts";
1414 import { GZIP_LEVEL as WIRE_LEVEL } from "../compress.ts";
1515 import { type Cost, GZIP_LEVEL } from "../cost.ts";
16 import { ENGINE_HREF } from "./engine.ts";
1617 import { navFor, noscriptFor, othersHtml } from "./games.ts";
1718 import { page, type PageOptions as ShellOptions, stage } from "./shell.ts";
1819
@@ -105,6 +106,8 @@ export function examplePage(options: PageOptions): string {
105106 sectionsHtml: ` <section>
106107 <h2>What this cost</h2>
107108 ${costHtml(options.cost)}
109 <p>The engine is yours to take: <a href="${ENGINE_HREF}">the package, its guide and every module behind
110 its door</a>, as one download.</p>
108111 <p>Nothing on this page is stored: no endpoint, no table and no cookie. The leaderboard, the
109112 database and the terminal belong to the games next door: ${othersHtml("/example")}</p>
110113 </section>`,
src/server.test.ts +69 −3
@@ -1,5 +1,5 @@
11 import { expect, test } from "bun:test";
2 import { existsSync, mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs";
2 import { existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
33 import { tmpdir } from "node:os";
44 import { join } from "node:path";
55 import { bundle } from "../scripts/build.ts";
@@ -13,6 +13,10 @@ import { LONGEST_PASTE } from "./matchlink.ts";
1313 import { NOTHING_POSTED } from "./client/pending.ts";
1414 import { BAD_ID, BAD_MAP, UNDRAWN } from "./pages/game.ts";
1515 import { WEAPONS_HREF, WEAPONS_SECTION } from "./pages/weapons.ts";
16 import { ENGINE_HREF } from "./pages/engine.ts";
17 import { archiveDir } from "./archive.ts";
18 import { untar } from "./tar.ts";
19 import { NAME as PACKAGE, VERSION } from "../scripts/pack.ts";
1620 import { WEAPONS } from "./client/weapons.ts";
1721 import { endpoints, escape } from "./pages/shell.ts";
1822 import { localPlan, parsePlan } from "./plan/plan.ts";
@@ -578,6 +582,61 @@ test("the reference stands on the same security floor as the games, and HEAD ans
578582 expect((await fetch(base + "/weapons/ak47")).status).toBe(404);
579583 });
580584
585 test("the engine's page answers on the floor, links the archive, and the example links it", async () => {
586 const [game, page] = await Promise.all([fetch(base + "/"), fetch(base + ENGINE_HREF)]);
587 expect(page.status).toBe(200);
588 expect(floorOf(page)).toEqual(floorOf(game));
589 const body = await page.text();
590 expect(body).toContain("<h1>engine</h1>");
591 expect(body).toContain(`<b>${PACKAGE} ${VERSION}</b>`);
592 expect(body).not.toContain("<script");
593 expect(page.headers.get("set-cookie")).toBeNull();
594 const head = await server.fetch(new Request(base + ENGINE_HREF, { method: "HEAD" }));
595 expect(head.status).toBe(200);
596 expect(await head.text()).toBe("");
597 expect(await (await fetch(base + "/example")).text()).toContain(`href="${ENGINE_HREF}"`);
598 });
599
600 test("the archive route answers with the bytes this app wrote under FORGE_APP_DATA, their length, and a year", async () => {
601 const body = await (await fetch(base + ENGINE_HREF)).text();
602 const href = new RegExp(`href="(/engine/${PACKAGE}-${VERSION.replaceAll(".", "\\.")}\\.[0-9a-f]{10}\\.tar\\.gz)"`).exec(body)?.[1];
603 expect(href).toBeDefined();
604 const file = join(archiveDir(), `${PACKAGE}-${VERSION}.tar.gz`);
605 const onDisk = new Uint8Array(readFileSync(file));
606 // Asked for gzip or not, it is the file as written: it is gzip already.
607 const response = await fetch(base + (href as string), { headers: { "accept-encoding": "gzip" }, decompress: false });
608 expect(response.status).toBe(200);
609 expect(response.headers.get("content-type")).toBe("application/gzip");
610 expect(response.headers.get("content-encoding")).toBeNull();
611 expect(response.headers.get("content-length")).toBe(String(onDisk.byteLength));
612 expect(response.headers.get("cache-control")).toBe("public, max-age=31536000, immutable");
613 expect(response.headers.get("content-disposition")).toBe(`attachment; filename="${PACKAGE}-${VERSION}.tar.gz"`);
614 expect(response.headers.get("x-content-type-options")).toBe("nosniff");
615 expect(response.headers.get("referrer-policy")).toBe("strict-origin-when-cross-origin");
616 expect(response.headers.get("content-security-policy")).toContain("default-src 'self'");
617 const sent = new Uint8Array(await response.arrayBuffer());
618 expect(sent).toEqual(onDisk);
619 // The href's hash is the bytes' hash.
620 const hex = new Bun.CryptoHasher("sha256").update(sent).digest("hex");
621 expect(href).toContain(`.${hex.slice(0, 10)}.tar.gz`);
622 expect(body).toContain(hex);
623 // And it unpacks to the engine, at its version.
624 const pkg = untar(Bun.gunzipSync(sent)).find(({ name }) => name === `${PACKAGE}-${VERSION}/package.json`);
625 expect(JSON.parse(new TextDecoder().decode(pkg?.data)).version).toBe(VERSION);
626
627 const head = await server.fetch(new Request(base + (href as string), { method: "HEAD" }));
628 expect(head.status).toBe(200);
629 expect(head.headers.get("content-length")).toBe(String(onDisk.byteLength));
630 expect(await head.text()).toBe("");
631 });
632
633 test("no other name under /engine is a file: the one href answers and the rest is the 404", async () => {
634 for (const path of [
635 `/engine/${PACKAGE}-${VERSION}.tar.gz`, `/engine/${PACKAGE}-${VERSION}/package.json`,
636 "/engine/", "/engine/..%2f..%2fsrc%2fserver.ts", `/engine/${PACKAGE}-0.0.0.0000000000.tar.gz`,
637 ]) expect((await fetch(base + path)).status).toBe(404);
638 });
639
581640 test("the shooter links the reference instead of carrying it", async () => {
582641 const body = await (await fetch(base + "/")).text();
583642 expect(body).toContain(`href="/weapons"`);
@@ -651,7 +710,7 @@ test("the policy names each page's own style block, hashed from the bytes the pa
651710 const seen = new Set<string>();
652711 // Every game, and the reference, which is not one: it is a read route on the
653712 // same terms, so its block is hashed into its own policy like theirs.
654 for (const href of [...GAMES.map((game) => game.href), WEAPONS_HREF]) {
713 for (const href of [...GAMES.map((game) => game.href), WEAPONS_HREF, ENGINE_HREF]) {
655714 const response = await fetch(base + href);
656715 const block = styleBlock(await response.text());
657716 expect(block.length).toBeGreaterThan(500);
@@ -1178,8 +1237,15 @@ test("every file in static/ is gzipped once, at start-up, and held beside its by
11781237 * `/` is measured on this file's database, which by now holds every match the
11791238 * tests above posted, so it is the page at its heaviest rather than at its
11801239 * emptiest: 8,830 gzipped against 26,188 when this was written.
1240 *
1241 * `/engine` is the exception in size and not in kind: it carries the whole
1242 * guide, 78 kB of text, because reading it without cloning is the point of the
1243 * page. Measured 2026-09-23 at 37,166 gzipped against 104,653 raw; the budget
1244 * is that plus room for the guide to grow a section or two.
11811245 */
1182 const BUDGET: Record<string, number> = { "/": 10500, "/weapons": 5600, "/range": 7500, "/example": 5500 };
1246 const BUDGET: Record<string, number> = {
1247 "/": 10500, "/weapons": 5600, "/range": 7500, "/example": 5500, "/engine": 40000,
1248 };
11831249
11841250 test("a first visit stays inside its byte budget, compressed", async () => {
11851251 for (const [path, budget] of Object.entries(BUDGET)) {
src/server.ts +50 −1
@@ -48,6 +48,8 @@
4848 import { existsSync, readdirSync, readFileSync } from "node:fs";
4949 import { join } from "node:path";
5050 import { assetOf, bundleOf } from "../scripts/build.ts";
51 import { ENGINE } from "../scripts/pack.ts";
52 import { archiveDir, buildArchive } from "./archive.ts";
5153 import { bind } from "./bind.ts";
5254 import { buildBundles, bundlesDir, buildSummary } from "./bundles.ts";
5355 import { kB } from "./bytes.ts";
@@ -57,6 +59,8 @@ import { openDatabase, saveMatch, saveRun } from "./db.ts";
5759 import { admit, type Gate, playerOf, readBody, refused } from "./gate.ts";
5860 import { GAMES } from "./games.ts";
5961 import { createLimiter } from "./limit.ts";
62 import { doorModules } from "./door.ts";
63 import { ENGINE_HREF, enginePage, STYLE as ENGINE_STYLE } from "./pages/engine.ts";
6064 import { examplePage } from "./pages/example.ts";
6165 import { type Asked, gamePage, STYLE as GAME_STYLE, statsHtml } from "./pages/game.ts";
6266 import { mapFrom } from "./mapname.ts";
@@ -629,6 +633,50 @@ const REFERENCE: Ready = {
629633 };
630634 READ.set(WEAPONS_HREF, () => REFERENCE);
631635
636 /**
637 * The engine, as a package a visitor can take: `src/archive.ts` packs
638 * `src/engine/` into `FORGE_APP_DATA/engine`, tars and gzips it, and hands back
639 * the bytes it wrote — once, here, before this listens. It is a file copy and a
640 * gzip of about 340 kB, measured at 30 ms on a laptop and printed in the boot
641 * log; a request never builds an archive and never opens a file.
642 *
643 * Two exact paths answer for it and nothing else does: `/engine`, the page,
644 * rendered and weighed once like `/weapons`, and the archive at the one href
645 * `buildArchive` named — the version and a slice of the archive's own sha256 —
646 * which is why it may be cached for a year. No part of either path is read
647 * from the request: a name that is not that href is the 404.
648 */
649 const archivedFrom = Bun.nanoseconds();
650 const ARCHIVE = buildArchive(archiveDir());
651 const archivedIn = (Bun.nanoseconds() - archivedFrom) / 1e6;
652 const ENGINE_SECURITY = documentFloor(ENGINE_STYLE);
653 const ENGINE_PAGE: Ready = {
654 body: weighText(
655 enginePage({
656 env, pull, commit, ssh, archive: ARCHIVE, cost: exampleCost,
657 door: doorModules(readFileSync(join(ENGINE, "index.ts"), "utf8")),
658 guide: readFileSync(join(ENGINE, "GUIDE.md"), "utf8"),
659 }),
660 HTML_TYPE,
661 ),
662 status: 200,
663 headers: { "content-type": HTML_TYPE, ...ENGINE_SECURITY },
664 };
665 READ.set(ENGINE_HREF, () => ENGINE_PAGE);
666 // Already gzip, so never gzipped again: `compressible` says no to this type,
667 // and the bytes go out as they are, with their length and the name to save them as.
668 const DOWNLOAD: Ready = {
669 body: { raw: ARCHIVE.bytes, gzip: null },
670 status: 200,
671 headers: {
672 "content-type": "application/gzip",
673 "content-length": String(ARCHIVE.bytes.byteLength),
674 "content-disposition": `attachment; filename="${ARCHIVE.file}"`,
675 ...IMMUTABLE,
676 },
677 };
678 READ.set(ARCHIVE.href, () => DOWNLOAD);
679
632680 // The two pages no game owns, rendered through the same shell every game is
633681 // served inside: `src/pages/miss.ts` writes both. A wrong path is a dead end
634682 // unless it says where the games are, so the 404 names every one of them and
@@ -930,7 +978,8 @@ if (listen.unusable !== undefined)
930978
931979 console.log(
932980 `cs listening on http://${server.hostname}:${server.port} (${env}) serving ${[...scripts.values()].join(", ")}` +
933 ` — ${buildSummary(built, builtIn, BUNDLES)} — ${staticSummary()} — up to ${ROOM_CAP} rooms on ${SOCKET_PATH}` +
981 ` — ${buildSummary(built, builtIn, BUNDLES)} — ${staticSummary()}` +
982 ` — ${ARCHIVE.file} packed, ${ARCHIVE.bytes.byteLength} B, in ${archivedIn.toFixed(1)} ms — up to ${ROOM_CAP} rooms on ${SOCKET_PATH}` +
934983 `${identityKey ? "" : " — no FORGE_APP_IDENTITY_KEY: every visitor plays as a guest"}` +
935984 `${planner ? "" : " — no TYPESAFE_API_KEY: the bots plan their rounds locally"}`,
936985 );
src/tar.test.ts added +108
@@ -0,0 +1,108 @@
1 import { expect, test } from "bun:test";
2 import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { BLOCK, checksum, header, splitName, tar, untar } from "./tar.ts";
6
7 const bytes = (s: string): Uint8Array => new TextEncoder().encode(s);
8 const ascii = (b: Uint8Array): string => new TextDecoder().decode(b);
9
10 test("a header is 512 bytes laid out the way ustar says, fields at their offsets", () => {
11 const block = header("pkg/index.ts", 1234, 0o17);
12 expect(block.length).toBe(BLOCK);
13 expect(ascii(block.subarray(0, 12))).toBe("pkg/index.ts");
14 expect(block[12]).toBe(0);
15 expect(ascii(block.subarray(100, 108))).toBe("0000644\0");
16 expect(ascii(block.subarray(108, 116))).toBe("0000000\0");
17 expect(ascii(block.subarray(116, 124))).toBe("0000000\0");
18 // 1234 is 2322 in octal, padded to eleven digits and a NUL.
19 expect(ascii(block.subarray(124, 136))).toBe("00000002322\0");
20 expect(ascii(block.subarray(136, 148))).toBe("00000000017\0");
21 expect(String.fromCharCode(block[156] as number)).toBe("0");
22 expect(ascii(block.subarray(257, 263))).toBe("ustar\0");
23 expect(ascii(block.subarray(263, 265))).toBe("00");
24 expect(block.subarray(345, 512).every((b) => b === 0)).toBe(true);
25 });
26
27 test("the checksum is the byte sum with its own field read as spaces, written as six octal digits, NUL, space", () => {
28 const block = header("a.ts", 0);
29 // By hand: every non-zero byte of the header, plus eight spaces for the field.
30 let sum = 8 * 0x20;
31 for (let i = 0; i < BLOCK; i++) if (i < 148 || i >= 156) sum += block[i] as number;
32 expect(checksum(block)).toBe(sum);
33 expect(ascii(block.subarray(148, 156))).toBe(`${sum.toString(8).padStart(6, "0")}\0 `);
34 });
35
36 test("data is padded to whole blocks and the archive ends in two zero blocks", () => {
37 const archive = tar([
38 { name: "one.txt", data: bytes("x") },
39 { name: "empty.txt", data: new Uint8Array(0) },
40 { name: "exact.bin", data: new Uint8Array(BLOCK).fill(7) },
41 ]);
42 // one: header + 1 data block; empty: header only; exact: header + 1; end: 2.
43 expect(archive.length).toBe(BLOCK * (2 + 1 + 2 + 2));
44 expect(archive.length % BLOCK).toBe(0);
45 expect(archive.subarray(BLOCK + 1, 2 * BLOCK).every((b) => b === 0)).toBe(true);
46 expect(archive.subarray(archive.length - 2 * BLOCK).every((b) => b === 0)).toBe(true);
47 });
48
49 test("an archive reads back to the files that went in, with every checksum checked", () => {
50 const long = `${"d".repeat(60)}/${"e".repeat(60)}/${"f".repeat(40)}.ts`;
51 const entries = [
52 { name: "cs-engine-1.2.3/package.json", data: bytes('{"version":"1.2.3"}\n') },
53 { name: "cs-engine-1.2.3/sub/deep.ts", data: bytes("export const x = 1;\n".repeat(50)) },
54 { name: long, data: bytes("long") },
55 ];
56 const back = untar(tar(entries, 1_700_000_000));
57 expect(back.map(({ name }) => name)).toEqual(entries.map(({ name }) => name));
58 for (const [i, read] of back.entries()) {
59 expect(read.data).toEqual((entries[i] as (typeof entries)[number]).data);
60 expect(read.mode).toBe(0o644);
61 expect(read.mtime).toBe(1_700_000_000);
62 }
63 // A flipped byte anywhere in a header is caught by its checksum.
64 const broken = tar(entries);
65 broken[3] = (broken[3] as number) ^ 1;
66 expect(() => untar(broken)).toThrow(/checksum/);
67 });
68
69 test("a long path is split into prefix and name at a slash; one that cannot be is refused", () => {
70 expect(splitName("a/b.ts")).toEqual({ name: "a/b.ts", prefix: "" });
71 const long = `${"p".repeat(120)}/b.ts`;
72 expect(splitName(long)).toEqual({ name: "b.ts", prefix: "p".repeat(120) });
73 expect(() => splitName("x".repeat(101))).toThrow(/too long/);
74 expect(() => splitName(`${"p".repeat(156)}/b.ts`)).toThrow(/too long/);
75 });
76
77 test("a path that could unpack outside where it is unpacked, or is not ASCII, is refused", () => {
78 for (const bad of ["/etc/passwd", "../up.ts", "a/../../b", "a//b", "a/", "./a", "ünï.ts", "a\nb", ""])
79 expect(() => tar([{ name: bad, data: new Uint8Array(0) }])).toThrow();
80 expect(() => tar([{ name: "a", data: bytes("1") }, { name: "a", data: bytes("2") }])).toThrow(/twice/);
81 });
82
83 test("the same files make the same bytes: an archive is a function of its entries", () => {
84 const entries = [{ name: "a.ts", data: bytes("a") }];
85 expect(tar(entries)).toEqual(tar(entries));
86 });
87
88 test("the system's tar, where there is one, unpacks it gzipped to the same files", () => {
89 const system = Bun.which("tar");
90 if (system === null) return;
91 const dir = mkdtempSync(join(tmpdir(), "tar-"));
92 try {
93 const long = `pkg/${"d".repeat(80)}/${"n".repeat(90)}.ts`;
94 const archive = Bun.gzipSync(tar([
95 { name: "pkg/index.ts", data: bytes("export {};\n") },
96 { name: long, data: bytes("x".repeat(1000)) },
97 ]));
98 writeFileSync(join(dir, "pkg.tar.gz"), archive);
99 // Fixed arguments, no text from anywhere else: this is the reader a stranger uses.
100 const run = Bun.spawnSync([system, "-xzf", "pkg.tar.gz"], { cwd: dir });
101 expect(run.stderr.toString()).toBe("");
102 expect(run.exitCode).toBe(0);
103 expect(readFileSync(join(dir, "pkg", "index.ts"), "utf8")).toBe("export {};\n");
104 expect(readFileSync(join(dir, long), "utf8")).toBe("x".repeat(1000));
105 } finally {
106 rmSync(dir, { recursive: true, force: true });
107 }
108 });
src/tar.ts added +186
@@ -0,0 +1,186 @@
1 /**
2 * A tar archive, written by hand: the one format `tar -xf` has read since
3 * POSIX.1-1988 (ustar), for regular files and nothing else.
4 *
5 * It exists because `/engine` hands a stranger the package as one file, and a
6 * tarball is the one file every machine with a shell can open. Bun gives the
7 * gzip (`Bun.gzipSync`); the tar is a 512-byte header per file, the bytes, zero
8 * padding to the next 512, and two zero blocks at the end. That is short enough
9 * to own, and a dependency for it would be more code than this.
10 *
11 * The header, field by field (POSIX.1-2017, pax "ustar Interchange Format"):
12 *
13 * offset size field
14 * 0 100 name — the path, or its last part when `prefix` is used
15 * 100 8 mode — octal, NUL-terminated: 0000644
16 * 108 8 uid — octal: 0
17 * 116 8 gid — octal: 0
18 * 124 12 size — octal bytes of data, NUL-terminated
19 * 136 12 mtime — octal seconds since the epoch
20 * 148 8 chksum — octal sum of all 512 header bytes, this field read as spaces
21 * 156 1 typeflag — '0', a regular file
22 * 157 100 linkname — empty
23 * 257 6 magic — "ustar\0"
24 * 263 2 version — "00"
25 * 265 32 uname, 297 32 gname — empty
26 * 329 8 devmajor, 337 8 devminor — empty
27 * 345 155 prefix — the leading directories of a name over 100 bytes
28 * 500 12 (padding)
29 *
30 * What it refuses rather than guesses: a name that is not printable ASCII, has
31 * a `..` or an absolute path, or does not fit the two fields; and a file of 8 GiB
32 * or more, which is the 11-digit octal size field's ceiling. None of those is in
33 * the engine, and each is a sentence rather than an archive that unpacks wrong.
34 *
35 * Every archive is a function of its entries and nothing else: the mtime is
36 * one number handed in (0 unless said), owner and group are 0 and nameless. So
37 * one tree always tars to the same bytes, and a hash of the archive is a hash of
38 * the tree.
39 *
40 * `untar` is the reader, and it is here rather than in a test because the
41 * format has two halves and the one that proves the other should sit beside it.
42 * It checks every checksum and reads only regular files — it is not a general
43 * tar reader, it is this writer's inverse.
44 */
45
46 /** One file in an archive: its path inside it, and its bytes. */
47 export type Entry = { name: string; data: Uint8Array };
48
49 /** A file read back out of an archive, with the header fields that matter. */
50 export type Read = Entry & { mode: number; mtime: number; size: number };
51
52 /** Every block in a tar is this many bytes, header and data alike. */
53 export const BLOCK = 512;
54
55 /** The largest size the 11-digit octal field can hold, plus one. */
56 const SIZE_LIMIT = 8 ** 11;
57
58 const ascii = new TextEncoder();
59 const text = new TextDecoder();
60
61 /** A number as the header writes one: `digits` octal digits, zero-padded, then NUL. */
62 function octal(n: number, digits: number): string {
63 const s = n.toString(8);
64 if (s.length > digits) throw new Error(`${n} does not fit ${digits} octal digits`);
65 return `${s.padStart(digits, "0")}\0`;
66 }
67
68 /**
69 * Split a path into ustar's two fields: `name` of at most 100 bytes and
70 * `prefix` of at most 155, joined back by a `/` the archive does not store.
71 */
72 export function splitName(path: string): { name: string; prefix: string } {
73 if (!/^[\x21-\x7e][\x20-\x7e]*$/.test(path)) throw new Error(`${JSON.stringify(path)} is not a printable ASCII path`);
74 if (path.startsWith("/") || path.split("/").some((part) => part === ".." || part === "" || part === "."))
75 throw new Error(`${JSON.stringify(path)} is not a relative path to a file`);
76 if (path.length <= 100) return { name: path, prefix: "" };
77 // The last `/` that leaves a name of 100 bytes or fewer and a prefix of 155.
78 for (let at = path.lastIndexOf("/"); at > 0; at = path.lastIndexOf("/", at - 1)) {
79 const name = path.slice(at + 1);
80 const prefix = path.slice(0, at);
81 if (name.length > 100) break;
82 if (prefix.length <= 155) return { name, prefix };
83 }
84 throw new Error(`${JSON.stringify(path)} is too long for a ustar header`);
85 }
86
87 /** Write `value` into `block` at `offset`, never past `size` bytes. */
88 function put(block: Uint8Array, offset: number, size: number, value: string): void {
89 const bytes = ascii.encode(value);
90 if (bytes.length > size) throw new Error(`${JSON.stringify(value)} overflows a ${size}-byte field`);
91 block.set(bytes, offset);
92 }
93
94 /** The sum of every header byte, with the checksum field counted as eight spaces. */
95 export function checksum(header: Uint8Array): number {
96 let sum = 0;
97 for (let i = 0; i < BLOCK; i++) sum += i >= 148 && i < 156 ? 0x20 : (header[i] as number);
98 return sum;
99 }
100
101 /** One file's 512-byte header. */
102 export function header(path: string, size: number, mtime = 0): Uint8Array {
103 if (!Number.isInteger(size) || size < 0 || size >= SIZE_LIMIT) throw new Error(`${size} is not a size a ustar header holds`);
104 const { name, prefix } = splitName(path);
105 const block = new Uint8Array(BLOCK);
106 put(block, 0, 100, name);
107 put(block, 100, 8, octal(0o644, 7));
108 put(block, 108, 8, octal(0, 7));
109 put(block, 116, 8, octal(0, 7));
110 put(block, 124, 12, octal(size, 11));
111 put(block, 136, 12, octal(mtime, 11));
112 block[156] = 0x30; // '0': a regular file
113 put(block, 257, 6, "ustar\0");
114 put(block, 263, 2, "00");
115 put(block, 345, 155, prefix);
116 // Six octal digits, a NUL and a space: the form every tar since V7 writes.
117 put(block, 148, 8, `${checksum(block).toString(8).padStart(6, "0")}\0 `);
118 return block;
119 }
120
121 /** How many bytes `size` takes once padded to whole blocks. */
122 const padded = (size: number): number => Math.ceil(size / BLOCK) * BLOCK;
123
124 /**
125 * The archive: each entry's header and its data padded to a block, then two
126 * zero blocks. A name twice is refused — two files at one path unpack as one.
127 */
128 export function tar(entries: readonly Entry[], mtime = 0): Uint8Array<ArrayBuffer> {
129 const seen = new Set<string>();
130 for (const { name } of entries) {
131 if (seen.has(name)) throw new Error(`${JSON.stringify(name)} is in the archive twice`);
132 seen.add(name);
133 }
134 const total = entries.reduce((sum, { data }) => sum + BLOCK + padded(data.length), 0) + 2 * BLOCK;
135 const out = new Uint8Array(total);
136 let at = 0;
137 for (const { name, data } of entries) {
138 out.set(header(name, data.length, mtime), at);
139 out.set(data, at + BLOCK);
140 at += BLOCK + padded(data.length);
141 }
142 return out;
143 }
144
145 /** A NUL-terminated field, as text. */
146 const field = (block: Uint8Array, offset: number, size: number): string => {
147 const bytes = block.subarray(offset, offset + size);
148 const end = bytes.indexOf(0);
149 return text.decode(end === -1 ? bytes : bytes.subarray(0, end));
150 };
151
152 /** An octal field, as a number; a field that is not octal is a broken archive. */
153 const number = (block: Uint8Array, offset: number, size: number): number => {
154 const s = field(block, offset, size).trim();
155 if (!/^[0-7]+$/.test(s)) throw new Error(`the header field at ${offset} is ${JSON.stringify(s)}, not octal`);
156 return parseInt(s, 8);
157 };
158
159 /**
160 * Read an archive this module wrote back into its files. Every header's
161 * checksum is checked, the magic is checked, and it stops at the first zero
162 * block — which must be followed by a second, as the format says.
163 */
164 export function untar(archive: Uint8Array): Read[] {
165 if (archive.length % BLOCK !== 0) throw new Error(`an archive is whole ${BLOCK}-byte blocks; this is ${archive.length} bytes`);
166 const out: Read[] = [];
167 for (let at = 0; at < archive.length; ) {
168 const block = archive.subarray(at, at + BLOCK);
169 if (block.every((b) => b === 0)) {
170 const next = archive.subarray(at + BLOCK, at + 2 * BLOCK);
171 if (next.length !== BLOCK || !next.every((b) => b === 0)) throw new Error("the archive ends with one zero block, not two");
172 return out;
173 }
174 if (number(block, 148, 8) !== checksum(block)) throw new Error(`the header at ${at} has a bad checksum`);
175 if (field(block, 257, 6) !== "ustar") throw new Error(`the header at ${at} is not ustar`);
176 if (block[156] !== 0x30) throw new Error(`the header at ${at} is not a regular file`);
177 const prefix = field(block, 345, 155);
178 const name = prefix === "" ? field(block, 0, 100) : `${prefix}/${field(block, 0, 100)}`;
179 const size = number(block, 124, 12);
180 const data = archive.slice(at + BLOCK, at + BLOCK + size);
181 if (data.length !== size) throw new Error(`${name} is cut short`);
182 out.push({ name, data, size, mode: number(block, 100, 8), mtime: number(block, 136, 12) });
183 at += BLOCK + padded(size);
184 }
185 throw new Error("the archive has no end-of-archive blocks");
186 }

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs