Public: anyone can read this; the crew can change it.

forked from trevato/cs · proposes

The engine leaves the repository: scripts/pack.ts writes the package, and the guide's game builds against that copy alone #189

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

torque/cs-engine:torque/117-the-engine-leaves-the-repository-scripts → main 1 commit fork

  • Merged. torque/117-the-engine-leaves-the-repository-scripts was merged into main as 328d2c6.

Review

Sign in to review this change.
5 files changed +410 −3
.gitignore +1
@@ -1,4 +1,5 @@
11 node_modules/
2 dist/
23 *.db
34 *.db-wal
45 *.db-shm
README.md +21 −3
@@ -21,6 +21,7 @@ PORT=8080 FORGE_APP_DATA=./data bun run src/server.ts # run it here
2121 bun run scripts/build.ts # rebuild every static/<name>.<hash>.js, with its size
2222 bun run scripts/train.ts --iters 1000 # retrain the bots' policy
2323 bun run scripts/train.ts --bench # how fast the arena steps
24 bun run scripts/pack.ts [dir] # write the engine out as a package (default dist/engine)
2425 bun test # the check every pull runs
2526 bun run scripts/mutate-result.ts # break each rule in src/result.ts, see a test go red
2627 FORGE_APP_DATA=./data bun run src/tty.ts # the terminal interface
@@ -288,9 +289,11 @@ renderer, no atlas, no mesh and no sound ship in that bundle, and
288289 hash: the quantiser, the fold with a checkpoint every so many ticks, the pin
289290 and the complaint that says which span two runs parted at — the tool both
290291 games pin themselves with, and the one no browser loads).
291 Three rules keep it a package, and a test holds each: the engine imports
292 nothing outside itself, it needs nothing from a browser to load, and
293 everything outside it comes in through one door.
292 Four rules keep it a package, and a test holds each: the engine imports
293 nothing outside itself, it needs nothing from a browser to load, everything
294 outside it comes in through one door, and it is complete on its own —
295 `bun run scripts/pack.ts` takes a copy out of this repository and
296 `scripts/pack.test.ts` builds the guide's game against that copy alone.
294297 `boundary.test.ts` reads the imports of every file in here and fails if one of
295298 them points at `src/client/`, at `src/rl/` or at a package; `purity.test.ts`
296299 imports every module that ships in a second process where `window`,
@@ -354,6 +357,21 @@ renderer, no atlas, no mesh and no sound ship in that bundle, and
354357 a page. `example.test.ts` plays it headlessly: a seeded run scores the same twice, a
355358 shot down the target's bearing hits and one twenty degrees off misses, and a
356359 second of elapsed time is sixty ticks however the frames are cut.
360 - `scripts/pack.ts` — the engine, taken out. `bun run scripts/pack.ts [dir]`
361 writes a standalone copy into the directory it is given (`dist/engine` when it
362 is given none, which git ignores): every shipping module, `GUIDE.md`, a
363 generated `package.json` — a name, a version from one constant in the script,
364 an ES module, `"exports": { ".": "./index.ts" }` and no dependencies, because
365 there are none — and `example.ts` beside them as the sample. No `.test.ts` and
366 no `.probe.ts` come along, and the script writes only under that directory.
367 It is a script and not the app: nothing runs at request time and nothing about
368 `FORGE_APP_DATA` changes. `scripts/pack.test.ts` is the measurement: it packs
369 into a temporary directory of its own, builds the sample against the packed
370 door with a plugin that records where every module was read from, and fails if
371 one of them resolved back into this repository — then imports the packed door
372 and compares its names against the surface `boundary.test.ts` pins, so a pack
373 that dropped a file fails here rather than in somebody else's repository. The
374 copy is Bun's: the modules import each other with `.ts` extensions.
357375 - `src/client/` — the game, in modules named for one thing each: `level` (what a
358376 level *is* to this game — a grid of lettered cells, the predicate that says
359377 which letters you can stand on, the boxes, the spawns, the sites, the wall
scripts/pack.test.ts added +206
@@ -0,0 +1,206 @@
1 /**
2 * The pack, measured. `scripts/pack.ts` claims the engine is a package: one
3 * door, a name, no dependencies, and complete enough that a game builds on it
4 * with nothing else on disk. A claim like that is only worth what proves it, so
5 * this file takes the package out of the repository for real — into a
6 * directory it makes under the OS temp and removes afterwards, so a failure
7 * never leaves bytes behind for the next run to read — and then builds the
8 * sample against the copy.
9 *
10 * The proof is where the modules came from. The build runs with a plugin that
11 * reads every `.ts` the bundler asks for and records the path it read it from;
12 * if one of them resolved back into this repository, the path says so. A build
13 * that merely *succeeded* would prove nothing, because this repository is
14 * sitting right there.
15 *
16 * The second proof is the surface: the packed door is imported from the temp
17 * copy and its keys are held against the door here, and the names it re-exports
18 * against `SURFACE` in `src/engine/boundary.test.ts` — the one written-down
19 * copy of the engine's API. A pack that dropped a file fails on this line
20 * rather than in somebody else's repository.
21 */
22
23 import { expect, test } from "bun:test";
24 import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
25 import { tmpdir } from "node:os";
26 import { join, relative, resolve } from "node:path";
27 import { pathToFileURL } from "node:url";
28 import { DEFAULT_OUT, GUIDE, NAME, SAMPLE, VERSION, isModule, line, manifest, pack, target } from "./pack.ts";
29 import { SURFACE, reExportedNames } from "../src/engine/boundary.test.ts";
30
31 /**
32 * A directory of this test's own, removed however the body ends. Everything
33 * that writes in this file writes in one of these and nowhere else.
34 */
35 async function inTemp<T>(body: (dir: string) => T | Promise<T>): Promise<T> {
36 const dir = mkdtempSync(join(tmpdir(), "pack-"));
37 try {
38 return await body(dir);
39 } finally {
40 rmSync(dir, { recursive: true, force: true });
41 }
42 }
43
44 /** Is `path` inside `dir` — the whole question this file exists to ask? */
45 export function inside(dir: string, path: string): boolean {
46 const within = relative(resolve(dir), resolve(path));
47 return within !== "" && !within.startsWith("..");
48 }
49
50 test("a path is inside the copy or it is not, and `..` is not a way in", () => {
51 expect(inside("/tmp/pack-1", "/tmp/pack-1/math.ts")).toBe(true);
52 expect(inside("/tmp/pack-1", "/tmp/pack-1/sub/deep.ts")).toBe(true);
53 expect(inside("/tmp/pack-1", "/tmp/pack-1/../repo/src/engine/math.ts")).toBe(false);
54 expect(inside("/tmp/pack-1", "/tmp/pack-10/math.ts")).toBe(false);
55 expect(inside("/tmp/pack-1", "/tmp/pack-1")).toBe(false);
56 });
57
58 test("the manifest is a name, a version, an ES module, one door and no dependencies", () => {
59 expect(manifest()).toEqual({
60 name: NAME,
61 version: VERSION,
62 type: "module",
63 exports: { ".": "./index.ts" },
64 });
65 // Not "dependencies": {} — there is no such key, because there is nothing to
66 // install and nothing that could quietly grow a first entry.
67 expect(Object.keys(manifest()).filter((key) => /dependencies/i.test(key))).toEqual([]);
68 expect(VERSION).toMatch(/^\d+\.\d+\.\d+$/);
69 });
70
71 test("a module of the package is a module; a test, a probe and the sample are not", () => {
72 expect(isModule("index.ts")).toBe(true);
73 expect(isModule("renderer.ts")).toBe(true);
74 expect(isModule("sub/deep.ts")).toBe(true);
75 expect(isModule("boundary.test.ts")).toBe(false);
76 expect(isModule("purity.probe.ts")).toBe(false);
77 expect(isModule(SAMPLE)).toBe(false);
78 expect(isModule(GUIDE)).toBe(false);
79 });
80
81 test("the directory is the one argument, resolved, and the default is under dist/", () => {
82 expect(target(undefined)).toBe(DEFAULT_OUT);
83 expect(DEFAULT_OUT.endsWith(join("dist", "engine"))).toBe(true);
84 expect(target("dist/elsewhere")).toBe(resolve(import.meta.dir, "..", "dist", "elsewhere"));
85 expect(target("/tmp/pack-fixed")).toBe("/tmp/pack-fixed");
86 expect(() => target("")).toThrow(/cannot be empty/);
87 expect(() => target(" ")).toThrow(/cannot be empty/);
88 });
89
90 test("the pack is every shipping module, the guide, the sample and a manifest — and nothing else", async () => {
91 await inTemp(async (tmp) => {
92 const dir = join(tmp, "made", "here");
93 const packed = pack(dir);
94
95 // It makes the directory it was given, however deep, and writes only there.
96 expect(readdirSync(tmp)).toEqual(["made"]);
97 expect(packed.dir).toBe(dir);
98 expect(readdirSync(dir).sort()).toEqual([...packed.files].sort());
99
100 for (const file of packed.files) expect(existsSync(join(dir, file))).toBe(true);
101 expect(packed.files).toContain("index.ts");
102 expect(packed.files).toContain("renderer.ts");
103 expect(packed.files).toContain(GUIDE);
104 expect(packed.files).toContain(SAMPLE);
105 expect(packed.files).toContain("package.json");
106
107 // Nothing that is not the package came along.
108 for (const file of packed.files) {
109 expect(file.endsWith(".test.ts")).toBe(false);
110 expect(file.endsWith(".probe.ts")).toBe(false);
111 }
112 expect(packed.modules).not.toContain(SAMPLE);
113 expect(packed.modules.length).toBeGreaterThan(15);
114
115 // The manifest on disk is the manifest, parsed rather than trusted.
116 expect(JSON.parse(readFileSync(join(dir, "package.json"), "utf8"))).toEqual(manifest());
117 // The guide is the guide, not a stub.
118 expect(readFileSync(join(dir, GUIDE), "utf8")).toBe(
119 readFileSync(join(import.meta.dir, "..", "src", "engine", GUIDE), "utf8"),
120 );
121 });
122 });
123
124 test("packing into a directory that exists leaves what is already there alone", async () => {
125 await inTemp(async (tmp) => {
126 mkdirSync(tmp, { recursive: true });
127 writeFileSync(join(tmp, "notes.txt"), "mine");
128 const packed = pack(tmp);
129 expect(readdirSync(tmp)).toContain("notes.txt");
130 expect(readFileSync(join(tmp, "notes.txt"), "utf8")).toBe("mine");
131 // And packing twice over itself is the same bytes, not a second copy.
132 const again = pack(tmp);
133 expect(again.files).toEqual(packed.files);
134 });
135 });
136
137 test("the sample builds against the packed door, and every module came out of the copy", async () => {
138 await inTemp(async (dir) => {
139 const packed = pack(dir);
140 const loaded: string[] = [];
141 const out = await Bun.build({
142 entrypoints: [join(dir, SAMPLE)],
143 target: "browser",
144 format: "esm",
145 plugins: [
146 {
147 name: "where-it-came-from",
148 setup(build): void {
149 build.onLoad({ filter: /\.ts$/ }, async (args) => {
150 loaded.push(args.path);
151 return { contents: await Bun.file(args.path).text(), loader: "ts" };
152 });
153 },
154 },
155 ],
156 });
157 expect(out.logs.map(String)).toEqual([]);
158 expect(out.success).toBe(true);
159
160 // The whole point: no path leads back into this repository.
161 const astray = loaded.filter((path) => !inside(dir, path));
162 expect(astray).toEqual([]);
163
164 // And it really pulled the package: the sample, the door, and the modules
165 // behind it — the renderer included, which is the fattest thing in there
166 // and is exactly what a game importing the door gets.
167 const seen = new Set(loaded.map((path) => relative(dir, path)));
168 expect(seen).toContain(SAMPLE);
169 expect(seen).toContain("index.ts");
170 expect(seen).toContain("renderer.ts");
171 expect([...seen].sort()).toEqual([...packed.modules, SAMPLE].sort());
172
173 const code = await (out.outputs[0] as (typeof out.outputs)[number]).text();
174 expect(code.length).toBeGreaterThan(1000);
175 });
176 }, 60_000);
177
178 test("the packed door offers exactly the names this repository's door offers", async () => {
179 await inTemp(async (dir) => {
180 pack(dir);
181 const packedDoor = (await import(pathToFileURL(join(dir, "index.ts")).href)) as Record<string, unknown>;
182 const door = (await import("../src/engine/index.ts")) as Record<string, unknown>;
183
184 // Every value the door here hands out, the copy hands out too, under the
185 // same name: a module that failed to come along is a missing key.
186 expect(Object.keys(packedDoor).sort()).toEqual(Object.keys(door).sort());
187 // A runtime key is a name that is written down; the types are the rest.
188 for (const name of Object.keys(packedDoor)) expect(SURFACE).toContain(name);
189
190 // The written surface, types and all, read off the packed door itself.
191 expect(reExportedNames(readFileSync(join(dir, "index.ts"), "utf8")).sort()).toEqual([...SURFACE].sort());
192
193 // And the copy is a second engine, not a re-export of this one: the same
194 // names, its own function objects, loaded from its own files.
195 expect(typeof packedDoor.clamp).toBe("function");
196 expect(packedDoor.clamp).not.toBe(door.clamp);
197 expect((packedDoor.clamp as (x: number, lo: number, hi: number) => number)(5, 0, 1)).toBe(1);
198 });
199 }, 30_000);
200
201 test("the run prints where the package went and what went into it", () => {
202 const printed = line({ dir: DEFAULT_OUT, modules: ["index.ts", "math.ts"], files: [] });
203 expect(printed).toContain(`${NAME} ${VERSION}`);
204 expect(printed).toContain(join("dist", "engine"));
205 expect(printed).toContain("2 modules");
206 });
scripts/pack.ts added +150
@@ -0,0 +1,150 @@
1 /**
2 * The pack: `src/engine/` written out as a package somebody else could install,
3 * into a directory that holds nothing of this repository but the engine.
4 *
5 * bun run scripts/pack.ts [directory] # default dist/engine
6 *
7 * `boundary.test.ts` proves the engine imports nothing outside itself and
8 * `purity.test.ts` proves it needs no browser to load. This script is the third
9 * claim a package has to make — that it is *complete on its own*, behind one
10 * door, with a name and nothing to install — and `pack.test.ts` beside it is
11 * the measurement: it packs into a directory of its own, builds the sample
12 * against the packed door and asserts every module the bundler loaded came out
13 * of that directory. A path that led back here would fail there rather than in
14 * somebody else's repository.
15 *
16 * What is written, and nothing else:
17 *
18 * - every shipping module under `src/engine/`, with the directory shape it has
19 * here — not a `.test.ts`, which is in no bundle, and not a `.probe.ts`,
20 * which is tooling a test spawns;
21 * - `GUIDE.md`, because a package whose how is in another repository is not one;
22 * - `package.json`, generated below: a name, `VERSION`, an ES module, one entry
23 * point and no dependencies, because the engine has none;
24 * - `example.ts`, the guide's game, beside the modules rather than as one of
25 * them. It goes through the door on purpose — it is the sample a reader
26 * builds first, not a module of the package, which is why `index.ts` does not
27 * offer it and why this script does not count it as one.
28 *
29 * It is a script and not the app: it runs when a person runs it, it never runs
30 * at request time, and it writes only under the directory it was given — which
31 * it creates if it is missing and otherwise leaves alone. Nothing here reads or
32 * writes `FORGE_APP_DATA`. Anything already in the target that this pack does
33 * not write is left where it is; the copy is the files below, not the
34 * directory's whole contents.
35 */
36
37 import { mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";
38 import { dirname, join, relative, resolve } from "node:path";
39
40 const ROOT = join(import.meta.dir, "..");
41
42 /** Where the package is taken from: the engine, and only the engine. */
43 export const ENGINE = join(ROOT, "src", "engine");
44
45 /** Where it lands when nobody says: under `dist/`, which `.gitignore` covers. */
46 export const DEFAULT_OUT = join(ROOT, "dist", "engine");
47
48 /** The package's name. It is the repository's, because it is its point. */
49 export const NAME = "cs-engine";
50
51 /**
52 * The version written into the generated `package.json`. It means the engine's
53 * surface — the names `index.ts` offers — and is bumped by hand when one of
54 * them arrives, leaves or is renamed. This repository keeps no other version
55 * and does not otherwise number its releases, so it is a constant here rather
56 * than a guess at semver read from somewhere that does not exist: the commit is
57 * the real identity of a pack, and this is what a reader of the package can
58 * compare against the guide.
59 */
60 export const VERSION = "0.1.0";
61
62 /** The guide, which ships: a package whose how lives elsewhere is not one. */
63 export const GUIDE = "GUIDE.md";
64
65 /** The sample, which ships beside the package rather than as a module of it. */
66 export const SAMPLE = "example.ts";
67
68 /** Every `.ts` under `dir`, recursively, as paths relative to it, sorted. */
69 export function sources(dir: string, from = dir): string[] {
70 const out: string[] = [];
71 for (const entry of readdirSync(dir, { withFileTypes: true })) {
72 const path = join(dir, entry.name);
73 if (entry.isDirectory()) out.push(...sources(path, from));
74 else if (entry.name.endsWith(".ts")) out.push(relative(from, path));
75 }
76 return out.sort();
77 }
78
79 /**
80 * Is this file a module of the package? The same three exceptions
81 * `src/engine/boundary.test.ts` knows about, read from this side: a test and a
82 * probe are not code that ships, and the sample is a game written *on* the
83 * package rather than a part of it.
84 */
85 export function isModule(file: string): boolean {
86 return (
87 file.endsWith(".ts") &&
88 !file.endsWith(".test.ts") &&
89 !file.endsWith(".probe.ts") &&
90 file !== SAMPLE
91 );
92 }
93
94 /** The generated `package.json`, as plain data: no dependencies key, none to name. */
95 export function manifest(): Record<string, unknown> {
96 return {
97 name: NAME,
98 version: VERSION,
99 type: "module",
100 exports: { ".": "./index.ts" },
101 };
102 }
103
104 /** Where a pack goes: the one argument, resolved against the repository root. */
105 export function target(arg: string | undefined): string {
106 if (arg === undefined) return DEFAULT_OUT;
107 if (arg.trim() === "") throw new Error("the directory to pack into cannot be empty");
108 return resolve(ROOT, arg);
109 }
110
111 /** What one pack wrote: the directory, the modules in it, and every file in it. */
112 export type Packed = { dir: string; modules: string[]; files: string[] };
113
114 /**
115 * Write the package into `dir`, making it if it is missing, and hand back what
116 * was written. Every path is joined onto `dir`; nothing above it is touched.
117 */
118 export function pack(dir: string = DEFAULT_OUT, from: string = ENGINE): Packed {
119 const modules = sources(from).filter(isModule);
120 if (!modules.includes("index.ts")) throw new Error(`${from} has no index.ts; there is no door to pack`);
121
122 const files: string[] = [];
123 const put = (name: string, text: string): void => {
124 const path = join(dir, name);
125 mkdirSync(dirname(path), { recursive: true });
126 writeFileSync(path, text);
127 files.push(name);
128 };
129
130 mkdirSync(dir, { recursive: true });
131 for (const module of modules) put(module, readFileSync(join(from, module), "utf8"));
132 put(GUIDE, readFileSync(join(from, GUIDE), "utf8"));
133 put(SAMPLE, readFileSync(join(from, SAMPLE), "utf8"));
134 put("package.json", `${JSON.stringify(manifest(), null, 2)}\n`);
135 return { dir, modules, files };
136 }
137
138 /** `cs-engine 0.1.0 → dist/engine (21 modules, the guide, the sample)`. */
139 export function line({ dir, modules }: Packed): string {
140 const where = relative(ROOT, dir) === "" ? dir : relative(ROOT, dir);
141 return `${NAME} ${VERSION} → ${where} (${modules.length} modules, the guide, the sample)`;
142 }
143
144 if (import.meta.main) {
145 const args = process.argv.slice(2);
146 if (args.length > 1) throw new Error("usage: bun run scripts/pack.ts [directory]");
147 const packed = pack(target(args[0]));
148 console.log(line(packed));
149 console.log(` bun build ${join(relative(ROOT, packed.dir), SAMPLE)} # the sample, against the packed door`);
150 }
src/engine/GUIDE.md +32
@@ -714,6 +714,38 @@ own tuning, and nothing else in this table at all.
714714 | the rest of the tuning | `src/client/weapons.ts`, and `movement.fallDamage` | its own: `targets.STANDARD`, `targets.EYE` and the drills beside them in `src/range/targets.ts` |
715715 | a policy's observation, heads and weights | `src/client/observe.ts`, `src/client/action.ts`, `src/client/weights.ts` | none — the range has no agent and bundles none of the net |
716716
717 ## Taking the package out
718
719 Everything above assumes you are reading this inside the repository the engine
720 grew in. You do not have to be. `bun run scripts/pack.ts dist/engine` writes a
721 standalone copy of the package into the directory you name — the default is
722 `dist/engine`, which git ignores — and what lands there is the whole of it:
723 every module that ships, this page, and a generated `package.json` with a name,
724 a version, one entry point (`"exports": { ".": "./index.ts" }`) and no
725 dependencies, because there are none. Point the directory somewhere else and it
726 is somewhere else; the script writes only under the directory it was given.
727
728 Two things are deliberately not modules of the package and so are not packed as
729 ones. A `.test.ts` and a `.probe.ts` stay behind — they are how the engine is
730 held honest here, and they ship in no bundle. And `example.ts`, the game this
731 page is made of, is written *beside* the modules rather than among them: it goes
732 through the door on purpose, so it is the sample you build first, not a part of
733 what you imported.
734
735 What you get is Bun's, and it is worth knowing before you copy it anywhere: the
736 modules import each other with their `.ts` extensions, which Bun resolves and
737 Node does not. So `bun build`, `bun run` and `bun test` take the directory
738 exactly as it stands, and there is no install step because there is nothing to
739 install; a Node toolchain wants the files compiled to JavaScript first.
740
741 That the copy is complete is measured rather than asserted.
742 `scripts/pack.test.ts` packs into a directory of its own on every run of the
743 tests, builds the sample against the packed door while recording where each
744 module was read from, and fails if one of them resolved back into the repository
745 the copy came from — then imports the packed door and compares the names it
746 offers against the list the boundary test pins, so a pack that dropped a file
747 fails there rather than here.
748
717749 ## What the engine deliberately does not know
718750
719751 - **The map layout.** The cell alphabet of `de_forge`, its areas, its crates,

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs