Public: anyone can read this; the crew can change it.

Socket: the connection, its rules and its limits, before any frame goes through it #37

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

vesper/drift:vesper/23-socket-the-connection-its-rules-and-its → main 2 commits fork 1 thread · 1 unresolved

  • Merged. vesper/23-socket-the-connection-its-rules-and-its was merged into main as ea28cb9.

Review

Sign in to review this change.
7 files changed +975 −16
README.md +28 −6
@@ -19,10 +19,13 @@ FORGE_APP_DATA=./data bun run src/db.ts --check # schema version, integr
1919 The page is the arena and nothing else: a square the sea will be drawn in,
2020 a round clock, one line of the rule, a roster and a ladder — all rendered
2121 on the server, all legible with JavaScript off, none of them fed by a
22 simulation yet. There is no world tick, no socket and no score behind
23 them; the regions are shells with honest empty states, and the round loop,
24 the canvas and the ladder fill them in the issues that follow. `POST /mcp`
25 answers and lists no tool.
22 simulation yet. There is no world tick and no score behind them; the
23 regions are shells with honest empty states, and the round loop, the canvas
24 and the ladder fill them in the issues that follow. `POST /mcp` answers and
25 lists no tool. `GET /ws` upgrades and enforces every rule a connection has
26 — origin, identity, one seat per person, a ration, backpressure — but
27 nothing publishes frames on it yet, so a socket that opens stays quiet
28 until the round drives it.
2629
2730 ## The game
2831
@@ -48,6 +51,7 @@ decide nothing themselves. `AGENTS.md` says how to add to it.
4851 | `src/who.ts` | who is asking, from the forge's signed headers |
4952 | `src/db.ts` | SQLite under `FORGE_APP_DATA`, WAL, forward-only migrations |
5053 | `src/live.ts` | the in-process bus and `GET /events` |
54 | `src/socket.ts` | `GET /ws`: who may open one, what it costs, what closes it |
5155 | `src/mcp.ts` | the actions as MCP tools at `POST /mcp` — none yet |
5256 | `src/routes.ts` | every path, in one table |
5357 | `src/concept.ts` | the design record as data: the rule, the round, the invariants |
@@ -77,8 +81,9 @@ Every address in the head — canonical, `og:url`, `og:image` — is built from
7781 `FORGE_APP_URL` (`publicOrigin`) and never from `Host`, `X-Forwarded-Host`
7882 or anything else a client sent; a test in `src/server.test.ts` holds it
7983 there. `GET /robots.txt` leaves the page crawlable and closes `/events`,
80 `/mcp` and `/__forge/`: a crawler has no business holding a stream open, and
81 joining a round is a socket event so that no fetch can ever take a seat.
84 `/ws`, `/mcp` and `/__forge/`: a crawler has no business holding a stream
85 open, and joining a round is a socket event so that no fetch can ever take a
86 seat.
8287
8388 The PNGs are encoded by `src/png.ts` — signature, IHDR, one deflated IDAT,
8489 adaptive row filters — using `node:zlib`, which Bun ships. No dependency was
@@ -121,6 +126,23 @@ fifty-six for one address, which behind the forge is every guest at once —
121126 and over the cap the answer is `429` rather than another socket held open.
122127 Every response, the stream included, carries the floor in `src/http.ts`.
123128
129 The world itself travels over one websocket, `GET /ws` (`src/socket.ts`).
130 It is the one door the request rate limit never sees, so its rules are its
131 own and they are strict. An upgrade must carry an `Origin` that is ours — a
132 missing one is refused, unlike a POST's, because a websocket has no
133 preflight. The identity is fixed at the upgrade and kept for the life of
134 the connection: no message ever names its own sender. Every message is
135 validated by `parseSteer` and by nothing else, and a message that fails
136 closes the socket with `1008`. Steering is rationed per socket — 20 Hz
137 passes forever, a burst over the bucket is dropped where it lands rather
138 than queued, and a client that keeps flooding is closed. **A seat is a
139 person, not a tab:** a second socket on the same player key closes the
140 first with `4000`, so one person is one mote however many tabs they open
141 and nobody can hold the roster alone. A socket is pinged every 25 seconds
142 and closed with `4001` after 90 seconds of silence, and a client whose send
143 buffer is backed up is skipped for that frame rather than waited on, so a
144 slow phone degrades alone.
145
124146 The terminal opens with the exact command in `FORGE_APP_SSH` when the app
125147 runs (the app is the login name, the forge the host). It is `src/tty.ts` on
126148 a real pty, as the account at the keyboard; today it prints where it is and
src/meta.ts +6 −4
@@ -54,16 +54,18 @@ export function social(path = "/"): string {
5454 }
5555
5656 /**
57 * The page is crawlable; the doors are not pages. `/events` is a stream that
58 * never ends and `/mcp` speaks JSON-RPC — a crawler that pulls on either
59 * gains nothing and holds a connection open. Joining a round is a socket
60 * event (#11) precisely so that no crawler can ever take a seat by fetching.
57 * The page is crawlable; the doors are not pages. `/events` and `/ws` are
58 * streams that never end and `/mcp` speaks JSON-RPC — a crawler that pulls on
59 * any of them gains nothing and holds a connection open. Joining a round is a
60 * socket event (#11) precisely so that no crawler can ever take a seat by
61 * fetching; a crawler that reaches `/ws` anyway is told `426` and nothing else.
6162 */
6263 export const robots = (): string =>
6364 [
6465 "# drift — one shared world. The page is for reading; the doors are not.",
6566 "User-agent: *",
6667 "Disallow: /events",
68 "Disallow: /ws",
6769 "Disallow: /mcp",
6870 "Disallow: /__forge/",
6971 "Allow: /",
src/routes.ts +28 −4
@@ -3,9 +3,10 @@
33 * calls an action, and shapes the answer for its door — HTML for a
44 * browser, JSON for a script — and that is all a route does.
55 *
6 * Today that is the page, the event stream, the MCP endpoint, the two
7 * pictures a pasted link shows, `robots.txt` and the files: the scaffold's
8 * routes left with #PAGE, and the round's JSON doors arrive with #DOORS.
6 * Today that is the page, the event stream, the socket, the MCP endpoint,
7 * the two pictures a pasted link shows, `robots.txt` and the files: the
8 * scaffold's routes left with #PAGE, and the round's JSON doors arrive
9 * with #DOORS.
910 */
1011
1112 import type { Database } from "bun:sqlite";
@@ -25,6 +26,7 @@ import { events } from "./live.ts";
2526 import { rpc } from "./mcp.ts";
2627 import { robots } from "./meta.ts";
2728 import { home } from "./page.ts";
29 import { upgrade } from "./socket.ts";
2830 import type { Who } from "./who.ts";
2931
3032 export interface Context {
@@ -35,12 +37,25 @@ export interface Context {
3537 readonly params: Record<string, string>;
3638 /** The address the request came from, for what is rationed per stranger. */
3739 readonly address: string;
40 /**
41 * The running server, for the one door that needs it: a websocket upgrade
42 * is `Bun.serve`'s to perform, not a response to write. Absent when a test
43 * calls a route without a server, and the socket door refuses rather than
44 * pretending.
45 */
46 readonly server?: { upgrade(request: Request, options?: unknown): boolean };
3847 }
3948
4049 export interface Route {
4150 readonly method: "GET" | "POST" | "DELETE";
4251 readonly path: string;
43 readonly handle: (ctx: Context) => Response | Promise<Response>;
52 /**
53 * A response, or nothing when the handler took the connection over — which
54 * only `/ws` does, because an upgraded socket has no response to give.
55 */
56 readonly handle: (
57 ctx: Context,
58 ) => Response | undefined | Promise<Response | undefined>;
4459 }
4560
4661 export const routes: readonly Route[] = [
@@ -63,6 +78,15 @@ export const routes: readonly Route[] = [
6378 path: "/events",
6479 handle: (c) => events(c.request, bucketOf(c.who, c.address)),
6580 },
81 // The socket. The rules that let it open, and everything it costs, are in
82 // `socket.ts`; this row is only here so the table stays the map of every
83 // path. A GET that is not an upgrade gets a plain 426 with the floor on it.
84 {
85 method: "GET",
86 path: "/ws",
87 handle: (c) => upgrade(c),
88 },
89
6690 {
6791 method: "POST",
6892 path: "/mcp",
src/server.test.ts +1 −1
@@ -327,7 +327,7 @@ test("robots.txt: the page is crawlable, the doors are not", async () => {
327327 const body = await response.text();
328328 expect(body).toContain("User-agent: *");
329329 expect(body).toContain("Allow: /");
330 for (const closed of ["/events", "/mcp", "/__forge/"])
330 for (const closed of ["/events", "/ws", "/mcp", "/__forge/"])
331331 expect(body).toContain(`Disallow: ${closed}`);
332332 });
333333
src/server.ts +11 −1
@@ -11,6 +11,7 @@ import { openDb } from "./db.ts";
1111 import { dataDir, identityKey, title, where } from "./env.ts";
1212 import { allow, bucketOf, html, sameOrigin, text } from "./http.ts";
1313 import { match } from "./routes.ts";
14 import { websocket } from "./socket.ts";
1415 import { who } from "./who.ts";
1516
1617 export interface ServeOptions {
@@ -33,8 +34,13 @@ export function serve(options: ServeOptions) {
3334 port: options.port,
3435 hostname: options.hostname ?? "127.0.0.1",
3536 // A connection that says nothing for a minute is dropped. The event
36 // stream heartbeats every 25 seconds to stay well inside that.
37 // stream heartbeats every 25 seconds to stay well inside that, and the
38 // socket pings on the same clock (`socket.ts`).
3739 idleTimeout: 60,
40 // The websocket handlers hang off the server rather than the route
41 // table, which is Bun's shape; `/ws` is still a row in `routes.ts`, and
42 // that row is what performs the upgrade.
43 websocket,
3844 async fetch(request, server) {
3945 const url = new URL(request.url);
4046 const me = who(request, options.identityKey);
@@ -57,7 +63,11 @@ export function serve(options: ServeOptions) {
5763 who: me,
5864 params: found.params,
5965 address,
66 server,
6067 });
68 // Nothing back means the handler took the connection: the websocket
69 // door upgraded it, and `Bun.serve` wants no response for that.
70 if (response === undefined) return undefined;
6171 // HEAD is GET without the body: a health check sees the app up. The
6272 // body it does not get is released, not dropped — a HEAD of a stream
6373 // would otherwise hold a listener and a slot for nothing.
src/socket.test.ts added +491
@@ -0,0 +1,491 @@
1 import { afterAll, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { publicOrigin } from "./env.ts";
6 import { SECURITY } from "./http.ts";
7 import { STEER_MAX_BYTES } from "./protocol.ts";
8 import { serve } from "./server.ts";
9 import {
10 admit,
11 backedUp,
12 CLOSE,
13 onSteer,
14 publish,
15 receive,
16 seatHolder,
17 SEND_MAX_BYTES,
18 SILENCE_MS,
19 socketCount,
20 socketsFrom,
21 SOCKET_MAX_SHARED,
22 type Sock,
23 type SocketData,
24 type Steering,
25 sweep,
26 websocket,
27 } from "./socket.ts";
28 import { ANON, sign, type Who } from "./who.ts";
29
30 const KEY = "a-key-only-the-forge-holds";
31 const server = serve({
32 port: 0,
33 dataDir: mkdtempSync(join(tmpdir(), "socket-")),
34 identityKey: KEY,
35 });
36 const base = `http://${server.hostname}:${server.port}`;
37 const wsUrl = `ws://${server.hostname}:${server.port}/ws`;
38 // Stopping is not awaited: a websocket the runtime is still tearing down can
39 // keep `stop(true)` pending, and the test process has nothing left to wait for.
40 afterAll(() => {
41 void server.stop(true);
42 });
43
44 /** The headers the forge sends for a signed-in person. */
45 const as = (id: string, name: string): Record<string, string> => {
46 const identity = `${id}|${name}|user|${Date.now() + 60_000}`;
47 return {
48 "x-forge-identity": identity,
49 "x-forge-signature": sign(identity, KEY),
50 };
51 };
52
53 interface Client {
54 readonly ws: WebSocket;
55 readonly opened: Promise<boolean>;
56 readonly closed: Promise<{ code: number; reason: string }>;
57 readonly got: string[];
58 readonly bye: () => void;
59 }
60
61 /** A client that connects the way the page will: same origin, real socket. */
62 function connect(headers: Record<string, string> = {}): Client {
63 const got: string[] = [];
64 const ws = new WebSocket(wsUrl, {
65 headers: { origin: publicOrigin, ...headers },
66 } as unknown as string[]);
67 let settleOpen: (ok: boolean) => void = () => {};
68 let settleClose: (end: { code: number; reason: string }) => void = () => {};
69 const opened = new Promise<boolean>((resolve) => (settleOpen = resolve));
70 const closed = new Promise<{ code: number; reason: string }>(
71 (resolve) => (settleClose = resolve),
72 );
73 ws.onopen = () => settleOpen(true);
74 ws.onerror = () => {};
75 ws.onmessage = (event) => got.push(String(event.data));
76 ws.onclose = (event) => {
77 settleOpen(false);
78 settleClose({ code: event.code, reason: event.reason });
79 };
80 return { ws, opened, closed, got, bye: () => ws.close() };
81 }
82
83 /** Wait for something the server does on its own clock, or give up loudly. */
84 async function until(
85 what: string,
86 ready: () => boolean,
87 ms = 2000,
88 ): Promise<void> {
89 const stop = Date.now() + ms;
90 while (!ready()) {
91 if (Date.now() > stop) throw new Error(`timed out waiting for ${what}`);
92 await Bun.sleep(5);
93 }
94 }
95
96 const upgradeHeaders = (extra: Record<string, string> = {}): Request =>
97 new Request(publicOrigin + "/ws", {
98 headers: { connection: "Upgrade", upgrade: "websocket", ...extra },
99 });
100
101 // ---------------------------------------------------------------- who may
102
103 test("an upgrade must say where it came from, and it must be us", () => {
104 const ours = admit(
105 upgradeHeaders({ origin: publicOrigin }),
106 ANON,
107 "10.0.0.1",
108 );
109 expect(ours.ok).toBe(true);
110
111 // A missing Origin is refused here even though a POST may pass without
112 // one: a websocket has no preflight to hide behind.
113 const bare = admit(upgradeHeaders(), ANON, "10.0.0.1");
114 expect(bare.ok).toBe(false);
115 expect(bare.ok === false && bare.status).toBe(403);
116
117 const foreign = admit(
118 upgradeHeaders({ origin: "https://evil.example" }),
119 ANON,
120 "10.0.0.1",
121 );
122 expect(foreign.ok).toBe(false);
123 expect(foreign.ok === false && foreign.status).toBe(403);
124 });
125
126 test("a client from another site never opens, and neither does one with no origin", async () => {
127 const foreign = connect({ origin: "https://evil.example" });
128 expect(await foreign.opened).toBe(false);
129
130 const bare = new WebSocket(wsUrl);
131 const opened = await new Promise<boolean>((resolve) => {
132 bare.onopen = () => resolve(true);
133 bare.onerror = () => {};
134 bare.onclose = () => resolve(false);
135 });
136 expect(opened).toBe(false);
137 expect(socketCount()).toBe(0);
138 });
139
140 test("GET /ws that is not an upgrade is a plain 426, standing on the floor", async () => {
141 const response = await fetch(base + "/ws");
142 expect(response.status).toBe(426);
143 for (const [header, value] of Object.entries(SECURITY))
144 expect(response.headers.get(header)).toBe(value);
145 await response.text();
146 });
147
148 test("the 101 carries the floor too, like every other response", async () => {
149 // The handshake is the only header exchange a socket has, so it is read
150 // the only way it can be: by speaking the handshake by hand.
151 const handshake = await shake({ origin: publicOrigin });
152 expect(handshake.split("\r\n")[0]).toContain("101");
153 for (const [header, value] of Object.entries(SECURITY))
154 expect(handshake.toLowerCase()).toContain(
155 `${header}: ${value}`.toLowerCase(),
156 );
157 await until("the socket to be let go", () => socketCount() === 0);
158 });
159
160 test("the shared cap is a ceiling on sockets from one address", () => {
161 const held = fill("10.9.9.9", SOCKET_MAX_SHARED);
162 expect(socketsFrom("10.9.9.9")).toBe(SOCKET_MAX_SHARED);
163 const over = admit(
164 upgradeHeaders({ origin: publicOrigin }),
165 ANON,
166 "10.9.9.9",
167 );
168 expect(over.ok).toBe(false);
169 expect(over.ok === false && over.status).toBe(503);
170 for (const sock of held) sock.close();
171 expect(socketsFrom("10.9.9.9")).toBe(0);
172 });
173
174 // ------------------------------------------------------------- what it is
175
176 test("identity is fixed at upgrade, and no message can name its own sender", async () => {
177 const heard: Steering[] = [];
178 const stop = onSteer((steering) => heard.push(steering));
179 const client = connect(as("u1", "alice"));
180 expect(await client.opened).toBe(true);
181
182 // The message claims to be somebody else. It is steering, and it steers
183 // the socket's own seat.
184 client.ws.send(JSON.stringify({ x: 1, y: 0, seq: 1, key: "user:u2" }));
185 await until("a steer", () => heard.length === 1);
186 expect(heard[0]!.key).toBe("user:u1");
187 expect(heard[0]!.who).toEqual({ role: "user", id: "u1", name: "alice" });
188 expect(Object.keys(heard[0]!.steer).sort()).toEqual(["seq", "x", "y"]);
189
190 stop();
191 client.bye();
192 await client.closed;
193 });
194
195 test("valid steers arrive in order, clamped into the unit disc", async () => {
196 const seqs: number[] = [];
197 const heard: Steering[] = [];
198 const stop = onSteer((steering) => {
199 seqs.push(steering.steer.seq);
200 heard.push(steering);
201 });
202 const client = connect(as("u2", "bo"));
203 expect(await client.opened).toBe(true);
204 for (let seq = 0; seq < 8; seq++)
205 client.ws.send(JSON.stringify({ x: 3, y: 4, seq }));
206 await until("eight steers", () => seqs.length === 8);
207 expect(seqs).toEqual([0, 1, 2, 3, 4, 5, 6, 7]);
208 expect(Math.hypot(heard[0]!.steer.x, heard[0]!.steer.y)).toBeCloseTo(1, 6);
209
210 stop();
211 client.bye();
212 await client.closed;
213 });
214
215 test("an oversized message closes the socket with 1008", async () => {
216 const client = connect(as("u3", "cass"));
217 expect(await client.opened).toBe(true);
218 const fat = JSON.stringify({
219 x: 0,
220 y: 0,
221 seq: 1,
222 pad: "z".repeat(STEER_MAX_BYTES * 2),
223 });
224 expect(fat.length).toBeGreaterThan(STEER_MAX_BYTES);
225 client.ws.send(fat);
226 const end = await client.closed;
227 expect(end.code).toBe(CLOSE.policy);
228 expect(end.reason).toBe("too_large");
229 });
230
231 test("a malformed message closes the socket with 1008", async () => {
232 const notJson = connect(as("u4", "dey"));
233 expect(await notJson.opened).toBe(true);
234 notJson.ws.send("steer left a bit");
235 expect((await notJson.closed).code).toBe(CLOSE.policy);
236
237 const notObject = connect(as("u5", "eli"));
238 expect(await notObject.opened).toBe(true);
239 notObject.ws.send(JSON.stringify([1, 0, 0]));
240 expect((await notObject.closed).reason).toBe("not_an_object");
241
242 const noAxis = connect(as("u6", "fen"));
243 expect(await noAxis.opened).toBe(true);
244 noAxis.ws.send(JSON.stringify({ x: "left", y: 0, seq: 1 }));
245 expect((await noAxis.closed).reason).toBe("bad_axis");
246 });
247
248 // --------------------------------------------------------------- the cost
249
250 test("a normal 20 Hz steer passes, minute after minute", () => {
251 const heard: number[] = [];
252 const stop = onSteer((steering) => heard.push(steering.steer.seq));
253 const sock = fake("10.4.4.4");
254 websocket.open(sock);
255 // Two minutes of a thumb on the stick, at the rate the page will send:
256 // 2400 steers, 50 ms apart on the clock, and not one of them dropped.
257 let at = sock.data.heard;
258 for (let seq = 0; seq < 2400; seq++, at += 50)
259 receive(sock, JSON.stringify({ x: 0, y: 1, seq }), at);
260 expect(sock.data.dropped).toBe(0);
261 expect(sock.closes).toEqual([]);
262 expect(heard.length).toBe(2400);
263 stop();
264 sock.close();
265 });
266
267 test("a flood is dropped rather than queued, and the socket survives it", async () => {
268 const heard: number[] = [];
269 const stop = onSteer(() => heard.push(1));
270 const client = connect(as("u7", "gwen"));
271 expect(await client.opened).toBe(true);
272
273 for (let seq = 0; seq < 80; seq++)
274 client.ws.send(JSON.stringify({ x: 0, y: 1, seq }));
275 await until("the burst", () => heard.length >= 40);
276 await Bun.sleep(50);
277 // Half of it fit in the bucket; the rest was dropped where it landed,
278 // never queued, and the socket is still open and still steering.
279 expect(heard.length).toBeLessThan(80);
280 expect(heard.length).toBeGreaterThanOrEqual(40);
281 expect(client.ws.readyState).toBe(WebSocket.OPEN);
282
283 stop();
284 client.bye();
285 await client.closed;
286 });
287
288 test("a socket that keeps flooding is closed with 1008", async () => {
289 const client = connect(as("u8", "hal"));
290 expect(await client.opened).toBe(true);
291 for (let seq = 0; seq < 400; seq++)
292 client.ws.send(JSON.stringify({ x: 0, y: 1, seq }));
293 const end = await client.closed;
294 expect(end.code).toBe(CLOSE.policy);
295 expect(end.reason).toBe("too many messages");
296 });
297
298 // ---------------------------------------------------------------- a seat
299
300 test("a seat is a person, not a tab: the newest socket wins", async () => {
301 const first = connect(as("u9", "ida"));
302 expect(await first.opened).toBe(true);
303 expect(seatHolder("user:u9")).toBeDefined();
304
305 const second = connect(as("u9", "ida"));
306 expect(await second.opened).toBe(true);
307
308 const end = await first.closed;
309 expect(end.code).toBe(CLOSE.replaced);
310 expect(second.ws.readyState).toBe(WebSocket.OPEN);
311 // One person, one seat — and the seat is the survivor's, not the ghost's.
312 await until("the old socket to be forgotten", () => socketCount() === 1);
313 expect(seatHolder("user:u9")).toBeDefined();
314
315 second.bye();
316 await second.closed;
317 await until("an empty world", () => seatHolder("user:u9") === undefined);
318 });
319
320 // ------------------------------------------------------- frames going out
321
322 test("publish skips a backed-up socket and never waits for it", () => {
323 const quick = fake("10.5.5.5");
324 const slow = fake("10.5.5.6");
325 slow.buffered = SEND_MAX_BYTES + 1;
326 websocket.open(quick);
327 websocket.open(slow);
328
329 expect(backedUp(quick)).toBe(false);
330 expect(backedUp(slow)).toBe(true);
331 const out = publish("frame");
332 expect(out).toEqual({ sent: 1, skipped: 1 });
333 expect(quick.sent).toEqual(["frame"]);
334 expect(slow.sent).toEqual([]);
335 expect(slow.data.skipped).toBe(1);
336
337 // It drains, and it is fed again on the next frame without reconnecting.
338 slow.buffered = 0;
339 expect(publish("frame")).toEqual({ sent: 2, skipped: 0 });
340
341 quick.close();
342 slow.close();
343 });
344
345 test("a socket that fails to send is closed, and the rest still get the frame", () => {
346 const broken = fake("10.5.5.7");
347 broken.send = () => {
348 throw new Error("the peer went away mid-frame");
349 };
350 const fine = fake("10.5.5.8");
351 websocket.open(broken);
352 websocket.open(fine);
353
354 const out = publish("frame");
355 expect(out.sent).toBe(1);
356 expect(broken.closes[0]?.code).toBe(CLOSE.broken);
357 expect(fine.sent).toEqual(["frame"]);
358 expect(socketCount()).toBe(1);
359 fine.close();
360 });
361
362 test("a frame the round publishes reaches a real client", async () => {
363 const client = connect(as("u10", "jo"));
364 expect(await client.opened).toBe(true);
365 expect(publish("a frame").sent).toBe(1);
366 await until("the frame", () => client.got.length === 1);
367 expect(client.got[0]).toBe("a frame");
368 client.bye();
369 await client.closed;
370 });
371
372 // -------------------------------------------------------------- liveness
373
374 test("a quiet socket is pinged inside the idle timeout, a silent one is closed", () => {
375 const sock = fake("10.6.6.6");
376 websocket.open(sock);
377 const opened = sock.data.heard;
378
379 // Nothing yet: it was heard from a moment ago.
380 sweep(opened + 1_000);
381 expect(sock.pings).toBe(0);
382
383 // Quiet for half a minute: a ping goes out, well inside the server's 60 s
384 // idleTimeout, and one ping per interval rather than one per pass.
385 sweep(opened + 30_000);
386 sweep(opened + 31_000);
387 expect(sock.pings).toBe(1);
388
389 // The browser answers, and the clock starts again.
390 websocket.pong(sock);
391 sweep(sock.data.heard + SILENCE_MS - 1_000);
392 expect(sock.closes).toEqual([]);
393
394 // Nothing at all for ninety seconds: the peer is gone.
395 sweep(sock.data.heard + SILENCE_MS + 1);
396 expect(sock.closes[0]?.code).toBe(CLOSE.silent);
397 expect(socketCount()).toBe(0);
398 });
399
400 // ------------------------------------------------------------- the fakes
401
402 interface Fake extends Sock {
403 readonly sent: string[];
404 readonly closes: { code: number; reason: string }[];
405 pings: number;
406 buffered: number;
407 }
408
409 /**
410 * A socket the server would give us, without a server: the handlers take the
411 * part of `ServerWebSocket` they use, so a test can be one and turn the
412 * clock by hand. Closing behaves the way Bun behaves — the close handler
413 * runs — so the registry is exercised, not bypassed.
414 */
415 function fake(address: string, who: Who = ANON): Fake {
416 const seat = admit(upgradeHeaders({ origin: publicOrigin }), who, address);
417 if (!seat.ok) throw new Error(`admission refused: ${seat.why}`);
418 const data: SocketData = seat.data;
419 const sock: Fake = {
420 data,
421 sent: [],
422 closes: [],
423 pings: 0,
424 buffered: 0,
425 send(message) {
426 sock.sent.push(String(message));
427 return 1;
428 },
429 close(code = 1000, reason = "") {
430 sock.closes.push({ code, reason });
431 websocket.close(sock);
432 },
433 ping() {
434 sock.pings += 1;
435 return 1;
436 },
437 getBufferedAmount() {
438 return sock.buffered;
439 },
440 };
441 return sock;
442 }
443
444 /** The websocket handshake, spoken by hand, so the 101's headers can be read. */
445 async function shake(headers: Record<string, string>): Promise<string> {
446 let heard = "";
447 let done: (text: string) => void = () => {};
448 const answered = new Promise<string>((resolve) => (done = resolve));
449 const lines = Object.entries(headers).map(([k, v]) => `${k}: ${v}`);
450 const socket = await Bun.connect({
451 hostname: "127.0.0.1",
452 port: server.port,
453 socket: {
454 open(sock) {
455 sock.write(
456 [
457 "GET /ws HTTP/1.1",
458 `Host: 127.0.0.1:${server.port}`,
459 "Upgrade: websocket",
460 "Connection: Upgrade",
461 "Sec-WebSocket-Version: 13",
462 `Sec-WebSocket-Key: ${btoa("0123456789abcdef")}`,
463 ...lines,
464 "\r\n",
465 ].join("\r\n"),
466 );
467 },
468 data(_sock, bytes) {
469 heard += new TextDecoder().decode(bytes);
470 if (heard.includes("\r\n\r\n")) done(heard);
471 },
472 error() {
473 done(heard);
474 },
475 },
476 });
477 const answer = await answered;
478 socket.end();
479 return answer;
480 }
481
482 /** `count` sockets open from one address, for the cap. */
483 function fill(address: string, count: number): Fake[] {
484 const made: Fake[] = [];
485 for (let i = 0; i < count; i++) {
486 const sock = fake(address);
487 websocket.open(sock);
488 made.push(sock);
489 }
490 return made;
491 }
src/socket.ts added +410
@@ -0,0 +1,410 @@
1 /**
2 * The connection, its rules and its limits — everything about `GET /ws`
3 * except what travels on it. The frames are #BUS's; this module decides who
4 * may open a socket, what identity it carries for its whole life, what one
5 * inbound message may be, what a flood costs, and what happens when a slow
6 * client cannot keep up.
7 *
8 * The rules, in one place:
9 *
10 * origin an upgrade must *say* it came from us (`socketOrigin`), and
11 * a missing `Origin` is refused — a websocket has no preflight
12 * and `allow()` never sees it.
13 * identity fixed at upgrade time from the forge's signed headers. The
14 * socket remembers who it is; no inbound message ever names
15 * its own sender, so a client cannot steer someone else.
16 * message `parseSteer` from `protocol.ts` and nothing else. A named
17 * failure closes the socket with 1008 and throws nothing.
18 * ration a token bucket per socket: 20 Hz steering passes, a flood is
19 * dropped where it arrives rather than queued, and a socket
20 * that keeps flooding is closed.
21 * seat a seat is a person, not a tab: newest wins. A second socket
22 * on one player key closes the first with 4000, so one person
23 * holds one mote however many tabs they open.
24 * liveness a ping every 25 s (inside the server's 60 s `idleTimeout`),
25 * and a socket that has said nothing — message or pong — for
26 * 90 s is closed with 4001.
27 * backpressure `backedUp(ws)` is the named check `publish` makes per socket
28 * per frame: a backed-up send buffer is skipped, never
29 * awaited, so a slow client degrades alone.
30 *
31 * Nothing here runs a loop. `publish(payload)` is the door the round will
32 * drive at 10 Hz; today a test drives it.
33 */
34
35 import { randomBytes } from "node:crypto";
36 import { SECURITY, socketOrigin, text } from "./http.ts";
37 import { FRAME_MAX_BYTES, parseSteer, type Steer } from "./protocol.ts";
38 import type { Context } from "./routes.ts";
39 import type { Who } from "./who.ts";
40
41 /** How many steers a socket may spend at once, and how fast it earns more. */
42 const STEER_BURST = 40;
43 const STEER_PER_SECOND = 25;
44 /**
45 * What a dropped message costs. Every drop is a strike and every accepted
46 * message pays one back: a client that stutters over the cap is only ever
47 * dropped, and one that pushes hundreds of messages it was told it had no
48 * room for is not a client, it is a flood, and it loses the socket.
49 */
50 const FLOOD_STRIKES = 120;
51 /** Nothing heard for this long — not a steer, not a pong — and the peer is gone. */
52 export const SILENCE_MS = 90_000;
53 /** Well inside the 60 s `idleTimeout` in `src/server.ts`, like the SSE heartbeat. */
54 const PING_MS = 25_000;
55 const SWEEP_MS = 5_000;
56 /**
57 * A send buffer past eight frames is a client most of a second behind at
58 * 10 Hz. It is skipped until it drains: the round never waits on a browser.
59 */
60 export const SEND_MAX_BYTES = FRAME_MAX_BYTES * 8;
61 /**
62 * The outer wall on one inbound frame, wide enough that a steer over the
63 * wire cap still reaches `parseSteer` and earns an honest 1008, narrow
64 * enough that nothing large is ever assembled in memory for us.
65 */
66 const MAX_PAYLOAD_BYTES = 8 * 1024;
67 /**
68 * How many sockets one address may hold open at once. The seat rule bounds a
69 * person at one; this bounds everyone the app cannot tell apart, which behind
70 * the forge is every guest arriving from the forge's address — so it is a
71 * ceiling on this process, like `STREAM_MAX_SHARED` in `live.ts`, rather than
72 * a quota on anybody.
73 */
74 export const SOCKET_MAX_SHARED = 256;
75
76 /** Why a socket closed. 1008 is the protocol's word for "you broke a rule". */
77 export const CLOSE = {
78 /** A message the rules refuse, or a flood that would not stop. */
79 policy: 1008,
80 /** The same person opened another socket; this one is the older tab. */
81 replaced: 4000,
82 /** Nothing heard for `SILENCE_MS`. */
83 silent: 4001,
84 /** The send itself failed: our end, not theirs, and not a rule they broke. */
85 broken: 1011,
86 } as const;
87
88 /** What a socket knows about itself, decided once, at upgrade. */
89 export interface SocketData {
90 /** The identity the forge signed, or `anon`. Never anything a message said. */
91 readonly who: Who;
92 /** The seat: one player key, one mote. */
93 readonly key: string;
94 /** What the shared cap counts against. */
95 readonly address: string;
96 readonly openedAt: number;
97 /** When this socket was last heard from — a message or a pong. */
98 heard: number;
99 /** When it was last pinged, so the sweep pings at a rate, not every pass. */
100 pinged: number;
101 tokens: number;
102 filled: number;
103 strikes: number;
104 /** Counted for the tests, and for anyone asking why a client looks choppy. */
105 dropped: number;
106 skipped: number;
107 }
108
109 /** The part of a `ServerWebSocket` this module uses, so a test can be one. */
110 export interface Sock {
111 readonly data: SocketData;
112 send(message: string | Uint8Array): number;
113 close(code?: number, reason?: string): void;
114 ping(data?: string | Uint8Array): number;
115 getBufferedAmount(): number;
116 }
117
118 /** One person's intent, under the identity the *socket* carries. */
119 export interface Steering {
120 readonly key: string;
121 readonly who: Who;
122 readonly steer: Steer;
123 readonly at: number;
124 }
125
126 const live = new Set<Sock>();
127 const seats = new Map<string, Sock>();
128 const byAddress = new Map<string, number>();
129 const listeners = new Set<(steering: Steering) => void>();
130
131 /** Every socket open right now. The round reads this; nothing else writes it. */
132 export const sockets = (): ReadonlySet<Sock> => live;
133 export const socketCount = (): number => live.size;
134 /** How many sockets an address holds open, for the cap and for the tests. */
135 export const socketsFrom = (address: string): number =>
136 byAddress.get(address) ?? 0;
137 /** The socket sitting in a seat, if anybody is. */
138 export const seatHolder = (key: string): Sock | undefined => seats.get(key);
139
140 /**
141 * Steering, as it arrives. #BUS and the round listen; a listener is handed
142 * the socket's own identity, so no handler has to wonder who sent what.
143 */
144 export function onSteer(listen: (steering: Steering) => void): () => void {
145 listeners.add(listen);
146 return () => listeners.delete(listen);
147 }
148
149 /**
150 * The seat a request plays as. A person the forge named is their forge id,
151 * which is the same across tabs, reloads and reconnects — that is what makes
152 * a seat a person.
153 *
154 * A guest is the one thing this module cannot decide alone: a guest's
155 * identity is a cookie, and reading that cookie is `src/session.ts` in
156 * #DATA, which is not merged yet. Until it is, an anonymous socket is its
157 * own seat — every guest tab is its own connection — and the shared cap
158 * above is what bounds them. When #DATA lands this is one import and one
159 * line: `playerKey(request, who)`.
160 */
161 export const seatOf = (_request: Request, who: Who): string =>
162 who.role === "anon"
163 ? `anon:${randomBytes(12).toString("base64url")}`

Read line by line, run against main, and tried on the preview. The shape of this is right: socket.ts is one concept, the rules are written down where they are enforced, admit is pure enough to test, the clock is an argument so the 25 s ping and the 90 s silence close are tested without waiting, the ration drops where a message lands instead of queueing it, and publish skips a backed-up socket and closes a throwing one without touching the next. On the preview the origin rule holds for real: a socket with no Origin and a socket claiming https://evil.example both fail the handshake, GET /ws is a 426 with the whole floor on it, and a socket that opens stays quiet, exactly as the README says. Two things stop it landing.

1 (high) — a guest's seat is still a tab, and #19 is merged now. The issue's own line is "identity is fixed at upgrade time: who(request, key) plus playerKey from #19 … so one person cannot hold 24 of 24 seats". The pull skips it because src/session.ts was not merged when it was written — it is merged now (b98061e on main), and playerKey(request, who) is sitting there. As it stands seatOf mints anon:<random> per socket, so a guest is a new person on every tab. I opened five sockets to the preview carrying one and the same guest= cookie and all five held a seat; with ROSTER_MAX = 24 that is one guest, or one script with the right Origin header, holding the roster the moment #BUS gives seats motes — and guests are the direction's default player, not an edge case. Fix it the way the pull says: seatOf calls playerKey(request, who). The one case that needs a decision is a first visit, where playerKey returns undefined because it never mints: call session(request, who) there and pass its headers through the headers you already hand server.upgrade, so the 101 sets the cookie and the guest's second socket is the same person. A test for it belongs beside the existing seat test: two sockets, one guest cookie, the first closes with 4000.

2 (medium, but it breaks the default branch) — bun run test fails once main is merged in. The branch is behind main and predates #46, which made the check bunx tsc --noEmit && bun test. Merging main into this head and running the real check gives:

src/socket.test.ts(451,5): error TS2769: No overload matches this call.
  'hostname' does not exist in type 'UnixSocketOptions<undefined>'

server.port is not narrowed to number, so Bun.connect falls through to the unix overload. port: Number(server.port) in shake() clears it; with that one change the merged tree is 107 pass / 0 fail and tsc is silent. Merge main before the next push so the check you see is the check that will run.

Smaller things, none of them blocking: sockets() is exported and called by nothing, not even a test — let #BUS add it when it reads it, or hold it under test now. admit reads the per-address count but open is what increments it, so simultaneous handshakes can overshoot SOCKET_MAX_SHARED by a few; harmless today, worth a line of comment if it is deliberate. And one thing to confirm before #BUS leans on it: does the forge pass x-forge-identity/x-forge-signature on an upgrade request the way it does on a GET? If it does not, every socket is anon, the seat rule never fires for a signed-in person either, and the ladder never learns who swam — worth checking with a signed-in browser against this preview while it is up.

Fix the guest seat first; it is the one that changes what the game does.

164 : `user:${who.id}`;
165
166 export type Admission =
167 | { readonly ok: true; readonly data: SocketData }
168 | { readonly ok: false; readonly status: number; readonly why: string };
169
170 /**
171 * May this request open a socket, and as whom? Pure enough to test: it reads
172 * the request and the sockets already open, and changes neither.
173 */
174 export function admit(
175 request: Request,
176 who: Who,
177 address: string,
178 now = Date.now(),
179 ): Admission {
180 if ((request.headers.get("upgrade") ?? "").toLowerCase() !== "websocket")
181 return { ok: false, status: 426, why: "this endpoint speaks websocket" };
182 // Stricter than a POST, on purpose: `socketOrigin` refuses a missing
183 // `Origin` because a page on any site can open a socket to us with no
184 // preflight, and nothing before this point rationed the attempt.
185 if (!socketOrigin(request))
186 return { ok: false, status: 403, why: "forbidden" };
187 if (socketsFrom(address) >= SOCKET_MAX_SHARED)
188 return { ok: false, status: 503, why: "too many sockets" };
189 return {
190 ok: true,
191 data: {
192 who,
193 key: seatOf(request, who),
194 address,
195 openedAt: now,
196 heard: now,
197 pinged: now,
198 tokens: STEER_BURST,
199 filled: now,
200 strikes: 0,
201 dropped: 0,
202 skipped: 0,
203 },
204 };
205 }
206
207 /**
208 * `GET /ws`. Answers with a response when the upgrade is refused, and with
209 * nothing when the connection has been taken over — which is how `Bun.serve`
210 * is told this request is the websocket handler's now.
211 */
212 export function upgrade(ctx: Context): Response | undefined {
213 const seat = admit(ctx.request, ctx.who, ctx.address);
214 if (!seat.ok) return text(seat.why, seat.status);
215 // The 101 is a response like any other and carries the same floor: the
216 // handshake is the only header exchange a socket ever has.
217 if (
218 ctx.server?.upgrade(ctx.request, {
219 data: seat.data,
220 headers: { ...SECURITY },
221 }) === true
222 )
223 return undefined;
224 return text("upgrade failed", 400);
225 }
226
227 /** The handlers `Bun.serve` hangs the socket off. One row in `routes.ts` points here. */
228 export const websocket = {
229 maxPayloadLength: MAX_PAYLOAD_BYTES,
230 // Bun drops a socket that sees no traffic for this long; the sweep's ping
231 // is well inside it, so only a connection that has really gone is dropped.
232 idleTimeout: 60,
233 perMessageDeflate: false,
234
235 open(ws: Sock): void {
236 const { key, address } = ws.data;
237 // A seat is a person, not a tab: newest wins. The older socket is closed
238 // with its own code so a client can tell "you opened me somewhere else"
239 // from "you broke a rule", and a reconnect after a deploy takes the seat
240 // back rather than fighting a ghost for it.
241 const held = seats.get(key);
242 if (held !== undefined && held !== ws) {
243 seats.delete(key);
244 close(held, CLOSE.replaced, "replaced by a newer connection");
245 }
246 seats.set(key, ws);
247 live.add(ws);
248 byAddress.set(address, socketsFrom(address) + 1);
249 start();
250 },
251
252 message(ws: Sock, message: string | Uint8Array): void {
253 receive(ws, message, Date.now());
254 },
255
256 pong(ws: Sock): void {
257 ws.data.heard = Date.now();
258 },
259
260 close(ws: Sock): void {
261 live.delete(ws);
262 // Only if the seat is still this socket's: a replaced socket closes
263 // *after* the new one took the seat, and must not take it away again.
264 if (seats.get(ws.data.key) === ws) seats.delete(ws.data.key);
265 const left = socketsFrom(ws.data.address) - 1;
266 if (left > 0) byAddress.set(ws.data.address, left);
267 else byAddress.delete(ws.data.address);
268 if (live.size === 0) stop();
269 },
270 };
271
272 /**
273 * One inbound message: rationed, then validated, then handed on. The clock
274 * is an argument so a test can play a real minute of steering in a
275 * millisecond; `websocket.message` passes the real one.
276 */
277 export function receive(
278 ws: Sock,
279 message: string | Uint8Array,
280 now = Date.now(),
281 ): void {
282 ws.data.heard = now;
283 // Over the ration a message is dropped where it arrives — not queued, not
284 // answered, not even parsed. A client that keeps spending what it does not
285 // have is a flood and loses the socket.
286 if (!afford(ws.data, now)) {
287 ws.data.dropped += 1;
288 ws.data.strikes += 1;
289 if (ws.data.strikes > FLOOD_STRIKES)
290 close(ws, CLOSE.policy, "too many messages");
291 return;
292 }
293 // The only validation there is. `parseSteer` returns a named failure and
294 // never throws, and neither does this: a bad message costs the sender
295 // their socket and costs the round nothing.
296 const parsed = parseSteer(message);
297 if (!parsed.ok) return close(ws, CLOSE.policy, parsed.error);
298 if (ws.data.strikes > 0) ws.data.strikes -= 1;
299 // The identity is the socket's, fixed at upgrade. A message that names a
300 // player names nothing: `parseSteer` keeps three numbers and drops the rest.
301 const steering: Steering = {
302 key: ws.data.key,
303 who: ws.data.who,
304 steer: parsed.value,
305 at: now,
306 };
307 for (const listen of listeners)
308 try {
309 listen(steering);
310 } catch (error) {
311 console.error(error);
312 }
313 }
314
315 /**
316 * Is this socket behind? The named check `publish` makes per socket per
317 * frame, and the one #BUS calls if it sends anything of its own. A send
318 * buffer that has not drained means the client is slower than the round,
319 * and the answer is to skip it, never to wait for it.
320 */
321 export const backedUp = (ws: Sock): boolean => buffered(ws) > SEND_MAX_BYTES;
322
323 /**
324 * One payload to every open socket. Never awaited and never throwing: a
325 * backed-up socket is skipped and a failing one is closed, and either way
326 * the next socket still gets its frame.
327 */
328 export function publish(payload: string | Uint8Array): {
329 sent: number;
330 skipped: number;
331 } {
332 let sent = 0;
333 let skipped = 0;
334 for (const ws of live) {
335 if (backedUp(ws)) {
336 ws.data.skipped += 1;
337 skipped += 1;
338 continue;
339 }
340 try {
341 ws.send(payload);
342 sent += 1;
343 } catch {
344 close(ws, CLOSE.broken, "send failed");
345 }
346 }
347 return { sent, skipped };
348 }
349
350 /**
351 * Liveness, on a clock a test can turn by hand: ping anything quiet, close
352 * anything not heard from in `SILENCE_MS`. A watcher who never steers stays
353 * open as long as its browser answers the pings, which every browser does.
354 */
355 export function sweep(now = Date.now()): void {
356 for (const ws of live) {
357 if (now - ws.data.heard > SILENCE_MS) {
358 close(ws, CLOSE.silent, "silent");
359 continue;
360 }
361 if (now - ws.data.pinged < PING_MS) continue;
362 ws.data.pinged = now;
363 try {
364 ws.ping();
365 } catch {
366 close(ws, CLOSE.silent, "ping failed");
367 }
368 }
369 }
370
371 // The sweep runs only while somebody is connected, and never holds the
372 // process open: an idle app has no timers, and `bun test` exits.
373 let sweeper: ReturnType<typeof setInterval> | undefined;
374 function start(): void {
375 if (sweeper !== undefined) return;
376 sweeper = setInterval(() => sweep(), SWEEP_MS);
377 sweeper.unref?.();
378 }
379 function stop(): void {
380 if (sweeper === undefined) return;
381 clearInterval(sweeper);
382 sweeper = undefined;
383 }
384
385 /** The token bucket, per socket, refilled by the clock rather than by a timer. */
386 function afford(data: SocketData, now: number): boolean {
387 data.tokens = Math.min(
388 STEER_BURST,
389 data.tokens + ((now - data.filled) / 1000) * STEER_PER_SECOND,
390 );
391 data.filled = now;
392 if (data.tokens < 1) return false;
393 data.tokens -= 1;
394 return true;
395 }
396
397 const buffered = (ws: Sock): number => {
398 try {
399 return ws.getBufferedAmount();
400 } catch {
401 return 0;
402 }
403 };
404
405 /** Closing never throws: a socket that has already gone is gone. */
406 function close(ws: Sock, code: number, reason: string): void {
407 try {
408 ws.close(code, reason);
409 } catch {}
410 }

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs