trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns15g rEventsFlows

Proper web architecture: HTML is small, cacheable and streamed; everything else is a long-lived asset #8104

Merged torquecrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/forge:torque/8016-proper-web-architecture-html-is-small-ca → main 9 commits fork

  • Merged. torque/8016-proper-web-architecture-html-is-small-ca was merged into main as 585be43.

Review

Sign in to review this change.
40 files changed +732 −230
README.md +1
@@ -440,6 +440,7 @@ Once a minute the forge measures its landing rate, names the one cause binding i
440440 - [`docs/DEVELOP.md`](docs/DEVELOP.md) — developing the forge: tests, checks, the gate, lint and types.
441441 - [`docs/STATUS.md`](docs/STATUS.md) — what works today, one bullet per feature.
442442 - [`docs/ON-YOUR-MACHINE.md`](docs/ON-YOUR-MACHINE.md) — the `forge` command on your machine: the local copy, offline work and building the binary.
443 - [`docs/web.md`](docs/web.md) — the rules for assets and HTML: what is cached where, how a page streams, and the byte budgets a test enforces.
443444 This page is the front door. Everything else is a file of its own under
444445 `docs/`, each about one thing:
445446
docs/web.md added +33
@@ -0,0 +1,33 @@
1 # The web, as the forge ships it
2
3 Short rules. `test/web-budget.test.ts` counts the bytes and fails when one is broken.
4
5 ## What is an asset
6
7 Anything that is the same bytes for everyone and changes only when the code does: the stylesheet, the shell and beacon scripts, each page's own script, the icon sprite (`/assets/i-<hash>.svg`), the speculation rules. Each is named by a hash of its content (`web/asset.ts`) and sent `public, max-age=31536000, immutable`. Nothing inline: a page never carries a drawing or a script. Runtime data reaches a page script through `data-*` attributes. A page script is also kept under `<dataDir>/page-scripts/` the first time a process renders it, so after a restart an edge copy or a cached page that names it is still answered; the body read back must hash to its name, and a file nothing has rendered for 30 days is pruned.
8
9 ## What is HTML
10
11 Markup only, and small. It names its assets; it never carries them. Budget: **home under 40 KB raw**, no inline `<script>` (a `type="application/json"` data block is data), no inline `<symbol>`. The two inline `<style>` blocks (the boot theme and a page's own) are admitted by hash in the policy, not by a nonce, so a page is the same bytes for every visitor and can be cached.
12
13 ## How each is cached
14
15 - Assets: immutable, a year.
16 - HTML, signed in or carrying any identity (cookie, `Authorization`, edge login headers) or a flash: `no-cache`, no ETag, never shared (`http/edge_cache.ts`, `http/respond.ts`).
17 - HTML, signed out, on a page that is the same for everyone (home, explore, the front page of a repository that exists and is public; any other path, two segments or not, is `no-cache`): `public, max-age=0, s-maxage=60, stale-while-revalidate=300` and a weak ETag; a revisit costs a 304. The shared copy carries `Vary: Cookie, Authorization, X-Plat-User, X-Plat-Edge-Token`, so an edge keys it on identity: a visitor with any of those misses it and reaches the origin, which answers `no-cache` and never the anonymous page. There is no purge call (the edge's credentials are not the forge's to hold), so staleness is bounded by those 60 + 300 seconds.
18
19 ## The script policy
20
21 `script-src 'self'` admits any same-origin script URL, so no route that serves user-controlled bytes may answer with a script type. Raw files are `text/plain` (or `application/octet-stream`) under `nosniff` and `default-src 'none'; sandbox`; media never serves HTML or script; the only script-typed responses are the forge's own hashed files under `/assets/`, whose code is static (`test/web-budget.test.ts` checks the raw route and that an unknown `/assets/page-*.js` is a 404). Page scripts are kept by name for the life of the process, never evicted, because a cached page may name one for days.
22
23 ## How a page streams
24
25 The response carries `Link: rel=preload` for the stylesheet and scripts, which an edge turns into 103 Early Hints, so the browser fetches them before the body arrives. The `<head>` comes first in the body, with the stylesheet link before any script. Scripts are `defer`.
26
27 ## Next pages and the back button
28
29 The `Speculation-Rules` header points at one hashed rules file: prefetch, moderate eagerness (hover or pointer-down), never the API, assets, sign-in and sign-out, `/-/confirm`, claim, first-admin and verify links, `/go/`, settings, app-open, MCP, raw files, media or archives, never a URL whose query carries a token, code, key, secret or state, and never a link marked `nofollow`, `download` or `data-no-prefetch`. Back/forward cache stays on: no page sends `no-store`, and no script listens for `unload` or `beforeunload`.
30
31 ## Not done
32
33 Streaming the body in chunks (the render is one string today); purge by event; the pages other than home, explore and a repository's front page measure the same budgets as a mechanical follow-up. Before/after numbers from #7559 are not producible from this sandbox.
src/http/edge_cache.ts added +43
@@ -0,0 +1,43 @@
1 /**
2 * Which signed-out pages a shared cache (the edge) may keep, and for how long
3 * (docs/web.md). The answer is `public` only for a request that carries no
4 * identity at all — no cookie, no Authorization, no edge-vouched login — on a
5 * page that is the same bytes for every visitor. Everything else is
6 * `no-cache`: revalidate, never share. `privateWhenSignedIn`
7 * (`http/respond.ts`) is the second lock on the same door.
8 *
9 * There is no purge call: the edge's own API credentials are not the forge's
10 * to hold, so staleness is bounded by the lifetime below instead.
11 */
12
13 /** Seconds a shared cache serves the page without asking. */
14 export const EDGE_FRESH = 60;
15 /** Seconds past that it may serve the old copy while it refetches. */
16 export const EDGE_STALE = 300;
17
18 /** What a shared copy must be keyed on: a request with any of these is never served the anonymous copy. */
19 export const EDGE_VARY = "Cookie, Authorization, X-Plat-User, X-Plat-Edge-Token";
20
21 /** Whether `owner/name` is an existing public repository. */
22 export type IsPublicRepo = (owner: string, name: string) => boolean;
23
24 /**
25 * The pages whose signed-out body is the same for everyone: home, explore,
26 * and the front page of a repository that exists and is public. Any other
27 * path, however it is shaped (`/apps/new`, `/settings/x`), is not shared.
28 */
29 export function sharedPath(pathname: string, isPublicRepo: IsPublicRepo): boolean {
30 if (pathname === "/" || pathname === "/explore") return true;
31 const m = /^\/([^/]+)\/([^/]+)$/.exec(pathname);
32 return m !== null && isPublicRepo(m[1]!, m[2]!);
33 }
34
35 /** The `Cache-Control` of a signed-out, flash-free 200 for `pathname`. */
36 export function sharedCache(request: Request, pathname: string, isPublicRepo: IsPublicRepo): string {
37 const h = request.headers;
38 const anonymous =
39 !h.has("cookie") && !h.has("authorization") && !h.has("x-plat-user") && !h.has("x-plat-edge-token");
40 return anonymous && sharedPath(pathname, isPublicRepo)
41 ? `public, max-age=0, s-maxage=${EDGE_FRESH}, stale-while-revalidate=${EDGE_STALE}`
42 : "no-cache";
43 }
src/http/etag.ts +4 −18
@@ -4,19 +4,12 @@
44 * body's bytes; a request holding it is answered 304 with the page's own
55 * headers.
66 *
7 * A page names a nonce in its body and in its policy. Left random, the 304
8 * would renew the policy over a cached body that holds the old nonce and the
9 * page's scripts would stop. So a page that is eligible takes its nonce from
10 * an HMAC, under a key that lives only in this process, of the body it
11 * guards: the same bytes give the same nonce, any byte an attacker reflects
12 * into the page gives one they cannot predict. Signed-in pages, any that
13 * show a flash, and any that are not 200 keep a random nonce and no ETag.
7 * A page carries no nonce (its scripts are hashed files, its inline styles
8 * are named by hash), so the same bytes are the same page and a 304 renews
9 * nothing the cached body disagrees with. Signed-in pages, any that show a
10 * flash, and any that are not 200 keep no ETag.
1411 */
1512
16 import { secret } from "../kernel/id.ts";
17
18 const key = new TextEncoder().encode(secret());
19
2013 /** A page may be tagged only when nothing in it is for one person. */
2114 export function taggable(request: Request, signedIn: boolean, flash: boolean): boolean {
2215 return (
@@ -26,13 +19,6 @@ export function taggable(request: Request, signedIn: boolean, flash: boolean): b
2619 );
2720 }
2821
29 /** The nonce a page's markup (written with `placeholder`) is bound to. */
30 export function nonceFor(markup: string, placeholder: string): string {
31 const h = new Bun.CryptoHasher("sha256", key);
32 h.update(markup.replaceAll(placeholder, ""));
33 return h.digest("base64url");
34 }
35
3622 export function weakEtag(bytes: string): string {
3723 return `W/"${Bun.hash(bytes).toString(36)}"`;
3824 }
src/http/server.ts +13
@@ -110,6 +110,10 @@ import {
110110 assetResponse,
111111 highlightAsset,
112112 beaconAsset,
113 fileResponse,
114 pageScriptNamed,
115 speculationAsset,
116 spriteAsset,
113117 shellAsset,
114118 stylesheetAsset,
115119 stylesheetRedirect,
@@ -731,10 +735,19 @@ export function buildRouter(): Router<Handler> {
731735 if (ctx.params.name === beaconAsset().name)
732736 return assetResponse(ctx.request, beaconAsset());
733737 if (ctx.params.name === TOOLBAR_NAME) return toolbarResponse(ctx.request);
738 const own = pageScriptNamed(ctx.params.name!, ctx.app.config.dataDir);
739 if (own !== undefined) return fileResponse(ctx.request, own);
734740 const asset = await highlightAsset();
735741 if (asset === null || ctx.params.name !== asset.name) return notFound();
736742 return assetResponse(ctx.request, asset);
737743 });
744 // The icon sprite and the speculation rules: hashed, immutable, one file each.
745 router.add("GET", "/assets/:name.svg", (ctx) =>
746 ctx.params.name === spriteAsset().name ? fileResponse(ctx.request, spriteAsset()) : notFound(),
747 );
748 router.add("GET", "/assets/:name.json", (ctx) =>
749 ctx.params.name === speculationAsset().name ? fileResponse(ctx.request, speculationAsset()) : notFound(),
750 );
738751 // The service worker: root scope needs a root path, and it must never be
739752 // cached by the browser's HTTP cache or an update would wait a day.
740753 router.add("GET", "/sw.js", serviceWorker);
src/web/asset.ts +181
@@ -16,11 +16,14 @@
1616 * JavaScript switched off.
1717 */
1818
19 import { existsSync, mkdirSync, readdirSync, readFileSync, renameSync, rmSync, statSync, utimesSync, writeFileSync } from "node:fs";
20 import { join } from "node:path";
1921 import { sha256 } from "../kernel/hash.ts";
2022 import { encodeOnce, MIN_BYTES, negotiate } from "../http/compress.ts";
2123 import { createLog } from "../kernel/log.ts";
2224 import { beaconJs } from "./beacon.ts";
2325 import { shellJs } from "./client.ts";
26 import { spriteSvg } from "./icons.ts";
2427 import { css } from "./style.ts";
2528
2629 const log = createLog("web");
@@ -167,3 +170,181 @@ export function beaconAsset(): BuiltAsset {
167170 }
168171 return beacon;
169172 }
173
174 /**
175 * Small files a page names, content-addressed the same way: the icon sprite,
176 * the speculation rules, and each page's own script. `name` is the basename
177 * without the extension, so one route per extension finds it.
178 */
179 export interface FileAsset extends BuiltAsset {
180 type: string;
181 }
182
183 function fileAsset(prefix: string, ext: string, body: string, type: string): FileAsset {
184 const hash = sha256(body);
185 const name = `${prefix}-${hash.slice(0, 16)}`;
186 return { path: `/assets/${name}.${ext}`, name, code: body, etag: `"${hash}"`, type };
187 }
188
189 let sprite: FileAsset | null = null;
190
191 /** Every icon, once: `<use href="/assets/i-<hash>.svg#i-name">` points here. */
192 export function spriteAsset(): FileAsset {
193 sprite ??= fileAsset("i", "svg", spriteSvg(), "image/svg+xml; charset=utf-8");
194 return sprite;
195 }
196
197 /**
198 * Where the browser may prefetch ahead of a click. Moderate eagerness: on
199 * hover or pointer-down, never on sight. Only plain GET pages — nothing under
200 * the API, the assets, the sign-in edge, the MCP or a raw file, and nothing a
201 * link marks `download` or `rel=nofollow`.
202 */
203 /**
204 * Paths a hover must never fetch: not a page of reading but a GET that
205 * signs in or out, confirms, claims, redeems, redirects or opens something
206 * (`/-/confirm`, `/claim`, `/first-admin`, `/verify`, `/go/<code>`, ...),
207 * plus everything that is a file, an API or an edge hand-off.
208 */
209 export const NO_PREFETCH_PATHS = [
210 "/assets/*", "/api/*", "/_plat/*", "/_measure*", "/mcp*", "/oauth/*", "/sw.js",
211 "/*/*/raw/*", "/*/*/media/*", "/*/*/archive/*", "/*/*/app/*/open",
212 "/logout*", "/signout*", "/sign-out*", "/signin*", "/sign-in*", "/login*",
213 "/claim*", "/first-admin*", "/verify*", "/go/*", "/-/*", "/apps/open*",
214 "/settings/*", "/home/*", "/*/-/*",
215 ];
216
217 /** Query strings that carry a secret or a one-time action, whatever the path. */
218 export const NO_PREFETCH_QUERY = [
219 "*token=*", "*code=*", "*key=*", "*secret=*", "*sig=*", "*nonce=*", "*confirm=*", "*csrf=*", "*session=*",
220 "*invite=*", "*otp=*", "*password=*", "*state=*",
221 ];
222
223 const SPECULATION = JSON.stringify({
224 prefetch: [
225 {
226 where: {
227 and: [
228 { href_matches: "/*" },
229 { not: { href_matches: NO_PREFETCH_PATHS } },
230 { not: { href_matches: NO_PREFETCH_QUERY.map((search) => ({ search })) } },
231 { not: { selector_matches: "[rel~=nofollow], [download], [data-no-prefetch]" } },
232 ],
233 },
234 eagerness: "moderate",
235 },
236 ],
237 });
238
239 let speculation: FileAsset | null = null;
240
241 export function speculationAsset(): FileAsset {
242 speculation ??= fileAsset("speculation", "json", SPECULATION, "application/speculationrules+json");
243 return speculation;
244 }
245
246 // The set of page scripts is static (the code is trusted, data rides in
247 // `data-*`), so these maps are never evicted: a cached page or an edge copy
248 // may name a script for as long as it lives, and it must still be served.
249 const pageScripts = new Map<string, FileAsset>();
250 const pageScriptsByName = new Map<string, FileAsset>();
251
252 /**
253 * A page's own script, as a file. The code is trusted and static (runtime
254 * data rides in `data-*` attributes), so the same page names the same file
255 * on every request and the browser keeps it for a year. Wrapped to wait for
256 * DOMContentLoaded, which fires after the deferred shell has run.
257 *
258 * Given the data dir, the first render in a process also keeps the file under
259 * `<dataDir>/page-scripts/`, so a process that has not rendered that page yet
260 * (a restart, a new deploy) still answers an edge copy or a cached page that
261 * names it. Scripts are composed per page from many constants, so there is no
262 * list to register at boot; the disk is the list.
263 */
264 export function pageScriptAsset(js: string, dataDir?: string): FileAsset {
265 const body = `document.addEventListener("DOMContentLoaded",()=>{${js}\n})`;
266 const key = sha256(body);
267 let asset = pageScripts.get(key);
268 if (asset === undefined) {
269 asset = fileAsset("page", "js", body, "text/javascript; charset=utf-8");
270 remember(key, asset);
271 }
272 if (dataDir !== undefined) keep(dataDir, asset);
273 return asset;
274 }
275
276 function remember(key: string, asset: FileAsset): void {
277 pageScripts.set(key, asset);
278 pageScriptsByName.set(asset.name, asset);
279 }
280
281 const PAGE_SCRIPT_NAME = /^page-[0-9a-f]{16}$/;
282 /** A kept script nothing has rendered for this long is dropped (edge copies live minutes). */
283 export const PAGE_SCRIPT_KEEP_MS = 30 * 24 * 60 * 60 * 1000;
284 const kept = new Set<string>();
285 const pruned = new Set<string>();
286
287 export const pageScriptDir = (dataDir: string): string => join(dataDir, "page-scripts");
288
289 /** Write once per data dir per process; touch it if it is there, so pruning spares what is in use. */
290 function keep(dataDir: string, asset: FileAsset): void {
291 const id = `${dataDir}\0${asset.name}`;
292 if (kept.has(id)) return;
293 kept.add(id);
294 const dir = pageScriptDir(dataDir);
295 const file = join(dir, `${asset.name}.js`);
296 try {
297 mkdirSync(dir, { recursive: true });
298 if (!pruned.has(dir)) {
299 pruned.add(dir);
300 prune(dir);
301 }
302 if (existsSync(file)) {
303 const now = new Date();
304 utimesSync(file, now, now);
305 return;
306 }
307 const tmp = `${file}.${process.pid}.tmp`;
308 writeFileSync(tmp, asset.code);
309 renameSync(tmp, file);
310 } catch (error) {
311 // Not fatal: this process still serves it from memory.
312 log.warn("web.page_script_keep_failed", { error: String(error) });
313 }
314 }
315
316 function prune(dir: string): void {
317 const cutoff = Date.now() - PAGE_SCRIPT_KEEP_MS;
318 for (const entry of readdirSync(dir)) {
319 const file = join(dir, entry);
320 try {
321 if (statSync(file).mtimeMs < cutoff) rmSync(file, { force: true });
322 } catch {
323 // Raced with another writer; the next boot looks again.
324 }
325 }
326 }
327
328 /**
329 * A page script by its hashed name: rendered by this process, or kept on disk
330 * by an earlier one. The name is checked before it touches the path, and the
331 * body read back must hash to it, or it is not served.
332 */
333 export function pageScriptNamed(name: string, dataDir?: string): FileAsset | undefined {
334 const known = pageScriptsByName.get(name);
335 if (known !== undefined || dataDir === undefined || !PAGE_SCRIPT_NAME.test(name)) return known;
336 let body: string;
337 try {
338 body = readFileSync(join(pageScriptDir(dataDir), `${name}.js`), "utf8");
339 } catch {
340 return undefined;
341 }
342 const asset = fileAsset("page", "js", body, "text/javascript; charset=utf-8");
343 if (asset.name !== name) return undefined;
344 remember(sha256(body), asset);
345 return asset;
346 }
347
348 export function fileResponse(request: Request, asset: FileAsset): Response {
349 return staticResponse(request, asset.code, asset.type, asset.etag);
350 }
src/web/icons.ts +21 −45
@@ -13,14 +13,12 @@
1313 * Icons are decorative: they carry `aria-hidden`, and any control that shows
1414 * only an icon must carry its own `aria-label`.
1515 *
16 * A drawing is sent once per page, not once per use. `icon()` renders the
17 * 16px slot with a one-line `<use>` inside it; the page's markup is read for
18 * those references and `iconSprite()` emits one `<symbol>` per distinct icon
19 * the page actually used, at the end of `<body>` (`layout.ts`) — legal, and
20 * resolved fine by a `<use>` earlier in the document. The stroke, the fill
21 * and the viewBox ride on the `<symbol>`, so the slot itself is four
22 * attributes; `currentColor` still resolves against whatever colour the
23 * `<use>` sits in, which is what keeps every theme working.
16 * A drawing is sent once per browser, not once per page. `icon()` renders the
17 * 16px slot with a one-line `<use>` inside it; `layout.ts` aims every
18 * reference at the one hashed sprite file (`spriteSvg()`, served immutable).
19 * The stroke, the fill and the viewBox ride on the `<symbol>`, so the slot
20 * itself is four attributes; `currentColor` still resolves against whatever
21 * colour the `<use>` sits in, which is what keeps every theme working.
2422 *
2523 * A fragment (`layout.fragment()`) is markup dropped into a document it does
2624 * not own, so it cannot reference a sprite it did not ship: `inlineIcons()`
@@ -117,51 +115,29 @@ export function icon(name: IconName, size: 16 | 20 = 16): Html {
117115 /** The reference `icon()` leaves behind, and the only thing read back. */
118116 const REFERENCE = /<use href="#i-([A-Za-z]+)"\/>/g;
119117
120 /** What `REFERENCE` starts with, so a page can be searched for it by `indexOf`. */
118 /** What `icon()` writes before the name; `pointAtSprite` finds it by this. */
121119 const REFERENCE_OPEN = '<use href="#i-';
122120
123121 /**
124 * Every icon this markup references, once each, in first-seen order. Found
125 * with `indexOf`, which skips through a long page far faster than a regular
126 * expression does, and accepts exactly what `REFERENCE` accepts: letters,
127 * then `"/>`.
122 * Every drawing the forge has, as one SVG document of `<symbol>`s. It is
123 * served once, hashed, immutable (`/assets/i-<hash>.svg`, `web/asset.ts`), so
124 * no page carries a drawing: a page carries four attributes and a `<use>`.
128125 */
129 export function referenced(markup: string): IconName[] {
130 const names: IconName[] = [];
131 let at = markup.indexOf(REFERENCE_OPEN);
132 while (at !== -1) {
133 const start = at + REFERENCE_OPEN.length;
134 let end = start;
135 while (end < markup.length) {
136 const c = markup.charCodeAt(end) | 32;
137 if (c < 97 || c > 122) break;
138 end++;
139 }
140 if (end > start && markup.startsWith('"/>', end)) {
141 const name = markup.slice(start, end);
142 if (name in PATHS && !names.includes(name as IconName))
143 names.push(name as IconName);
144 }
145 at = markup.indexOf(REFERENCE_OPEN, end);
146 }
147 return names;
126 export function spriteSvg(): string {
127 const symbols = (Object.keys(PATHS) as IconName[])
128 .map((name) => `<symbol id="i-${name}" ${DRAW}>${PATHS[name]}</symbol>`)
129 .join("");
130 return `<svg xmlns="http://www.w3.org/2000/svg">${symbols}</svg>`;
148131 }
149132
150133 /**
151 * The drawings a page used, defined once each. Sat at the end of `<body>`
152 * (`layout.ts`), zero-sized by `.sprite` so it can never move a layout;
153 * `<symbol>` content is never rendered where it is defined, only where a
154 * `<use>` instantiates it.
134 * A document's markup with every `<use href="#i-name"/>` aimed at the hashed
135 * sprite file at `spritePath`. Same-origin, so `img-src 'self'` and
136 * `connect-src 'self'` both admit it; `currentColor` still resolves against
137 * the `<use>`'s own colour, so every theme works.
155138 */
156 export function iconSprite(markup: string): Html {
157 const names = referenced(markup);
158 if (names.length === 0) return raw("");
159 const symbols = names
160 .map((name) => `<symbol id="i-${name}" ${DRAW}>${PATHS[name]}</symbol>`)
161 .join("");
162 return raw(
163 `<svg class="sprite" width="0" height="0" aria-hidden="true" focusable="false">${symbols}</svg>`,
164 );
139 export function pointAtSprite(markup: string, spritePath: string): string {
140 return markup.replaceAll(REFERENCE_OPEN, `<use href="${spritePath}#i-`);
165141 }
166142
167143 /**
src/web/layout.ts +55 −34
@@ -24,8 +24,8 @@ import type { Ctx } from "../http/context.ts";
2424 import { activeEdgeToken, edgeLoginUrl, edgeLogoutUrl, edgeStrangerLogin } from "../http/edge.ts";
2525 import { json } from "../http/respond.ts";
2626 import { sha256 } from "../kernel/hash.ts";
27 import { holds, nonceFor, taggable, weakEtag } from "../http/etag.ts";
28 import { secret } from "../kernel/id.ts";
27 import { holds, taggable, weakEtag } from "../http/etag.ts";
28 import { EDGE_VARY, sharedCache } from "../http/edge_cache.ts";
2929 import { runningLine } from "../kernel/version.ts";
3030 import {
3131 bootCss,
@@ -34,7 +34,14 @@ import {
3434 THEMES,
3535 type ThemeName,
3636 } from "../design/themes.ts";
37 import { beaconAsset, shellAsset, stylesheetAsset } from "./asset.ts";
37 import {
38 beaconAsset,
39 pageScriptAsset,
40 shellAsset,
41 speculationAsset,
42 spriteAsset,
43 stylesheetAsset,
44 } from "./asset.ts";
3845 import { linkHeader, pageLinks } from "./standards.ts";
3946 import {
4047 clearFlashCookie,
@@ -47,7 +54,7 @@ import { notePhase } from "../core/route_timing.ts";
4754 import { html, raw, when, type Html } from "./html.ts";
4855 import { noRoom } from "../spawn/door.ts";
4956 import { quotaFull } from "../spawn/quota.ts";
50 import { icon, iconSprite, inlineIcons, themeGlyph } from "./icons.ts";
57 import { icon, inlineIcons, pointAtSprite, themeGlyph } from "./icons.ts";
5158 import { btn, copyField, kbd, menu, type MenuItem } from "./ui.ts";
5259 import { SITE_LINE } from "./site_line.ts";
5360
@@ -632,20 +639,17 @@ export function tabTitle(title: string): string {
632639 }
633640
634641 export function page(options: PageOptions, status = 200): Response {
635 // A nonce lets exactly this page's <style> and <script> run while
636 // everything else stays forbidden; it is random, or bound to the body when
637 // the page is tagged (http/etag.ts).
638 const placeholder = secret();
642 // No nonce: a script is a hashed file the policy admits as 'self', and an
643 // inline <style> is admitted by the hash of its own text, so the same page
644 // is the same bytes for everyone and can be cached (docs/web.md).
639645 const { ctx } = options;
640646 const theme = ctx.theme ?? options.theme;
641647 const cookieFlash = readFlash(ctx.request);
642648 const flash = options.flash ?? cookieFlash;
643649 const links = pageLinks(ctx);
644650
645 // The body's markup, built before the document so the icons it used can be
646 // counted: every `icon()` left a `<use>` behind, and the sprite defining
647 // them goes last, after the markup that references it (legal, and resolved
648 // once the document is parsed). Only what this page used ships.
651 // The body's markup. Every `icon()` left a `<use>` behind; they are aimed
652 // at the one hashed sprite file below, so no page carries a drawing.
649653 const shell = html`${options.bare ? null : topbar(ctx, theme, options.crumbs)}
650654 <main
651655 class="${options.bare
@@ -660,6 +664,8 @@ export function page(options: PageOptions, status = 200): Response {
660664 ${options.bare ? null : footer(ctx)} ${overlays()}
661665 ${paletteIndex(ctx, options.palette ?? [])}`;
662666
667 const own = options.scriptJs ? pageScriptAsset(options.scriptJs, ctx.app.config.dataDir) : undefined;
668
663669 const doc = html`<!doctype html>
664670 <html lang="en" ${theme ? html`data-theme="${theme}"` : null}>
665671 <head>
@@ -746,13 +752,12 @@ export function page(options: PageOptions, status = 200): Response {
746752 rel="icon"
747753 href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Cdefs%3E%3ClinearGradient id='g' x1='0' y1='0' x2='1' y2='1'%3E%3Cstop offset='0' stop-color='%232563eb'/%3E%3Cstop offset='1' stop-color='%237c3aed'/%3E%3C/linearGradient%3E%3C/defs%3E%3Crect x='3' y='3' width='26' height='26' rx='8' fill='url(%23g)'/%3E%3C/svg%3E"
748754 />
749 ${raw(`<style nonce="${placeholder}">${bootCss}</style>`)}
755 ${raw(`<style>${bootCss}</style>`)}
750756 <link rel="stylesheet" href="${stylesheetAsset().path}" />
751757 ${when(
752758 options.turnstile,
753759 () =>
754760 html`<script
755 nonce="${placeholder}"
756761 src="${TURNSTILE_ORIGIN}/turnstile/v0/api.js"
757762 async
758763 defer
@@ -763,41 +768,57 @@ export function page(options: PageOptions, status = 200): Response {
763768 (src) => html`<script type="module" src="${src}"></script>`,
764769 )}
765770 ${options.styleCss
766 ? raw(`<style nonce="${placeholder}">${options.styleCss}</style>`)
771 ? raw(`<style>${options.styleCss}</style>`)
767772 : null}
768773 </head>
769774 <body>
770 ${shell}${iconSprite(shell.text)}
771 <script nonce="${placeholder}" src="${shellAsset().path}" defer></script>
772 <script nonce="${placeholder}" src="${beaconAsset().path}" defer></script>
773 ${options.scriptJs
774 ? raw(
775 `<script nonce="${placeholder}">document.addEventListener("DOMContentLoaded",()=>{${options.scriptJs}\n})</script>`,
776 )
777 : null}
775 ${shell}
776 <script src="${shellAsset().path}" defer></script>
777 <script src="${beaconAsset().path}" defer></script>
778 ${own ? html`<script src="${own.path}" defer></script>` : null}
778779 </body>
779780 </html>`;
780781
781 // Signed-out, flash-free pages bind their nonce to their own bytes so a
782 // 304 can renew nothing the cached body disagrees with (`http/etag.ts`).
782 // Signed-out, flash-free pages are tagged by their own bytes so a revisit
783 // costs a 304 (`http/etag.ts`).
783784 const tagged = taggable(ctx.request, ctx.auth.user !== undefined, flash !== undefined) && status === 200;
784 const bound = tagged ? nonceFor(doc.text, placeholder) : placeholder;
785 const text = tagged ? doc.text.replaceAll(placeholder, bound) : doc.text;
785 const text = pointAtSprite(doc.text, spriteAsset().path);
786 const styleSources = [bootCss, ...(options.styleCss ? [options.styleCss] : [])]
787 .map((css) => `'sha256-${new Bun.CryptoHasher("sha256").update(css).digest("base64")}'`)
788 .join(" ");
789 const cacheControl = tagged
790 ? sharedCache(ctx.request, ctx.url.pathname, (o, n) => getRepo(ctx.app.db, o, n)?.visibility === "public")
791 : "no-cache";
786792 const headers = new Headers({
787793 "content-type": "text/html; charset=utf-8",
788 // The external stylesheet plus this page's own nonce'd <style>/<script>
789 // are the only resources allowed; connect-src 'self' lets the live
790 // event stream (EventSource) and fragment fetches reach same-origin.
791 "content-security-policy": `default-src 'none'; style-src 'self' 'nonce-${bound}'; script-src 'nonce-${bound}'${options.scriptAsset === undefined ? "" : " 'self'"}${options.turnstile ? ` ${TURNSTILE_ORIGIN}` : ""}; connect-src 'self'${options.turnstile ? ` ${TURNSTILE_ORIGIN}` : ""}; ${options.turnstile ? `frame-src ${TURNSTILE_ORIGIN}; ` : ""}worker-src 'self'; manifest-src 'self'; img-src 'self' data:; ${options.objectSrc ? "object-src 'self'; " : ""}form-action 'self'; base-uri 'none'; frame-ancestors 'none'`,
794 // The hashed stylesheet and scripts are 'self'; the two inline <style>s
795 // are named by hash. connect-src 'self' lets the live event stream
796 // (EventSource) and fragment fetches reach same-origin.
797 "content-security-policy": `default-src 'none'; style-src 'self' ${styleSources}; script-src 'self'${options.turnstile ? ` ${TURNSTILE_ORIGIN}` : ""}; connect-src 'self'${options.turnstile ? ` ${TURNSTILE_ORIGIN}` : ""}; ${options.turnstile ? `frame-src ${TURNSTILE_ORIGIN}; ` : ""}worker-src 'self'; manifest-src 'self'; img-src 'self' data:; ${options.objectSrc ? "object-src 'self'; " : ""}form-action 'self'; base-uri 'none'; frame-ancestors 'none'`,
792798 "referrer-policy": "same-origin",
793799 "x-content-type-options": "nosniff",
794800 // Markup is revalidated every time; only the hashed assets it names are
795 // cached, so no cache can pair new HTML with an old stylesheet.
796 "cache-control": "no-cache",
797 link: linkHeader(links),
801 // cached, so no cache can pair new HTML with an old stylesheet. A
802 // signed-out public page may also sit at the edge for a minute
803 // (`sharedCache`); a signed-in one never does.
804 "cache-control": cacheControl,
805 // The assets the head names, as preloads: an edge turns them into 103
806 // Early Hints, so the browser fetches them before the body arrives.
807 link: [
808 linkHeader(links),
809 `<${stylesheetAsset().path}>; rel=preload; as=style`,
810 `<${shellAsset().path}>; rel=preload; as=script`,
811 ...(own ? [`<${own.path}>; rel=preload; as=script`] : []),
812 ]
813 .filter((l) => l !== "")
814 .join(", "),
815 "speculation-rules": `"${speculationAsset().path}"`,
798816 });
799817 // A flash is shown exactly once: rendering it is also what clears it.
800818 if (cookieFlash) headers.append("set-cookie", clearFlashCookie(ctx.url, ctx.app.config.url));
819 // A shared copy is keyed on identity, so a request that carries any is a
820 // miss at the edge and never sees the anonymous page (docs/web.md).
821 if (cacheControl.startsWith("public")) headers.set("vary", EDGE_VARY);
801822 if (tagged) {
802823 const etag = weakEtag(text);
803824 headers.set("etag", etag);
test/api-pulls-check-preview.test.ts +10 −4
@@ -117,12 +117,18 @@ test("a preview address only where an environment exists, and only for those who
117117 const address = runtime.urlFor(record(), `pull-${pulls["green"]!.number}`);
118118
119119 // A public app: every reader gets the address for the pull that has an environment, null for the rest.
120 const heldNow = () => new Set(runtime.status(record()).map((e) => e.env as string));
121 const before = heldNow();
120122 const open = await listed();
123 const after = heldNow();
121124 expect(open["green"]!["preview"]).toBe(address);
122 // The runtime may have brought some others up itself; an address is given exactly where one exists.
123 const held = new Set(runtime.status(record()).map((e) => e.env));
124 for (const item of Object.values(open))
125 expect(item["preview"] === null).toBe(!held.has(`pull-${item["number"]}` as never));
125 // The runtime brings others up (and drops them) on its own while the request runs; an address is
126 // given exactly where one exists, judged on the pulls whose environment held steady across it.
127 for (const item of Object.values(open)) {
128 const env = `pull-${item["number"]}`;
129 if (before.has(env) !== after.has(env)) continue;
130 expect(item["preview"] === null).toBe(!after.has(env));
131 }
126132
127133 // A private app on a public repository: a guest and a plain reader get none; the writer does.
128134 app.db.query("UPDATE app SET access = 'private' WHERE repo_id = ?1").run(repoId);
test/board.test.ts +3 −2
@@ -30,6 +30,7 @@ import { createUser, getUserByName, type User } from "../src/core/users.ts";
3030 import { serve, type Servers } from "../src/http/server.ts";
3131 import { boardCss } from "../src/web/pages/board.ts";
3232 import { meter } from "../src/web/ui.ts";
33 import { withScripts } from "./helpers/page-script.ts";
3334
3435 describe("the board", () => {
3536 let dataDir: string;
@@ -151,8 +152,8 @@ describe("the board", () => {
151152 expect(html).toMatch(/\$0\.00<\/b> of\s+\$20/);
152153 expect(html).toContain('id="board-live"');
153154 expect(html).toContain('id="board-clock"');
154 expect(html).toContain("new EventSource('/events')");
155 expect(html).toContain("requestFullscreen");
155 expect(await withScripts(servers.url, html)).toContain("new EventSource('/events')");
156 expect(await withScripts(servers.url, html)).toContain("requestFullscreen");
156157 // A wall left open all day fetches its 13 KB of rules once, with the rest
157158 // of the design, instead of carrying them inline on every reload. The one
158159 // style element every page carries is the boot style that paints the
test/confirm-app-stop.test.ts +2 −1
@@ -18,6 +18,7 @@ import type { Repo } from "../src/core/repos.ts";
1818 import { createUser } from "../src/core/users.ts";
1919 import { serve, type Servers } from "../src/http/server.ts";
2020 import { promote } from "./helpers/members.ts";
21 import { withScripts } from "./helpers/page-script.ts";
2122
2223 const PASSWORD = "a-good-passphrase";
2324 const TO = "/alice/notes/app/production/stop";
@@ -172,7 +173,7 @@ describe("Stop on an app's environment goes through the confirm page", () => {
172173 test("a reader is never offered Stop", async () => {
173174 putLive();
174175 const html = await (await get(bob, "/alice/notes")).text();
175 expect(html).toContain("serving");
176 expect(await withScripts(base, html)).toContain("serving");
176177 expect(html).not.toContain(encodeURIComponent(TO));
177178 });
178179 });
test/etag.test.ts +10 −7
@@ -58,14 +58,17 @@ test("a signed-in page carries no ETag, and If-None-Match never makes it a 304",
5858 expect(res.headers.has("etag")).toBe(false);
5959 });
6060
61 test("the nonce follows the body: stable for the same page, different once the page differs", async () => {
61 test("no page carries a nonce: the same page is the same bytes, and its inline style is named by hash", async () => {
6262 const a = await get("/signin");
63 const b = await get("/signin");
64 expect(a.headers.get("content-security-policy")).toBe(b.headers.get("content-security-policy"));
65 const nonce = /nonce-([\w-]+)/.exec(a.headers.get("content-security-policy")!)![1]!;
66 expect(await a.text()).toContain(`nonce="${nonce}"`);
67 const other = await get("/explore");
68 expect(other.headers.get("content-security-policy")).not.toBe(a.headers.get("content-security-policy"));
63 const csp = a.headers.get("content-security-policy")!;
64 expect(csp).toBe((await get("/signin")).headers.get("content-security-policy")!);
65 expect(csp).not.toContain("nonce");
66 expect(csp).toContain("script-src 'self'");
67 const body = await a.text();
68 expect(body).not.toContain("nonce=");
69 const style = /<style>([\s\S]*?)<\/style>/.exec(body)![1]!;
70 const hash = new Bun.CryptoHasher("sha256").update(style).digest("base64");
71 expect(csp).toContain(`'sha256-${hash}'`);
6972 });
7073
7174 test("a signed-out page showing a flash carries no ETag and is never a 304", async () => {
test/feed-page.test.ts +2 −1
@@ -16,6 +16,7 @@ import { serve, type Servers } from "../src/http/server.ts";
1616 import { readFeed } from "../src/feed/feed.ts";
1717 import { feedCard } from "../src/web/studio/feed.ts";
1818 import { promote } from "./helpers/members.ts";
19 import { withScripts } from "./helpers/page-script.ts";
1920
2021 describe("the feed on the home page", () => {
2122 let dataDir: string;
@@ -153,6 +154,6 @@ describe("the feed on the home page", () => {
153154 test("the page carries the feed script and a card is focusable for j/k", async () => {
154155 const text = await (await get("/feed", aliceCookie)).text();
155156 expect(text).toContain("data-nav");
156 expect(text).toContain("/api/v1/feed/interactions");
157 expect(await withScripts(servers.url, text)).toContain("/api/v1/feed/interactions");
157158 });
158159 });
test/first-admin.test.ts +2 −1
@@ -40,6 +40,7 @@ import { setSignup } from "../src/settings/signup.ts";
4040 import { readClaim, spawnedOrg } from "../src/spawn/claim.ts";
4141 import { quietLogs } from "./helpers/globals.ts";
4242 import { passkeyFields } from "./helpers/passkey.ts";
43 import { withScripts } from "./helpers/page-script.ts";
4344
4445 quietLogs();
4546
@@ -296,7 +297,7 @@ describe("a person: /first-admin with a passkey", () => {
296297 test("the page holds no code and takes it from the fragment", async () => {
297298 const html = await (await fetch(`${servers.url}/first-admin`)).text();
298299 expect(html).not.toContain(code);
299 expect(html).toContain("location.hash");
300 expect(await withScripts(servers.url, html)).toContain("location.hash");
300301 expect(html).toContain('name="token"');
301302 });
302303
test/helpers/page-script.ts added +16
@@ -0,0 +1,16 @@
1 /**
2 * A page's own script is a hashed file (`web/asset.ts`), not inline text.
3 * A test that asserts what the page's script does reads the file the page names.
4 */
5 export async function pageScriptOf(base: string, html: string): Promise<string> {
6 const src = /<script src="(\/assets\/page-[0-9a-f]+\.js)" defer><\/script>/.exec(html)?.[1];
7 if (src === undefined) return "";
8 return await (await fetch(base + src)).text();
9 }
10
11 /** The page's markup followed by every script file it names: what the browser would run. */
12 export async function withScripts(base: string, html: string): Promise<string> {
13 const srcs = [...html.matchAll(/<script src="(\/assets\/[\w.-]+\.js)"/g)].map((m) => m[1]!);
14 const code = await Promise.all(srcs.map(async (s) => (await fetch(base + s)).text()));
15 return html + "\n" + code.join("\n");
16 }
test/issue-teams-doors.test.ts +2 −1
@@ -34,6 +34,7 @@ import { local } from "../src/tui/screens/where.ts";
3434 import { headlessTerminal } from "../src/tui/term/terminal.ts";
3535 import { quietLogs } from "./helpers/globals.ts";
3636 import { settleApp } from "./helpers/tui.ts";
37 import { withScripts } from "./helpers/page-script.ts";
3738
3839 quietLogs();
3940
@@ -301,7 +302,7 @@ describe("naming teams, at every door", () => {
301302 // The page's timeline carries it, and its live client listens for it.
302303 const page = await (await fetch(`${base()}/issues/${issue.number}`, { headers: { cookie: cookie(alice) } })).text();
303304 expect(page).toContain("changed the teams");
304 expect(page).toContain("'issue.teams'");
305 expect(await withScripts(servers.url, page)).toContain("'issue.teams'");
305306 // A reader of the terminal who cannot write still sees the teams.
306307 const terminal = headlessTerminal(140, 40);
307308 const { app: tui, session } = createForgeApp(
test/live-pages.test.ts +7 −6
@@ -16,6 +16,7 @@ import { createApp, type App } from "../src/app.ts";
1616 import { createRepo } from "../src/core/repos.ts";
1717 import { createUser } from "../src/core/users.ts";
1818 import { serve, type Servers } from "../src/http/server.ts";
19 import { withScripts } from "./helpers/page-script.ts";
1920
2021 describe("live browsing pages", () => {
2122 let dataDir: string;
@@ -57,19 +58,19 @@ describe("live browsing pages", () => {
5758 const html = await (await fetch(`${base}/lex/proj`)).text();
5859 expect(html).toContain('id="live"');
5960 expect(html).toContain('data-repo="/lex/proj"');
60 expect(html).toContain("new EventSource");
61 expect(await withScripts(servers.url, html)).toContain("new EventSource");
6162 });
6263
6364 test("the commit history wires up the live client", async () => {
6465 const html = await (await fetch(`${base}/lex/proj/commits`)).text();
6566 expect(html).toContain('id="live"');
66 expect(html).toContain("new EventSource");
67 expect(await withScripts(servers.url, html)).toContain("new EventSource");
6768 });
6869
6970 test("the branches page wires up the live client", async () => {
7071 const html = await (await fetch(`${base}/lex/proj/branches`)).text();
7172 expect(html).toContain('id="live"');
72 expect(html).toContain("new EventSource");
73 expect(await withScripts(servers.url, html)).toContain("new EventSource");
7374 });
7475
7576 test("the studio app page wires up the live client with data-app, so a deploy run event refetches the app's own fragment", async () => {
@@ -103,11 +104,11 @@ describe("live browsing pages", () => {
103104 ).text();
104105 expect(html).toContain('id="live"');
105106 expect(html).toContain('data-app="1"');
106 expect(html).toContain("new EventSource");
107 expect(await withScripts(servers.url, html)).toContain("new EventSource");
107108 // `app.serving` fires at the swap-in, seconds to half a minute before
108109 // the deploy run's own `run.finished` — the client must listen for it
109110 // directly, not only learn a preview is live once its tests are done.
110 expect(html).toContain("app.serving");
111 expect(await withScripts(servers.url, html)).toContain("app.serving");
111112 // The polite region a swap must speak into — the one place a screen
112113 // reader learns a preview just went live.
113114 expect(html).toContain('id="live-status"');
@@ -162,7 +163,7 @@ describe("live browsing pages", () => {
162163 ).text();
163164 expect(html).toContain('id="live-apps"');
164165 expect(html).not.toContain('id="live-on-you"');
165 expect(html).toContain("new EventSource('/events')");
166 expect(await withScripts(servers.url, html)).toContain("new EventSource('/events')");
166167 });
167168
168169 test("a live region never swaps out from under a card that is mid-decision", async () => {
test/live.test.ts +2 −1
@@ -14,6 +14,7 @@ import { createUser, type User } from "../src/core/users.ts";
1414 import { liveClientJs } from "../src/web/live.ts";
1515 import { serve, type Servers } from "../src/http/server.ts";
1616 import { promote } from "./helpers/members.ts";
17 import { withScripts } from "./helpers/page-script.ts";
1718
1819 async function readEvents(
1920 res: Response,
@@ -133,7 +134,7 @@ describe("the viewer's stream", () => {
133134 const page = await res.text();
134135 expect(page).toContain('id="live-apps"');
135136 expect(page).toContain('id="live-apps"');
136 expect(page).toContain("new EventSource('/events')");
137 expect(await withScripts(servers.url, page)).toContain("new EventSource('/events')");
137138 });
138139
139140 test("a grant to run a pull reaches a page watching that pull", async () => {
test/map.test.ts +3 −2
@@ -33,6 +33,7 @@ import {
3333 type Layout,
3434 } from "../src/web/pages/map.ts";
3535 import { promote } from "./helpers/members.ts";
36 import { withScripts } from "./helpers/page-script.ts";
3637
3738 // ---- the layout, on paper ----
3839
@@ -412,8 +413,8 @@ describe("the map, per app and across the forge", () => {
412413 /<div[^>]*class="map-canvas"[^>]*id="map-canvas"[^>]*data-w="\d+"[^>]*data-h="\d+"/,
413414 );
414415 expect(page).toContain('data-map="fit"');
415 expect(page).toContain("new EventSource('/events')");
416 expect(page).toContain("forge:swapped");
416 expect(await withScripts(servers.url, page)).toContain("new EventSource('/events')");
417 expect(await withScripts(servers.url, page)).toContain("forge:swapped");
417418 // A card is a link to its issue, so it takes keyboard focus.
418419 expect(page).toMatch(
419420 /<a[^>]*class="map-node"[^>]*href="\/alice\/engine\/issues\/1"/,
test/measure.test.ts +1 −1
@@ -44,7 +44,7 @@ describe("the page-view beacon", () => {
4444 expect(await script.text()).toBe(beaconJs);
4545 const home = await fetch(base + "/");
4646 expect(await home.text()).toContain(asset.path);
47 expect(home.headers.get("content-security-policy")).toContain("script-src 'nonce-");
47 expect(home.headers.get("content-security-policy")).toContain("script-src 'self'");
4848 expect(home.headers.get("content-security-policy")).not.toContain("unsafe");
4949 });
5050
test/migrate.test.ts +2 −1
@@ -66,6 +66,7 @@ import { STAGE_BADGE_JS } from "../src/web/stage-row.ts";
6666 import { prefill } from "../src/web/pages/migrate-prefill.ts";
6767 import { quietLogs } from "./helpers/globals.ts";
6868 import { promote } from "./helpers/members.ts";
69 import { withScripts } from "./helpers/page-script.ts";
6970
7071 quietLogs();
7172
@@ -1418,7 +1419,7 @@ describe("migrate", () => {
14181419 expect(page).not.toContain("stage-dot");
14191420 expect(page).not.toContain("stage-detail");
14201421 // The page's script is built from the table the rows are drawn from.
1421 expect(page).toContain(`var badges = ${STAGE_BADGE_JS};`);
1422 expect(await withScripts(servers.url, page)).toContain(`var badges = ${STAGE_BADGE_JS};`);
14221423 expect(page).toMatch(/<li data-stage-name="creating" data-state="done">[\s\S]*?class="row"[\s\S]*?>Done</);
14231424 expect(page).toMatch(/data-stage-name="fetching" data-state="now">[\s\S]*?receiving 61%[\s\S]*?>Running</);
14241425 expect(page).toMatch(/data-stage-name="crew" data-state="todo">[\s\S]*?>Waiting</);
test/offline.test.ts +7 −4
@@ -15,6 +15,7 @@ import { listIssues } from "../src/core/issues.ts";
1515 import { serve, type Servers } from "../src/http/server.ts";
1616 import { outboxJs } from "../src/web/outbox.ts";
1717 import { serviceWorker, swSource } from "../src/web/sw.ts";
18 import { withScripts } from "./helpers/page-script.ts";
1819
1920 type Send = (url: string, init: any) => Promise<{ status: number }>;
2021 const make = new Function(`${outboxJs}; return forgeOutbox;`)() as (
@@ -158,16 +159,18 @@ describe("the service worker and the server", () => {
158159 test("the home page registers the worker, has the note, and allows workers", async () => {
159160 const res = await fetch(`${servers.url}/feed`, { headers: { cookie } });
160161 const text = await res.text();
161 expect(text).toContain("serviceWorker");
162 const all = await withScripts(servers.url, text);
163 expect(all).toContain("serviceWorker");
162164 expect(text).toContain('id="feed-note"');
163 expect(text).toContain("forgeOutbox");
165 expect(all).toContain("forgeOutbox");
164166 expect(res.headers.get("content-security-policy")).toContain("worker-src 'self'");
165167 });
166168
167169 test("the live client resumes /events from the last seen id", async () => {
168170 const text = await (await fetch(`${servers.url}/feed`, { headers: { cookie } })).text();
169 expect(text).toContain("/events?since=");
170 expect(text).toContain("'online'");
171 const all = await withScripts(servers.url, text);
172 expect(all).toContain("/events?since=");
173 expect(all).toContain("'online'");
171174 });
172175
173176 test("a tell sent twice with one key makes one issue", async () => {
test/pull-checks-line-route.test.ts +3 −2
@@ -15,6 +15,7 @@ import { createUser, type User } from "../src/core/users.ts";
1515 import { serve, type Servers } from "../src/http/server.ts";
1616 import { appendRunLog } from "../src/runs/log.ts";
1717 import { quietLogs } from "./helpers/globals.ts";
18 import { withScripts } from "./helpers/page-script.ts";
1819
1920 quietLogs();
2021
@@ -138,7 +139,7 @@ test("the page that draws the line also ships the script that drives it", async
138139 expect(text).not.toMatch(/id="live-checks"[^>]*data-live-every/);
139140 // liveRegionsJs reads data-live-src and arms the beat; liveClientJs alone does not.
140141 expect(text).toContain("data-live-src");
141 expect(text).toContain("function arm(");
142 expect(await withScripts(servers.url, text)).toContain("function arm(");
142143 });
143144
144145 test("once the check has passed, the line has no beat, no counts, and reads as passed", async () => {
@@ -164,5 +165,5 @@ test("the change page the owner lands on draws the same live line, with the coun
164165 expect(text).toContain('id="live-checks"');
165166 expect(text).not.toMatch(/id="live-checks"[^>]*data-live-every/);
166167 expect(text).toContain("2 of 170 files · 500 passed, 0 failed");
167 expect(text).toContain("function arm(");
168 expect(await withScripts(servers.url, text)).toContain("function arm(");
168169 });
test/releases-page.test.ts +1
@@ -172,6 +172,7 @@ describe("the releases page", () => {
172172 expect(body).toContain('href="/releases"');
173173 // An admin's tile, or the plain line anyone else gets: both name the live version.
174174 expect(body).toMatch(/live: <span class="sha">aaaaaaa|Live version: a{7}/);
175 expect(body).toContain("aaaaaaa");
175176 });
176177
177178 test("a signed-out visitor is sent to sign in", async () => {
test/review-act.test.ts +5 −3
@@ -16,6 +16,7 @@ import { serve, type Servers } from "../src/http/server.ts";
1616 import { id } from "../src/kernel/id.ts";
1717 import { orderedAsks, rowId } from "../src/web/studio/review_act.ts";
1818 import { quietLogs } from "./helpers/globals.ts";
19 import { withScripts } from "./helpers/page-script.ts";
1920
2021 quietLogs();
2122
@@ -201,10 +202,11 @@ test("a cross-origin post, a stranger, and a bad verb change nothing", async ()
201202 test("the key script ships only when there are asks, and sanitises the hash it focuses", async () => {
202203 const n = seeded("Keyed");
203204 const withAsks = await (await fetch(`${servers.url}/review`, { headers: { cookie } })).text();
204 expect(withAsks).toContain("m: 'merge'");
205 expect(withAsks).toContain("location.hash.replace(/[^#a-z0-9-]/g, '')");
205 const asksAll = await withScripts(servers.url, withAsks);
206 expect(asksAll).toContain("m: 'merge'");
207 expect(asksAll).toContain("location.hash.replace(/[^#a-z0-9-]/g, '')");
206208 for (const a of orderedAsks(app.db, alice)) await act({ number: String(numberOf(a)), verb: "close" });
207209 expect(n).toBeGreaterThan(0);
208210 const empty = await (await fetch(`${servers.url}/review`, { headers: { cookie } })).text();
209 expect(empty).not.toContain("m: 'merge'");
211 expect(await withScripts(servers.url, empty)).not.toContain("m: 'merge'");
210212 });
test/studio-door.test.ts +2 −1
@@ -50,6 +50,7 @@ import { liveClientJs } from "../src/web/live.ts";
5050 import { createUser, getUserByName, type User } from "../src/core/users.ts";
5151 import { serve, type Servers } from "../src/http/server.ts";
5252 import { promote } from "./helpers/members.ts";
53 import { withScripts } from "./helpers/page-script.ts";
5354
5455 let dataDir: string;
5556 let app: App;
@@ -595,7 +596,7 @@ describe("the studio home keeps itself current", () => {
595596 const home = await res.text();
596597 expect(home).toMatch(/id="live-apps"\s*\n?\s*data-live="/);
597598 expect(home).not.toContain("live-on-you");
598 expect(home).toContain("new EventSource('/events')");
599 expect(await withScripts(servers.url, home)).toContain("new EventSource('/events')");
599600 });
600601
601602 test("chips do not touch Your apps: it lists apps, not feed items", async () => {
test/studio-runs.test.ts +9 −6
@@ -57,6 +57,7 @@ import {
5757 } from "../src/web/studio/words.ts";
5858 import { repoDir } from "../src/git/repos.ts";
5959 import { appendRunLog } from "../src/runs/log.ts";
60 import { withScripts } from "./helpers/page-script.ts";
6061
6162 describe("the studio's slot count", () => {
6263 let dataDir: string;
@@ -603,9 +604,10 @@ describe("working: a run's own page in the studio", () => {
603604 appendRunLog(app.config.dataDir, run.id, "first");
604605 appendRunLog(app.config.dataDir, run.id, "second");
605606
606 const page = await (
607 await get(`/alice/engine/runs/${run.id}`, writerCookie)
608 ).text();
607 const page = await withScripts(
608 base,
609 await (await get(`/alice/engine/runs/${run.id}`, writerCookie)).text(),
610 );
609611 // Every EventSource the page's HTML and script open is /events.
610612 const opened = [...page.matchAll(/new EventSource\(([^,;]*)/g)].map(
611613 (m) => m[1]!,
@@ -957,9 +959,10 @@ describe("working: a run's own page in the studio", () => {
957959 writerCookie,
958960 )
959961 ).text();
960 expect(script).toContain("getElementById('check-steps')");
961 expect(script).toContain("\\(fail\\) ");
962 expect(script).toContain("['role', 'tabindex', 'aria-label']");
962 const scripts = await withScripts(base, script);
963 expect(scripts).toContain("getElementById('check-steps')");
964 expect(scripts).toContain("\\(fail\\) ");
965 expect(scripts).toContain("['role', 'tabindex', 'aria-label']");
963966 const plain = await createRepo(app, alice, {
964967 name: "plain",
965968 visibility: "public",
test/studio-search.test.ts +2 −1
@@ -38,6 +38,7 @@ import {
3838 import { createUser, getUserByName, type User } from "../src/core/users.ts";
3939 import { serve, type Servers } from "../src/http/server.ts";
4040 import { promote } from "./helpers/members.ts";
41 import { withScripts } from "./helpers/page-script.ts";
4142
4243 describe("finding things in the studio: /search", () => {
4344 let dataDir: string;
@@ -797,7 +798,7 @@ describe("the studio's Stop and Redeploy come back to the page they were pressed
797798 test("the head card shows the new state where it stands: the live fragment, no second navigation", async () => {
798799 putLive();
799800 const serving = await studioPage();
800 expect(serving).toContain("serving");
801 expect(await withScripts(base, serving)).toContain("serving");
801802 expect(serving).toContain('href="/-/confirm?to=%2Falice%2Fnotes%2Fapp%2Fproduction%2Fstop"');
802803 // The page a person stays on is the one that refreshes itself: its live
803804 // mount asks for the app's state on a deploy event.
test/studio-words.test.ts +2 −2
@@ -533,8 +533,8 @@ describe("dev's vocabulary never reaches a studio page", () => {
533533 word: "app",
534534 path: repoPath,
535535 title: "vocab",
536 shows: [`href="#i-issue"/></svg>Issues`, `href="#i-pull"/></svg>Pulls`],
537 hides: [`href="#i-eye"/></svg>Look`, `>Look<`, 'class="where"'],
536 shows: [`#i-issue"/></svg>Issues`, `#i-pull"/></svg>Pulls`],
537 hides: [`#i-eye"/></svg>Look`, `>Look<`, 'class="where"'],
538538 // #630: the app page is the home in every mode; its code, with dev's
539539 // where strip, is the Files step one tab over.
540540 },
test/surfaces-adversarial.test.ts +4 −7
@@ -305,21 +305,18 @@ describe("surfaces adversarial: web", () => {
305305 for (const path of ["/apps", "/apps/new", "/settings/crew"]) {
306306 const response = await web(alice, path);
307307 const csp = response.headers.get("content-security-policy") ?? "";
308 expect(`${path}: ${csp}`).toContain("script-src 'nonce-");
308 expect(`${path}: ${csp}`).toContain("script-src 'self';");
309309 expect(csp).not.toContain("unsafe-inline");
310310 const body = await response.text();
311 const nonceMatch = /nonce-([A-Za-z0-9+/=_-]+)/.exec(csp);
312 expect(nonceMatch).not.toBeNull();
313 // Every *executable* <script> tag carries exactly that nonce — a
311 expect(csp).not.toContain("nonce");
312 // Every *executable* <script> is a file on this origin — a
314313 // `type="application/json"` data island is inert and CSP's script-src
315314 // never gates it, so it is excluded rather than asserted on.
316315 const scripts = [...body.matchAll(/<script\b[^>]*>/g)].filter(
317316 ([tag]) => !/type="application\/json"/.test(tag),
318317 );
319318 expect(scripts.length).toBeGreaterThan(0);
320 for (const [tag] of scripts) {
321 expect(tag).toContain(`nonce="${nonceMatch![1]}"`);
322 }
319 for (const [tag] of scripts) expect(tag).toMatch(/ src="\/assets\/[a-z]+-[0-9a-f]+\.js"/);
323320 // No inline event handler, no interpolated style attribute.
324321 expect(body).not.toMatch(/\son[a-z]+="/i);
325322 }
test/system-stream.test.ts +2 −1
@@ -11,6 +11,7 @@ import { eventStream, sseDroppedCount } from "../src/http/sse.ts";
1111 import { createUser } from "../src/core/users.ts";
1212 import { serve, type Servers } from "../src/http/server.ts";
1313 import { createLive, useLive } from "../src/system/live.ts";
14 import { withScripts } from "./helpers/page-script.ts";
1415
1516 describe("/system over server-sent events", () => {
1617 let dataDir: string;
@@ -56,7 +57,7 @@ describe("/system over server-sent events", () => {
5657 expect(body).not.toContain("location.reload");
5758 expect(body).not.toContain("setTimeout");
5859 expect(body).not.toContain("setInterval");
59 expect(body).toContain("new EventSource('/system/stream')");
60 expect(await withScripts(servers.url, body)).toContain("new EventSource('/system/stream')");
6061 // First paint is server-rendered with the latest sample.
6162 expect(body).toContain('id="summary"');
6263 expect(body).toMatch(/id="summary"[^>]*>\s*<[^>]*>CPU /);
test/thread-render-timing.test.ts −28
@@ -3,7 +3,6 @@ import { afterAll, beforeAll, expect, test } from "bun:test";
33 import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts";
44 import { addComment, openIssue } from "../src/actions/issues.ts";
55 import { listEvents } from "../src/core/events.ts";
6 import { inlineIcons, referenced } from "../src/web/icons.ts";
76 import { markdownMemoStats, renderMarkdown } from "../src/web/markdown.ts";
87 import { seedBody } from "./thread-seed.ts";
98
@@ -80,30 +79,3 @@ test("listEvents with kinds is the unfiltered window, filtered", () => {
8079 }
8180 expect(listEvents(f.app.db, { repoId, limit: 500, kinds: ["issue.commented"] }).length).toBe(200);
8281 });
83
84 test("referenced() finds the icons the old pattern did", () => {
85 const old = (markup: string): string[] => {
86 const names: string[] = [];
87 for (const m of markup.matchAll(/<use href="#i-([A-Za-z]+)"\/>/g)) {
88 const name = m[1]!;
89 if (inlineIcons(`<use href="#i-${name}"/>`) !== `<use href="#i-${name}"/>` && !names.includes(name)) names.push(name);
90 }
91 return names;
92 };
93 const use = (name: string): string => `<use href="#i-${name}"/>`;
94 const cases = [
95 "",
96 use("comment") + use("issue") + use("comment"),
97 use("Comment") + use("COMMENT"),
98 '<use href="#i-comment">',
99 '<use href="#i-comment"',
100 '<use href="#i-comment2"/>',
101 '<use href="#i-"/>',
102 use("nosuchicon") + use("clock"),
103 '<use href="#i-<use href="#i-clock"/>',
104 "x" + use("clock") + "<use href=\"#i-" ,
105 use("constructor") + use("bot"),
106 ];
107 for (const markup of cases) expect([...referenced(markup)] as string[]).toEqual(old(markup));
108 expect(referenced(use("comment") + use("issue")).length).toBe(2);
109 });
test/tui-limbs.test.ts +2 −1
@@ -31,6 +31,7 @@ import type { TuiIdentity } from "../src/tui/session.ts";
3131 import { headlessTerminal } from "../src/tui/term/terminal.ts";
3232 import { quietLogs } from "./helpers/globals.ts";
3333 import { settleApp } from "./helpers/tui.ts";
34 import { withScripts } from "./helpers/page-script.ts";
3435
3536 quietLogs();
3637
@@ -258,7 +259,7 @@ describe("raise and lower a limb in the terminal", () => {
258259
259260 test("the milestones page carries the live client and hears limb.moved", async () => {
260261 const res = await fetch(`${servers.url}/alice/order/milestones`);
261 const body = await res.text();
262 const body = await withScripts(servers.url, await res.text());
262263 expect(body).toContain('id="live"');
263264 expect(body).toContain("limb.moved");
264265 // A hand-over to the planner, or taking it back, reaches an open page too.
test/web-browse.test.ts +4 −3
@@ -29,6 +29,7 @@ import { settled } from "../src/origins/jobs.ts";
2929 import { forgetProviders } from "../src/origins/providers/index.ts";
3030 import { clearViewCache } from "../src/view/cache.ts";
3131 import { quietLogs } from "./helpers/globals.ts";
32 import { pageScriptOf } from "./helpers/page-script.ts";
3233 import { newFakeState, serveFakeForge, type FakeForge } from "./fake-origin.ts";
3334 import { codeJs, mirrorJs } from "../src/web/client.ts";
3435 import { STAGE_BADGE, STAGE_BADGE_JS } from "../src/web/stage-row.ts";
@@ -548,7 +549,7 @@ describe("browsing an origin repository on the web", () => {
548549 expect(running).toContain(
549550 `data-stream="/api/v1/user/origins/${originId}/mirror/`,
550551 );
551 expect(running).toContain("new EventSource(url)");
552 expect(await pageScriptOf(base, running)).toContain("new EventSource(url)");
552553 // Each stage is a §5 row with a state badge, never a bare glyph.
553554 expect(running).not.toContain("stage-dot");
554555 expect(running).not.toContain("stage-detail");
@@ -559,8 +560,8 @@ describe("browsing an origin repository on the web", () => {
559560 // The live update is built from the same table the markup is.
560561 expect(mirrorJs).toContain(STAGE_BADGE_JS);
561562 for (const b of Object.values(STAGE_BADGE)) expect(STAGE_BADGE_JS).toContain(b.text);
562 expect(running).toMatch(/<script nonce="[^"]+">/);
563 expect(running).not.toMatch(/<script[^>]*src="(?!\/assets\/(?:shell|beacon)-)/);
563 expect(running).toMatch(/<script src="\/assets\/page-[0-9a-f]+\.js" defer>/);
564 expect(running).not.toMatch(/<script[^>]*src="(?!\/assets\/(?:shell|beacon|page)-)/);
564565
565566 const done = await settled(jobPath.split("/").pop()!);
566567 expect(done?.stage).toBe("done");
test/web-budget.test.ts added +233
@@ -0,0 +1,233 @@
1 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
2 import { mkdtemp, rm } from "node:fs/promises";
3 import { mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import { createApp, type App } from "../src/app.ts";
7 import { serve, type Servers } from "../src/http/server.ts";
8 import { sharedCache } from "../src/http/edge_cache.ts";
9 import { commitToBranch, createLabel, createRepo } from "../src/actions/repos.ts";
10 import { NO_PREFETCH_PATHS, NO_PREFETCH_QUERY, pageScriptAsset, pageScriptDir, pageScriptNamed, speculationAsset } from "../src/web/asset.ts";
11 import { rawResponse } from "../src/web/pages/repo.ts";
12 import { getUserById } from "../src/core/users.ts";
13 import { adminCookie } from "./helpers/admin.ts";
14 import { quietLogs } from "./helpers/globals.ts";
15
16 // The budgets of docs/web.md, counted in bytes of the raw response body.
17 const BUDGET = { html: 40_000, inlineScripts: 0, inlineSvg: 0 };
18
19 describe("web budgets (docs/web.md)", () => {
20 quietLogs();
21 let dataDir = "";
22 let app: App;
23 let servers: Servers;
24 let base = "";
25 let cookie = "";
26
27 beforeAll(async () => {
28 dataDir = await mkdtemp(join(tmpdir(), "forge-budget-"));
29 app = createApp({ dataDir, port: 0 });
30 servers = await serve(app);
31 base = servers.url;
32 cookie = await adminCookie(app, { username: "ada", email: "[email protected]", password: "analytical-engine" });
33 const ada = app.db.query<{ id: string }, []>("SELECT id FROM user WHERE username = 'ada'").get()!;
34 const user = getUserById(app.db, ada.id)!;
35 for (const [name, visibility] of [["open", "public"], ["closed", "private"]] as const) {
36 const made = await createRepo(app, user, { name, visibility });
37 if (!made.ok) throw new Error("repo failed");
38 if (name === "open") {
39 await commitToBranch(app, user, made.value, {
40 message: "files",
41 files: [{ path: "x.js", content: "alert(document.cookie)\n" }, { path: "y.svg", content: "<svg/>" }],
42 });
43 createLabel(app, user, made.value, "bug", "d73a4a");
44 }
45 }
46 });
47 afterAll(async () => {
48 await servers.stop();
49 await rm(dataDir, { recursive: true, force: true });
50 });
51
52 for (const path of ["/", "/explore"]) {
53 test(`${path}: small, no inline script, no inline drawing`, async () => {
54 const res = await fetch(base + path);
55 const text = await res.text();
56 expect(new TextEncoder().encode(text).length).toBeLessThan(BUDGET.html);
57 expect(/<script(?![^>]*\s(src=|type="application\/json"))/.test(text)).toBe(BUDGET.inlineScripts > 0);
58 expect(text.includes("<symbol")).toBe(BUDGET.inlineSvg > 0);
59 expect(text).toContain("/assets/i-");
60 });
61 }
62
63 test("the policy carries no nonce, and the page names its assets as preloads", async () => {
64 const res = await fetch(base + "/");
65 const csp = res.headers.get("content-security-policy") ?? "";
66 expect(csp).not.toContain("nonce");
67 expect(csp).toContain("script-src 'self'");
68 expect(csp).toMatch(/style-src 'self' 'sha256-/);
69 expect(res.headers.get("link")).toContain("rel=preload");
70 expect(res.headers.get("speculation-rules")).toMatch(/^"\/assets\/speculation-/);
71 });
72
73 test("the sprite and speculation rules are immutable files", async () => {
74 const text = await (await fetch(base + "/")).text();
75 const sprite = /\/assets\/i-[0-9a-f]+\.svg/.exec(text)![0];
76 const res = await fetch(base + sprite);
77 expect(res.status).toBe(200);
78 expect(res.headers.get("cache-control")).toContain("immutable");
79 expect(await res.text()).toContain('<symbol id="i-');
80 const rules = await fetch(base + (/\/assets\/speculation-[0-9a-f]+\.json/.exec(
81 (await fetch(base + "/")).headers.get("speculation-rules") ?? "")![0]));
82 expect(rules.headers.get("content-type")).toContain("speculationrules");
83 });
84
85 test("a signed-out public page is edge-cacheable and revalidates with a 304", async () => {
86 const first = await fetch(base + "/");
87 expect(first.headers.get("cache-control")).toMatch(/^public, max-age=0, s-maxage=\d+, stale-while-revalidate=\d+$/);
88 const etag = first.headers.get("etag")!;
89 expect(etag).toBeTruthy();
90 const again = await fetch(base + "/", { headers: { "if-none-match": etag } });
91 expect(again.status).toBe(304);
92 });
93
94 test("a signed-in page is never edge-cached, with or without a cookie header", async () => {
95 for (const path of ["/", "/explore"]) {
96 const res = await fetch(base + path, { headers: { cookie } });
97 const cc = res.headers.get("cache-control") ?? "";
98 expect(cc).not.toContain("public");
99 expect(cc).not.toContain("s-maxage");
100 expect(res.headers.get("etag")).toBeNull();
101 }
102 const bearer = new Request("http://x/", { headers: { authorization: "Bearer t" } });
103 expect(sharedCache(bearer, "/", () => false)).toBe("no-cache");
104 expect(sharedCache(new Request("http://x/", { headers: { "x-plat-user": "u" } }), "/", () => false)).toBe("no-cache");
105 expect(sharedCache(new Request("http://x/"), "/ada/engine/issues", () => false)).toBe("no-cache");
106 });
107
108 test("back/forward cache: no no-store, no unload handlers", async () => {
109 for (const init of [{}, { headers: { cookie } }]) {
110 const res = await fetch(base + "/", init);
111 const text = await res.text();
112 expect(res.headers.get("cache-control")).not.toContain("no-store");
113 expect(text).not.toMatch(/\bunload\b/);
114 for (const src of text.match(/\/assets\/[\w.-]+\.js/g) ?? []) {
115 const js = await (await fetch(base + src)).text();
116 expect(js).not.toMatch(/["']unload["']|onunload|beforeunload/);
117 }
118 }
119 });
120
121 test("only home, explore and an existing public repo's front page are shared", async () => {
122 // A two-segment path that is not a repository is never shared, however it renders.
123 for (const path of ["/apps/new", "/settings/tokens", "/-/confirm", "/ada/nope"]) {
124 const res = await fetch(base + path, { redirect: "manual" });
125 expect(res.headers.get("cache-control") ?? "no-cache").not.toContain("public");
126 expect(res.headers.get("cache-control") ?? "no-cache").not.toContain("s-maxage");
127 }
128 const isPublic = (o: string, n: string): boolean => `${o}/${n}` === "ada/open";
129 expect(sharedCache(new Request("http://x/"), "/apps/new", isPublic)).toBe("no-cache");
130 expect(sharedCache(new Request("http://x/"), "/ada/closed", isPublic)).toBe("no-cache");
131 expect(sharedCache(new Request("http://x/"), "/ada/open", isPublic)).toContain("s-maxage");
132 // The live pages agree: the public repo is shared, the private one is not.
133 const open = await fetch(base + "/ada/open");
134 expect(open.status).toBe(200);
135 expect(open.headers.get("cache-control")).toContain("s-maxage");
136 const closed = await fetch(base + "/ada/closed");
137 expect(closed.headers.get("cache-control") ?? "no-cache").not.toContain("public");
138 });
139
140 test("a shared copy varies on identity, so a signed-in visitor misses it", async () => {
141 for (const path of ["/", "/explore", "/ada/open"]) {
142 const vary = (await fetch(base + path)).headers.get("vary") ?? "";
143 for (const h of ["Cookie", "Authorization", "X-Plat-User", "X-Plat-Edge-Token"]) expect(vary).toContain(h);
144 }
145 });
146
147 test("no user-controlled route is a script: raw files are text/plain or octet-stream under nosniff", async () => {
148 const res = await fetch(base + "/ada/open/raw/main/x.js");
149 expect(res.status).toBe(200);
150 expect(res.headers.get("content-type")).toBe("text/plain; charset=utf-8");
151 expect(res.headers.get("x-content-type-options")).toBe("nosniff");
152 expect(res.headers.get("content-security-policy")).toContain("sandbox");
153 const svg = await fetch(base + "/ada/open/raw/main/y.svg");
154 expect(svg.headers.get("content-type")).not.toMatch(/javascript|ecmascript|html|svg/);
155 const bin = rawResponse({ kind: "binary", size: 3, head: new Uint8Array(3) } as never);
156 expect(bin.headers.get("content-type")).toBe("application/octet-stream");
157 // The only script-typed responses under /assets/ are the trusted, static files the forge renders.
158 expect((await fetch(base + "/assets/page-0000000000000000.js")).status).toBe(404);
159 });
160
161 test("speculation rules exclude every GET with a side effect or a token", async () => {
162 const rules = JSON.parse(speculationAsset().code);
163 const where = rules.prefetch[0].where.and;
164 expect(rules.prefetch[0].eagerness).toBe("moderate");
165 const glob = (g: string): RegExp =>
166 new RegExp("^" + g.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*") + "$");
167 const excluded = (path: string, search = ""): boolean =>
168 (where[1].not.href_matches as string[]).some((g) => glob(g).test(path)) ||
169 (where[2].not.href_matches as { search: string }[]).some((m) => glob(m.search).test(search));
170 for (const p of [
171 "/logout", "/signout", "/sign-out", "/signin/parent/callback", "/login", "/-/confirm", "/claim",
172 "/first-admin", "/verify", "/go/abc", "/settings/tokens", "/apps/open", "/ada/open/app/prod/open",
173 "/ada/open/raw/main/x.js", "/api/v1/x", "/assets/a.css", "/mcp", "/ada/-/teams/t",
174 ])
175 expect(excluded(p), p).toBe(true);
176 for (const [p, q] of [["/", "?token=abc"], ["/ada/open", "?code=1"], ["/x/y", "?a=1&key=z"], ["/explore", "?state=s"]])
177 expect(excluded(p!, q), p! + q!).toBe(true);
178 for (const p of ["/", "/explore", "/ada/open", "/ada/open/issues"]) expect(excluded(p), p).toBe(false);
179 expect(NO_PREFETCH_PATHS.length).toBeGreaterThan(10);
180 expect(NO_PREFETCH_QUERY.length).toBeGreaterThan(5);
181 });
182
183 test("a page script is served by name for as long as the process lives, however many others render", () => {
184 const first = pageScriptAsset("/* first */ 1");
185 for (let i = 0; i < 700; i++) pageScriptAsset(`/* filler ${i} */ ${i}`);
186 expect(pageScriptNamed(first.name)).toBe(first);
187 expect(pageScriptNamed("page-nothing")).toBeUndefined();
188 });
189
190 test("a page script rendered before a restart is still served by a process that never rendered it", async () => {
191 const dir = mkdtempSync(join(tmpdir(), "page-scripts-"));
192 try {
193 const before = pageScriptAsset("/* before the restart */ 2", dir);
194 // A fresh copy of the module is a restarted process: its maps are empty.
195 const fresh = (await import(`../src/web/asset.ts?restart=${Date.now()}`)) as typeof import("../src/web/asset.ts");
196 expect(fresh.pageScriptNamed(before.name)).toBeUndefined();
197 const served = fresh.pageScriptNamed(before.name, dir);
198 expect(served?.code).toBe(before.code);
199 expect(served?.etag).toBe(before.etag);
200 // A body that does not hash to its name, or a name that is not a page script's, is never read.
201 writeFileSync(join(fresh.pageScriptDir(dir), "page-0000000000000000.js"), "alert(1)");
202 expect(fresh.pageScriptNamed("page-0000000000000000", dir)).toBeUndefined();
203 expect(fresh.pageScriptNamed("../../etc/passwd", dir)).toBeUndefined();
204 } finally {
205 rmSync(dir, { recursive: true, force: true });
206 }
207 });
208
209 test("a rendered page keeps its script under the data dir, and the forge serves it by name", async () => {
210 const page = await (await fetch(base + "/ada/open", { headers: { cookie } })).text();
211 const name = page.match(/\/assets\/(page-[0-9a-f]{16})\.js/)?.[1];
212 expect(name).toBeDefined();
213 const files = readdirSync(pageScriptDir(dataDir));
214 expect(files).toContain(`${name}.js`);
215 const res = await fetch(`${base}/assets/${name}.js`);
216 expect(res.status).toBe(200);
217 expect(res.headers.get("content-type")).toContain("text/javascript");
218 });
219
220 test("the style hash in the policy matches the inline <style> on a page with its own styleCss", async () => {
221 const res = await fetch(base + "/ada/open/settings/labels", { headers: { cookie } });
222 expect(res.status).toBe(200);
223 const text = await res.text();
224 const styles = [...text.matchAll(/<style>([\s\S]*?)<\/style>/g)].map((m) => m[1]!);
225 expect(styles.length).toBe(2);
226 expect(styles[1]).toContain(".lc-d73a4a");
227 const csp = res.headers.get("content-security-policy") ?? "";
228 for (const css of styles) {
229 const hash = new Bun.CryptoHasher("sha256").update(css).digest("base64");
230 expect(csp).toContain(`'sha256-${hash}'`);
231 }
232 });
233 });
test/web-editor.test.ts +5 −10
@@ -647,14 +647,12 @@ describe("editing files on the web", () => {
647647 expect(await editor.text()).toContain(
648648 `<script type="module" src="${asset.path}">`,
649649 );
650 expect(policy).toContain(`script-src 'nonce-`);
651 expect(/script-src 'nonce-[^']+' 'self'/.test(policy)).toBe(true);
652
653 // Every other page keeps the stricter policy: nonce only.
650 // Scripts are hashed files, so every page's policy is the same: this
651 // origin's files, no inline script, no nonce, nothing unsafe.
652 expect(policy).toContain("script-src 'self';");
653 expect(policy).not.toContain("unsafe");
654654 const blob = await get("/alice/beacon/blob/main/src/lamp.ts", alice);
655 const strict = blob.headers.get("content-security-policy") ?? "";
656 expect(/script-src 'nonce-[^']+';/.test(strict)).toBe(true);
657 expect(/script-src [^;]*'self'/.test(strict)).toBe(false);
655 expect(blob.headers.get("content-security-policy")).toContain("script-src 'self';");
658656 });
659657
660658 test("the bundle is served immutably at its hash, and nowhere else", async () => {
@@ -771,9 +769,6 @@ describe("editing files on the web", () => {
771769 expect(html).not.toContain('id="editor-mirror"');
772770 expect(html).not.toContain('data-mirror="1"');
773771 expect(html).not.toContain('<script type="module"');
774 const res = await get("/alice/beacon/edit/main/src/generated.ts", alice);
775 const policy = res.headers.get("content-security-policy") ?? "";
776 expect(/script-src [^;]*'self'/.test(policy)).toBe(false);
777772 });
778773
779774 test("suggestBranch makes a name git will take", () => {
test/web-origins.test.ts +5 −3
@@ -35,6 +35,8 @@ import {
3535 type FakeForge,
3636 type FakeOAuth,
3737 } from "./fake-origin.ts";
38 import { pointAtSprite } from "../src/web/icons.ts";
39 import { spriteAsset } from "../src/web/asset.ts";
3840 import { adminCookie, seedAdmin } from "./helpers/admin.ts";
3941
4042 quietLogs();
@@ -247,9 +249,9 @@ describe("origins on the web", () => {
247249 // The lead is the caret (it is rotated when open); the brand glyph
248250 // is in the title beside the name.
249251 const lead = /<span class="row-lead">([\s\S]*?)<\/span>/.exec(summary)![1]!;
250 expect(lead).toBe(icon("chevronRight").text);
251 expect(lead).not.toBe(icon(brand).text);
252 expect(/<span\s+class="row-title"[^>]*>([\s\S]*?)<\/span\s*>/.exec(summary)![1]).toContain(icon(brand).text);
252 expect(lead).toBe(pointAtSprite(icon("chevronRight").text, spriteAsset().path));
253 expect(lead).not.toBe(pointAtSprite(icon(brand).text, spriteAsset().path));
254 expect(/<span\s+class="row-title"[^>]*>([\s\S]*?)<\/span\s*>/.exec(summary)![1]).toContain(pointAtSprite(icon(brand).text, spriteAsset().path));
253255 }
254256 }
255257 expect(fold(closed, "connect-token").open).toBe(true);
test/web-ui.test.ts +1 −1
@@ -198,7 +198,7 @@ describe("web ui", () => {
198198 expect(page).toContain(">bug</span>");
199199 // A colour mixed from the labels on *this* page is what `styleCss` is
200200 // for, and the only thing left using it: nothing static rides along.
201 expect(page).toMatch(/<style nonce="[^"]+">\.lc-d73a4a\{--label:#d73a4a\}<\/style>/);
201 expect(page).toMatch(/<style>\.lc-d73a4a\{--label:#d73a4a\}<\/style>/);
202202
203203 // And the list can filter by it.
204204 const filtered = await (
test/web.test.ts +30 −31
@@ -13,12 +13,13 @@ import { shellJs } from "../src/web/client.ts";
1313 import { escapeHtml, html, raw, when } from "../src/web/html.ts";
1414 import {
1515 icon,
16 iconSprite,
16 pointAtSprite,
17 spriteSvg,
1718 inlineIcons,
1819 type IconName,
1920 } from "../src/web/icons.ts";
2021 import { renderMarkdown } from "../src/web/markdown.ts";
21 import { highlightAsset } from "../src/web/asset.ts";
22 import { highlightAsset, spriteAsset } from "../src/web/asset.ts";
2223 import { securityFloor } from "../src/http/respond.ts";
2324 import { THEME_NAMES } from "../src/design/themes.ts";
2425 import { sha256 } from "../src/kernel/hash.ts";
@@ -43,6 +44,7 @@ describe("stylesheet asset", () => {
4344 import { gitClientEnv } from "./helpers/git-client.ts";
4445 import { adminCookie } from "./helpers/admin.ts";
4546 import { quietLogs } from "./helpers/globals.ts";
47 import { pageScriptOf } from "./helpers/page-script.ts";
4648
4749 describe("html template", () => {
4850 test("escapes interpolations by default", () => {
@@ -147,14 +149,14 @@ describe("icons", () => {
147149 expect(icon("loader").text).toContain('class="ico ico-loader"');
148150 expect(icon("issue", 20).text).toContain('width="20" height="20"');
149151
150 const sprite = iconSprite(icon("issue").text.repeat(4)).text;
151 expect(sprite.match(/<symbol /g)).toHaveLength(1);
152 const sprite = spriteSvg();
153 expect(sprite.match(/<symbol /g)!.length).toBeGreaterThan(20);
152154 expect(sprite).toContain(
153155 `<symbol id="i-issue" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75"`,
154156 );
155157 expect(sprite).toContain('<circle cx="12" cy="12" r="1"/>');
156 // Markup with no icons defines none at all.
157 expect(iconSprite("<p>a page of words</p>").text).toBe("");
158 // A page aims every reference at the one hashed file.
159 expect(pointAtSprite(icon("issue").text, "/assets/i-x.svg")).toContain('<use href="/assets/i-x.svg#i-issue"/>');
158160 });
159161
160162 test("a fragment carries its drawings inline, referencing no sprite", () => {
@@ -196,7 +198,7 @@ describe("icons", () => {
196198 const markup = census
197199 .map(([name, count]) => icon(name).text.repeat(count))
198200 .join("");
199 const shipped = markup.length + iconSprite(markup).text.length;
201 const shipped = pointAtSprite(markup, "/assets/i-0123456789abcdef.svg").length;
200202 // What the same page would weigh with a drawing per use.
201203 const perUse = inlineIcons(markup).length;
202204 expect(perUse - shipped).toBeGreaterThan(15_000);
@@ -259,31 +261,30 @@ describe("a page ships each drawing once", () => {
259261 const text = await (
260262 await fetch(`${base}/ada/engine/issues/1`, { headers: { cookie } })
261263 ).text();
262 expect(text.match(/<use href="#i-comment"\/>/g)!.length).toBeGreaterThan(
264 expect(text.match(/<use href="[^"#]+#i-comment"\/>/g)!.length).toBeGreaterThan(
263265 11,
264266 );
265 expect(text.match(/<symbol id="i-comment"/g)).toHaveLength(1);
267 // The drawing is not in the page at all: it is the one hashed sprite file.
268 expect(text).not.toContain("<symbol");
269 expect(text).toContain(`<use href="${spriteAsset().path}#i-comment"/>`);
266270 });
267271
268 test("what a page defines is exactly what it used — no more, no less", async () => {
272 test("every icon a page uses is defined by the sprite file it points at", async () => {
269273 for (const path of ["/", "/ada/engine/issues", "/settings"]) {
270274 const text = await (
271275 await fetch(`${base}${path}`, { headers: { cookie } })
272276 ).text();
273277 const used = new Set(
274 [...text.matchAll(/<use href="#i-([A-Za-z]+)"\/>/g)].map((m) => m[1]),
278 [...text.matchAll(/<use href="([^"#]+)#i-([A-Za-z]+)"\/>/g)].map((m) => {
279 expect(m[1]).toBe(spriteAsset().path);
280 return m[2];
281 }),
275282 );
276 const defined = [...text.matchAll(/<symbol id="i-([A-Za-z]+)"/g)].map(
277 (m) => m[1],
278 );
279 expect(defined.length).toBe(used.size); // each defined once
280 expect(new Set(defined)).toEqual(used); // and nothing else shipped
281283 expect(used.size).toBeGreaterThan(0);
284 expect(text).not.toContain("<symbol");
285 const file = await (await fetch(base + spriteAsset().path)).text();
286 for (const name of used) expect(file).toContain(`<symbol id="i-${name}"`);
282287 }
283 // The front door is not the settings page's icons: a page with three
284 // drawings does not ship twenty.
285 const home = await (await fetch(base, { headers: { cookie } })).text();
286 expect(home).not.toContain('id="i-webhook"');
287288 });
288289
289290 test("the thread fragment inlines its own drawings", async () => {
@@ -377,9 +378,7 @@ describe("web pages end to end", () => {
377378 test("the shell is a cached file, linked before the page's own inline script", async () => {
378379 const shell = shellAsset();
379380 const home = await (await fetch(base, { headers: { cookie } })).text();
380 expect(home).toMatch(
381 new RegExp(`<script nonce="[^"]+" src="${shell.path}" defer></script>`),
382 );
381 expect(home).toContain(`<script src="${shell.path}" defer></script>`);
383382 // Its source is no longer in the HTML, and the page is a good deal lighter.
384383 expect(home).not.toContain(shellJs.slice(0, 400));
385384 expect(home).not.toContain(shellJs.slice(-400));
@@ -390,10 +389,10 @@ describe("web pages end to end", () => {
390389 await fetch(`${base}/apps/new`, { headers: { cookie } })
391390 ).text();
392391 const at = apps.indexOf(`src="${shell.path}" defer>`);
393 const inline = apps.indexOf("<script nonce=", at);
392 const inline = apps.indexOf('<script src="/assets/page-', at);
394393 expect(at).toBeGreaterThan(-1);
395394 expect(inline).toBeGreaterThan(at);
396 expect(apps).toContain('document.addEventListener("DOMContentLoaded"');
395 expect(await pageScriptOf(base, apps)).toContain('document.addEventListener("DOMContentLoaded"');
397396 expect(apps).not.toContain(shellJs.slice(0, 400));
398397
399398 const file = await fetch(base + shell.path);
@@ -690,7 +689,7 @@ describe("themes", () => {
690689 expect(text).not.toContain("theme-apply");
691690 // Critical paint before the stylesheet: system pair (light default + dark
692691 // media), then a rule per theme so a toggle repaints without a reload.
693 const boot = text.match(/<style nonce="[^"]+">([^<]*)<\/style>/)?.[1];
692 const boot = text.match(/<style>([^<]*)<\/style>/)?.[1];
694693 expect(boot).toStartWith(
695694 "html,body{background-color:#fafafa;color-scheme:light}@media (prefers-color-scheme: dark){html,body{background-color:#09090b;color-scheme:dark}}",
696695 );
@@ -724,12 +723,12 @@ describe("themes", () => {
724723 expect(text).toMatch(/value="midnight"\s+checked/);
725724 // Boot paint follows the attribute: midnight's ground for this page, and
726725 // the same rules as a system page, so switching away repaints at once.
727 const boot = text.match(/<style nonce="[^"]+">([^<]*)<\/style>/)?.[1];
726 const boot = text.match(/<style>([^<]*)<\/style>/)?.[1];
728727 expect(boot).toContain(
729728 'html[data-theme="midnight"],html[data-theme="midnight"] body{background-color:#16161e;color-scheme:dark}',
730729 );
731730 const system = (await (await fetch(base)).text()).match(
732 /<style nonce="[^"]+">([^<]*)<\/style>/,
731 /<style>([^<]*)<\/style>/,
733732 )?.[1];
734733 expect(boot).toBe(system!);
735734 });
@@ -806,7 +805,7 @@ describe("HEAD mirrors the GET it stands in for", () => {
806805 expect(head.headers.get("content-type")).toBe(get.headers.get("content-type"));
807806 expect(head.headers.get("x-content-type-options")).toBe("nosniff");
808807 expect(head.headers.get("referrer-policy")).toBeTruthy();
809 expect(head.headers.get("content-security-policy")).toContain("nonce-");
808 expect(head.headers.get("content-security-policy")).toContain("script-src 'self'");
810809 });
811810
812811 test("HEAD /healthz is 200 with an empty body", async () => {
@@ -892,9 +891,9 @@ describe("the security floor is on every response", () => {
892891 const page = await fetch(`${base}/`);
893892 expect(page.status).toBe(200);
894893 expectFloor("html page", page);
895 // An HTML body keeps the nonce'd policy it writes for itself: the floor
894 // An HTML body keeps the policy it writes for itself: the floor
896895 // adds the other two headers and does not touch a policy already there.
897 expect(page.headers.get("content-security-policy")).toContain("nonce-");
896 expect(page.headers.get("content-security-policy")).toContain("script-src 'self'");
898897
899898 const list = await fetch(`${base}/api/v1/repos`);
900899 expect(list.status).toBe(200);

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs