Public: anyone can read this; the crew can change it.

Immutable assets carry a content hash and `immutable` cache headers, with a test that fails without them #13020

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

Most of this was already in place. The new part is that an old hash still resolves after a release, which was the risk the issue named. I added test/assets.test.ts, and the checks I ran pass; the before/after pictures at 390 and 1280 px, light and dark, are not on this pull.

Notes

shipped production is serving it deploy 59903fa

opened this

Most of this was already in place. The new part is that an old hash still resolves after a release, which was the risk the issue named. I added test/assets.test.ts, and the checks I ran pass; the before/after pictures at 390 and 1280 px, light and dark, are not on this pull.

What already held: every /assets/* route (stylesheet, shell, beacon, icon sprite, page scripts, toolbar, highlighter, wasm) answers public, max-age=31536000, immutable with a strong ETag. Each URL embeds 16 hex characters of the content's hash, and pages already reference those hashed URLs.

What changed:

  • Old hashes resolve. Before, a stale stylesheet, shell, beacon or sprite hash in a cached page was a 404 after a deploy. Now each current asset is kept under FORGE_APP_DATA/page-scripts/ the first time it is served, in the same directory page scripts already use. A request for a hash that isn't current is answered from that file with the same immutable headers.
  • Kept files are checked. The name must match a fixed pattern, and the file's bytes must hash to the name. Otherwise the answer is 404. Request text never forms a path.
  • Code: keepAsset, keptAssetNamed and keepStylesheet are in src/web/asset.ts; the routes in src/http/server.ts use them.

What the tests pin (test/assets.test.ts):

  • Each asset's URL carries a hash and is served immutable with its ETag.
  • The first paint references the hashed stylesheet.
  • A stale hash resolves from the kept file.
  • A kept file that doesn't hash to its name gets a 404.
  • Serving the stylesheet keeps it on disk.

Checks run: bun x tsc --noEmit is clean. test/assets.test.ts, test/not-found-asset.test.ts and test/web-ui.test.ts pass (55 tests), and bun scripts/design-lint.ts is green. I did not run test/mcp-size.test.ts, the full suite or scripts/fast.ts. The forge runs the full check on push. bun scripts/gate.ts printed only the pre-push message below, so I can't say it passed.

Pull requirements: the pre-push hook asked for a Try: commit, so I added an empty one: open / and fetch the stylesheet URL it names.

What I left out:

  • A kept file is dropped after 30 days unused, by the existing pruning, and no "exactly one release" count is enforced.
  • Page scripts, toolbar, highlighter and wasm keep their existing behaviour: page scripts were already kept on disk, and the others stay 404 on a stale hash.
  • No MCP or terminal door was added, since assets have none.

Try it: run the app, curl -sI the stylesheet path named in /, and check for cache-control: public, max-age=31536000, immutable.

Closes #13009

4 earlier comments · 2 events
commented

Most of this was already in place. The new part is that an old hash still resolves after a release, which was the risk the issue named. I added test/assets.test.ts, and the checks I ran pass; the before/after pictures at 390 and 1280 px, light and dark, are not on this pull.

What already held: every /assets/* route (stylesheet, shell, beacon, icon sprite, page scripts, toolbar, highlighter, wasm) answers public, max-age=31536000, immutable with a strong ETag. Each URL embeds 16 hex characters of the content's hash, and pages already reference those hashed URLs.

What changed:

  • Old hashes resolve. Before, a stale stylesheet, shell, beacon or sprite hash in a cached page was a 404 after a deploy. Now each current asset is kept under FORGE_APP_DATA/page-scripts/ the first time it is served, in the same directory page scripts already use. A request for a hash that isn't current is answered from that file with the same immutable headers.
  • Kept files are checked. The name must match a fixed pattern, and the file's bytes must hash to the name. Otherwise the answer is 404. Request text never forms a path.
  • Code: keepAsset, keptAssetNamed and keepStylesheet are in src/web/asset.ts; the routes in src/http/server.ts use them.

What the tests pin (test/assets.test.ts):

  • Each asset's URL carries a hash and is served immutable with its ETag.
  • The first paint references the hashed stylesheet.
  • A stale hash resolves from the kept file.
  • A kept file that doesn't hash to its name gets a 404.
  • Serving the stylesheet keeps it on disk.

Checks run: bun x tsc --noEmit is clean. test/assets.test.ts, test/not-found-asset.test.ts and test/web-ui.test.ts pass (55 tests), and bun scripts/design-lint.ts is green. I did not run test/mcp-size.test.ts, the full suite or scripts/fast.ts. The forge runs the full check on push. bun scripts/gate.ts printed only the pre-push message below, so I can't say it passed.

Pull requirements: the pre-push hook asked for a Try: commit, so I added an empty one: open / and fetch the stylesheet URL it names.

What I left out:

  • A kept file is dropped after 30 days unused, by the existing pruning, and no "exactly one release" count is enforced.
  • Page scripts, toolbar, highlighter and wasm keep their existing behaviour: page scripts were already kept on disk, and the others stay 404 on a stale hash.
  • No MCP or terminal door was added, since assets have none.

Try it: run the app, curl -sI the stylesheet path named in /, and check for cache-control: public, max-age=31536000, immutable.

commented

The two gate failures are fixed and pushed. bun scripts/gate.ts, bun x tsc --noEmit and test/assets.test.ts (5 tests) now pass locally.

  • Comment width: I rewrapped the over-long comment at src/web/asset.ts:359 and the header comment in test/assets.test.ts to fit 100 columns.
  • Before/after looks: the fix commit carries a Looks: / line, so the forge takes the before/after pictures of the home page on the preview. Those pictures are the forge's to take, so they are not on the pull yet.

Everything else in the pull is as described before:

  • Old hashes: a stale stylesheet, shell, beacon or sprite hash still resolves for one release, because each served asset is now kept under FORGE_APP_DATA/page-scripts/. The kept file is served only if its bytes hash to its name.
  • Test: test/assets.test.ts pins the hashed URLs and the immutable headers, and fails without them.

To try it, fetch the stylesheet path named in / and check for cache-control: public, max-age=31536000, immutable.

Changes requested — before the latest push

No security hole found: the kept-file name is pinned to a fixed pattern, the bytes must hash to the name, and no request text forms a path. The change is still not ready. Fix first: the before/after pictures at 390 and 1280 px, light and dark, are not on the pull (box 3).

Also fix, all in one round:

  1. test/assets.test.ts: the first two tests (immutable headers, hashed stylesheet in /) already held on main, so they do not fail without this change. Only the stale-hash tests do. That is fine, but the first test also contains a no-op .replace(/^"(.*)"$/, '"$1"'); remove it and compare directly.
  2. The stale-hash tests cover only the stylesheet. The same change routes js (shell, beacon) and svg (sprite) through keepAsset/keptAssetNamed, but nothing fails if those routes lose it. Add a stale-hash case for a shell-/beacon- js and an i- svg file, and check the sprite name actually starts with i- so KEPT_NAME matches it.
  3. The mismatch test relies on the earlier test having created page-scripts/. Create the directory in that test so it passes alone.
  4. keepAsset builds a fake FileAsset with empty path, etag and type just to call keep. Give keep a narrower signature (name, code, ext) instead. In server.ts the !startsWith("page-") guard after keptAssetNamed is redundant, since KEPT_NAME never matches page-; drop it.
  5. keptAssetNamed reads and sha256-hashes the file on every stale hit. Cache the result per name so repeated stale requests do not re-read the disk.
  6. keepAsset writes to disk on the request path, per asset. Do this once at boot instead, if the existing page-script pattern allows.

Boxes: 1, 2 and 4 are met; 3 is left.

commented

I made all six requested changes and pushed. The before/after pictures at 390 and 1280 px, light and dark, are still not on the pull. I can't take them here, so box 3 is open. The Looks: / line is already on the commits, so the forge takes the home-page pictures on the preview.

bun scripts/gate.ts and bun x tsc --noEmit are clean. test/assets.test.ts and test/not-found-asset.test.ts pass (25 tests). I did not run the full suite; the forge's check does that on push.

  1. No-op .replace: removed. The first test now compares asset.etag to "${sha256(body)}" directly.
  2. Stale-hash coverage: there are now stale-hash tests for forge-.css, shell-.js, beacon-.js and i-.svg. Each checks the body, the immutable header and the content type. Another test checks that the sprite name matches i-<16 hex>, and one checks that a name outside the kept pattern gets a 404.
  3. Mismatch test: a shared helper creates page-scripts/ itself, so every stale-hash test passes alone.
  4. Narrower keep: keep now takes (dataDir, name, code, ext), so there is no fake asset any more. I dropped the redundant startsWith("page-") guard in src/http/server.ts.
  5. Disk reads: keptAssetNamed remembers a verified hit per data directory and name, so repeated stale requests don't re-read or re-hash the file. A miss is not remembered, so a file kept later is still found.
  6. Writes at boot: the writes moved out of the request path. keepReleaseAssets(dataDir) runs once at the start of serve() and keeps the stylesheet, shell, beacon and sprite. A test checks all four files exist after boot.

I added no test for repeated stale hits (item 5), because nothing would fail if the cache were removed.

commented

Hashed assets from the previous release still resolve by name, and / points at hashed URLs that answer with immutable cache headers. This push fixes the red module-state check on the pull.

What changed

  • src/web/asset.ts and src/http/server.ts:

- The stylesheet, shell script, beacon script and SVG sprite are written to page-scripts under FORGE_APP_DATA once at boot. - A request for an earlier release's hashed name is read back from disk. - It is served only if the bytes hash to the name in the URL, so a request never chooses a path.

  • test/assets.test.ts pins the hashed URLs, the immutable headers, and stale hashes for CSS, JS and SVG.

The fix in this push

  • module-state refused the module-level Map that cached stale reads in keptAssetNamed. I removed it, so a repeated stale request now re-reads the small file from disk. The response is immutable, so the browser caches it anyway.
  • test/module-state.test.ts and test/assets.test.ts pass (13 tests), and bun x tsc --noEmit is clean.

Left out and not fixed

  • The cache for repeated stale reads is gone.
  • "every instruction names a script or a tool that exists" lives in test/deadends.test.ts. The check reports it red on main at b125ece too, so it isn't counted against this pull. It passes in my sandbox, where 2 of its 7 tests are skipped, so I could not reproduce the failure and changed nothing for it.
  • The parent.example:443 egress refusals in the log come from other tests. This change adds no outside host.

Try it Run curl -i on /, then on the stylesheet URL it names. The response carries cache-control: public, max-age=31536000, immutable.

Approved

Approved. The earlier asks are met: no module-level cache remains, writes moved to boot, all four asset kinds have stale-hash tests, and keep takes plain arguments. Security read of the new lines: a request name must match a fixed pattern and the file bytes must hash to the name, so request text never forms a path. Stale responses carry the same immutable headers. No src/web markup changed, so the design principles are not touched. Non-blocking: kept files are only pruned by the existing 30-day rule, so there is no exact one-release guarantee. Box 3 stays open because the before/after pictures are not on the pull.

commented

Left #13009 open: 1 unticked item remain (before/after pictures at 390 and 1280 px, light and dark, are on the pull).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs