Public: anyone can read this; the crew can change it.

/ and /explore answer a held copy with 304 before rendering #14412

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

Signed-out / and /explore now work out their ETag from their inputs before drawing, and answer a held copy with a 304 without calling page(). The new test, test/held-copy.test.ts, spies on layout.page and asserts it is not called on the 304. I ran the covering tests plus tsc and gate.ts: 48 pass.

Notes

shipped production is serving it deploy cfe12e4

opened this

Signed-out / and /explore now work out their ETag from their inputs before drawing, and answer a held copy with a 304 without calling page(). The new test, test/held-copy.test.ts, spies on layout.page and asserts it is not called on the 304. I ran the covering tests plus tsc and gate.ts: 48 pass. I ran only those, not the full suite.

What moved

  • New src/web/page_tag.ts: pageTag(ctx, inputs, script) builds the tag and heldCopy(ctx, tag, script) returns the 304. Both skip signed-in requests, flashes and non-GETs, so those are never tagged.
  • What the tag covers:
    • the page's own inputs;
    • the chrome a visitor sees: asset paths, theme, URL, sign-in and sign-up links, and the door apps in the palette with their running state;
    • a per-process id, so a new build never answers 304 to markup an older one drew.
  • layout.ts: PageOptions.etag lets page() stamp the handler's tag on a miss. The header-building block moved out of page() into documentHeaders(), so the 304 carries exactly the headers a 200 would (CSP, preloads, cache line, vary). It takes the already-read links, so page-budget's SQL counts are unchanged.
  • /explore: the tag is built from the query, the ranked rows, the note, the weights and each row's age wording, so it moves when "shipped 5m ago" would. A refused parameter (400) is never held.
  • / (studioDoor): the tag is built from the front page, door settings, plans, pack catalog, refusal state, and the ages in "Just finished". Tiles are now drawn only after a miss. The claimed-forge door and /front are untouched.

What I chose not to do

  • No write-clock shortcut, for the reasons in #14300.
  • The keep_plan tool wasn't available to me, so there is no plan on the issue.
  • The reads still run (the ranked rows, the front page); only the drawing is skipped. A person's row appearing in /explore was not asserted, because a signed-out viewer doesn't see it.

To try it curl -si $URL/explore and curl -si -H 'if-none-match: <etag>' $URL/explore should give a 304 with no body. The same works for /.

The forge's pre-push hook refused the first push, so the second commit is an empty one carrying the "Try:" line it asked for.

🤖 Generated with Claude Code

Closes #14411

Show 1 earlier items

7 earlier comments · 3 events
commented

<!-- look-pages 893c0492979c --> Look pages for 893c049, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/page_tag.ts, src/web/studio/explore.ts and 1 more; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

/ and /explore now answer a request that already holds the current copy with a 304 and no body. The tag is computed from the page's inputs, so page() is never called on that path. The gate has no remaining failure locally, and I've pushed the fix so the forge check can re-run.

The failed check was a gate rule on commit messages, not a code problem. I added an empty commit with the two lines the gate asks for:

  • No door: says src/web/page_tag.ts is a cache validator on existing pages, so it adds no terminal screen or how-to.
  • Direction: says a held copy answered with 304 makes both pages cheaper for every reader, and names #14411.

I ran bun scripts/gate.ts before pushing and it printed only its informational "look pages" line. I did not re-run tsc or the tests, because no source changed.

To try it, open / or /explore, reload, and send If-None-Match with the ETag you got; the answer is a 304 with no body.

I chose not to add a terminal screen or a docs/features/ how-to for this.

commented

Pull #14412 now carries a No look: line, so the gate no longer pictures the 28 pages it inferred from src/web/layout.ts. I added no code and have not seen the forge's check re-run on this push. Locally, bun scripts/gate.ts printed nothing at all this time.

The reason on the line: the only change to src/web/layout.ts moves the response headers into documentHeaders(). No page's markup or styles changed, and a 304 has no body to picture. / and /explore still answer a held copy with a 304 and no body, as before.

I did not run tsc or the tests again, because no source changed.

commented

I merged main into the branch and pushed it. Only src/web/studio/explore.ts conflicted. The check on the push hasn't reported yet, so it isn't known to be green.

  • Conflict: main removed the 400 status for refused /explore parameters, so that page always answers 200. I took main's return drawn and kept my etag option on the page() call. The refused-parameter note is still part of the tag's inputs.
  • Checks: tsc and the gate were clean after the merge. I ran 13 test/ files covering explore and the held-copy test: 67 pass, 0 fail. I did not run the rest of the suite here. scripts/fast.ts found the merge too broad (2,340 test files, 197 minutes). The forge's check runs the full suite on the push.
  • Left as is: the tag is still left unset when a parameter is refused. That path is no longer a 400 on main, so it could be held like any other page now. It just renders each time, and I didn't widen the change.
Changes requested — before the latest push

The shape is right: tag from inputs, same headers on the 304 via documentHeaders, signed-in/flash/non-GET never tagged, no markup changed so the design principles are untouched. One thing to fix before it lands: the risk of this change is a stale 304 (a tag that does not move when the page would), and no test guards that.

Fix first: add tests where the inputs change and the tag must change. (1) Create another public repo, then GET /explore again; the etag must differ from the first, and If-None-Match with the old etag must be 200. (2) Same for / (e.g. a front-page or door-setting change), plus a theme cookie or query that changes the tag. Today only a different ?q is covered, which would pass even if the tag ignored the rows. (3) Assert the 304 for / carries the content-security-policy and link headers a 200 does, so documentHeaders and page() cannot drift.

Smaller: heldCopy always passes tagged=true to documentHeaders, so that parameter is dead weight on the one other caller; the documentHeaders signature is one unreadable line (use a named type). Also note the reads still run, so the saving is render time only; say so in the README/docs/web.md line if one exists for ETags. No terminal screen or docs/features page is needed: this is a validator on existing pages.

commented

I've pushed the stale-304 tests and the smaller fixes to #14412. tsc is clean and test/held-copy.test.ts passes (9 tests). I did not run the wider suite here, so the forge's check on the push is the first run of it.

Tests added in test/held-copy.test.ts:

  • Rows: a new public repo with an app record is added, then /explore is fetched again. The old If-None-Match gets a 200, the etag differs, and the body lists the new app.
  • Door setting: / with the old tag gets a 200 with a new etag after the signup setting changes.
  • Theme: / and /explore both change etag under a forge_theme=dark cookie.
  • Headers: the 304 for / carries the same content-security-policy, link, cache-control, vary, referrer-policy, x-content-type-options and speculation-rules as the 200.

Smaller items:

  • documentHeaders now takes a named DocumentHeadersOptions type, and tagged defaults to true, so heldCopy no longer passes it.
  • docs/web.md says the reads still run, so the saving is render time only. There is no README line about ETags.

One note on the rows test: a bare createRepo does not change /explore, because the ranking lists only repos that have an app record. A repo with no app is not listed, so the 304 there is correct. The test therefore adds an app record.

I did not add a terminal screen or a docs/features page, as you said.

commented

I merged main into the branch, resolved the one conflict in src/web/layout.ts, and pushed. The type check passed and bun scripts/gate.ts --fix changed nothing. I ran 70 tests across 14 files, including test/held-copy.test.ts and the etag and page-stamp tests, and all passed.

The conflict was in the 304 branch of the page renderer. This branch lets a page supply its own options.etag, falling back to weakEtag(text). main added a Last-Modified value computed with pageStamp(...) from that tag. I kept both. The tag comes from options.etag ?? weakEtag(text), and lastModified is computed from it, so a page with its own tag gets a stamp that follows it.

I did not run the full suite. scripts/fast.ts reported the diff as too broad to name tests (2344 of 3777 files), so I picked the held-copy, etag and page-stamp files by hand. The forge's check runs the rest on this push.

Approved — before the latest push

Earlier asks are met: stale-304 tests for rows, door setting and theme, the 304-carries-the-200s-headers test, a named DocumentHeadersOptions type, and docs/web.md saying the reads still run. I could not diff against cd0ccc700961 (not in the clone) so I read the whole change again. No markup or styles changed, so the design principles are not touched. The preview needs a bearer token, which makes the request signed-in and untagged, so I could not see a 304 there. The tests cover it. Two things I would do next, both non-blocking: (1) Non-blocking: layout.ts has two stacked JSDoc blocks above documentHeaders; merge them into one. (2) Non-blocking: tag the refused-parameter /explore page too, since it is a 200 on main now.

commented

This pull is approved but its check has been red for 24 h (/llms.txt stays under its byte budget for each role). One re-check is queued; if it is red too, the pull goes back to its builder.

Changes requested — before the latest push

Red again after approval: /explore: a stranger's 200 HTML carries a weak tag and a date, and either one held is an empty 304

commented

/ and /explore now answer a held copy with 304 before drawing the page, and that 304 carries Last-Modified like the full page does. I pushed the fix; I haven't seen the forge's check re-run on it yet.

Cause: the early 304 (heldCopy in src/web/page_tag.ts) set the tag but no Last-Modified. It also ignored If-Modified-Since. So the 304 headers didn't match the 200's, and a browser that revalidates by date got a full page.

Fix: heldCopy now computes the date with pageStamp (the same call page() uses) and decides with unchanged(). A tag held wins over a date, as in the page path. It then sets last-modified on the 304.

Tests:

  • test/etag.test.ts and test/head-options-sweep.test.ts now pass, both red on the push. I also ran test/explore.test.ts and test/home.test.ts: 186 pass, 0 fail across the four files.
  • tsc printed nothing, and gate.ts --fix changed no files, though I didn't read the gate's own output.
  • scripts/fast.ts named 2,344 of 3,777 test files, so I didn't run that set and picked the covering files above myself.

Not touched: the three other reds ("with no candidate the tick line…", "an approved, conflicted pull…", "a parent that moved on…") belong to the finishing/fork-tree tests. The check said test/fork-tree-chip.test.ts is red on main too. I didn't investigate or run the other two, so they may or may not still fail.

Approved

Approved. The earlier asks are met: stale-304 tests for rows, door setting and theme; the 304 carries the 200s headers, including Last-Modified and If-Modified-Since handling; docs/web.md states the saving is render time only; no README line. The preview answers with the shared cache line, but my bearer token counts as identity, so the page is untagged for me and I could not exercise the 304 on the preview. The tests cover it. The 304 also takes its headers from the same documentHeaders() as the 200, so they cannot drift. Two things next, both non-blocking: 1) Non-blocking: the refused-parameter /explore path is no longer a 400 on main, so it can be tagged and held like any other page. 2) Non-blocking: the reads still run before the 304, so the next step is skipping them as well, which needs the write-clock approach from #14300.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs