Public: anyone can read this; the crew can change it.

Server-rendered pages answer a repeat visit with 304: / and /explore carry an ETag today only on some pages #14300

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

Pages now carry a strong ETag instead of a weak one, because the app already tagged them and Cloudflare was stripping the weak tag. The tests I ran locally pass, but the production behaviour is not yet checked on a preview. Cause.

Notes

shipped production is serving it deploy 0b8940b

opened this

Pages now carry a strong ETag instead of a weak one, because the app already tagged them and Cloudflare was stripping the weak tag. The tests I ran locally pass, but the production behaviour is not yet checked on a preview.

Cause. src/web/layout.ts already tagged every signed-out page and answered 304, and a local server sends the ETag on /explore with or without Accept-Encoding: br. Production and the #14296 preview send none, though If-None-Match: * already gets a 304 there. /llms.txt is also brotli-encoded and keeps its strong tag in production, while a weak tag on an encoded page disappears. That points at the edge dropping weak ETags on encoded HTML. I haven't confirmed that on a preview with this change, so please check after deploy.

Change

  • src/http/etag.ts gains strongEtag. The tag is still a hash of the rendered body, and holds still compares weakly.
  • layout.ts uses strongEtag for every page. /changelog builds its own tag, so I made that tag strong too.
  • Cache-Control and Vary are unchanged, and signed-in pages stay private, never tagged, and never stored by a shared cache.

Page routes without an ETag

  • Signed-in or admin pages (/board, /flow, /settings/* and the rest): untagged by design.

Tests

  • test/etag.test.ts now checks nine public pages. Each must return 200 with a strong tag that survives brotli, HEAD must return the same tag, and a held tag must get a 304 in under 50 ms. The 304 must keep the 200's Cache-Control and Vary and must not be private. A separate test checks that a signed-in page is never public.
  • The weak-tag assertions in etag.test.ts and changelog-cache.test.ts were wrong for the new tags, so I changed them to strong.
  • I ran test/etag.test.ts, the changelog tests, test/forge-webspec.test.ts and test/web-ui.test.ts (160 pass), plus tsc, gate.ts and the design lint (all clean). The full suite is left to the forge.

Not done

  • The issue says weak, but a weak tag is what the edge drops. A strong tag shared across br and gzip is the same trade-off /llms.txt already makes.
  • I didn't measure the 304's Server-timing under 5 ms in production. Locally the 304 test passes the 50 ms bound I set.
  • The desk note "production is rolling, hold changes to it" doesn't block this: it goes through a pull and preview, not straight to production.

The push needed a "Try:" commit to pass the pre-push hook, so the second commit is empty. To try it: curl -sI -H 'accept-encoding: br' https://lab--<pull>--forge-trevato-apps.open-platform.sh/explore should now show an etag, and sending it back in If-None-Match should return 304.

Closes #14277

13 earlier comments · 3 events
commented

Pages now carry a strong ETag instead of a weak one, because the app already tagged them and Cloudflare was stripping the weak tag. The tests I ran locally pass, but the production behaviour is not yet checked on a preview.

Cause. src/web/layout.ts already tagged every signed-out page and answered 304, and a local server sends the ETag on /explore with or without Accept-Encoding: br. Production and the #14296 preview send none, though If-None-Match: * already gets a 304 there. /llms.txt is also brotli-encoded and keeps its strong tag in production, while a weak tag on an encoded page disappears. That points at the edge dropping weak ETags on encoded HTML. I haven't confirmed that on a preview with this change, so please check after deploy.

Change

  • src/http/etag.ts gains strongEtag. The tag is still a hash of the rendered body, and holds still compares weakly.
  • layout.ts uses strongEtag for every page. /changelog builds its own tag, so I made that tag strong too.
  • Cache-Control and Vary are unchanged, and signed-in pages stay private, never tagged, and never stored by a shared cache.

Page routes without an ETag

  • Signed-in or admin pages (/board, /flow, /settings/* and the rest): untagged by design.

Tests

  • test/etag.test.ts now checks nine public pages. Each must return 200 with a strong tag that survives brotli, HEAD must return the same tag, and a held tag must get a 304 in under 50 ms. The 304 must keep the 200's Cache-Control and Vary and must not be private. A separate test checks that a signed-in page is never public.
  • The weak-tag assertions in etag.test.ts and changelog-cache.test.ts were wrong for the new tags, so I changed them to strong.
  • I ran test/etag.test.ts, the changelog tests, test/forge-webspec.test.ts and test/web-ui.test.ts (160 pass), plus tsc, gate.ts and the design lint (all clean). The full suite is left to the forge.

Not done

  • The issue says weak, but a weak tag is what the edge drops. A strong tag shared across br and gzip is the same trade-off /llms.txt already makes.
  • I didn't measure the 304's Server-timing under 5 ms in production. Locally the 304 test passes the 50 ms bound I set.
  • The desk note "production is rolling, hold changes to it" doesn't block this: it goes through a pull and preview, not straight to production.

The push needed a "Try:" commit to pass the pre-push hook, so the second commit is empty. To try it: curl -sI -H 'accept-encoding: br' https://lab--<pull>--forge-trevato-apps.open-platform.sh/explore should now show an etag, and sending it back in If-None-Match should return 304.

commented

<!-- look-pages bbec238133dc --> Look pages for bbec238, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/changelog.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Pages now send a strong ETag instead of a weak one. Production sent none on / and /explore, because the edge drops a weak ETag from encoded HTML. The edge keeps a strong one, so a repeat visit can send it back in If-None-Match and get a 304.

  • The fix: src/http/etag.ts gets a small helper that makes a tag strong, src/web/layout.ts uses it for the pages built there, and the changelog page takes its tag through a local strongOf in src/web/pages/changelog.ts.
  • Tests: test/etag.test.ts and test/changelog-cache.test.ts cover it. The 25 tests in those two files pass, and tsc --noEmit shows no errors.
  • The failed gate: the "both doors" rule wanted a terminal change or a docs/features how-to for the changelog.ts edit. The edit only changes a header validator and shows nothing new on the page, so the latest commit carries a No door: line saying so. The gate rule no longer fires locally.
  • Left alone: the over-100-column comment in test/changelog-cache.test.ts is only a gate note, and the nightly rewrap handles it.
  • Not run: I did not run the preview check.

To try it, run curl -sI -H 'accept-encoding: br' https://lab--14300--forge-trevato-apps.open-platform.sh/explore and look for an etag that doesn't start with W/. Send that value back in If-None-Match and the response should be a 304.

commented

<!-- look-pages eceb03aedd22 --> Look pages for eceb03a, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/changelog.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Pages now send a strong ETag instead of a weak one, so a repeat visit can answer a 304. The edge drops a weak ETag from encoded HTML, which is why / and /explore sent none in production.

This push fixes the one failure the pull caused. test/etag.test.ts held a 304 to a 50 ms clock, which the repo's "no raw wall-clock budget" rules reject. I removed that assertion and the 304 checks stay. The etag tests pass and tsc --noEmit is clean.

Two other failures I did not fix:

  • counts.test.ts: the "issue listing costs the same for 5 rows as for 50" test passes on its own, twice. It failed in the check (and once here when run together with flake.test.ts) with 8 statements against 17. That test calls the issues API, which this pull doesn't touch, so I read it as order-dependent. I haven't proven that, and the forge counted it against this pull. If it fails again, that needs a look.
  • flake.test.ts: also red in the check and not counted against main. I did not look into it beyond running it together with counts.test.ts.

The other failures, in stuck-pile.test.ts, deadends.test.ts and keeper-measures.test.ts, are red on main too, and the forge did not count them against this pull.

Change: src/http/etag.ts adds strongEtag, src/web/layout.ts uses it for server-rendered pages, and src/web/pages/changelog.ts makes its own tag strong.

Try it: run curl -sI -H 'accept-encoding: br' https://lab--14300--forge-trevato-apps.open-platform.sh/explore. The etag should not start with W/. Send it back in If-None-Match and the answer should be a 304.

commented

<!-- look-pages 8460702b9b99 --> Look pages for 8460702, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/changelog.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Pages now send a strong ETag instead of a weak one, so a repeat visit to / or /explore can answer 304. The edge drops a weak ETag from encoded HTML, which is why production sent none on those pages. This round needed no code change, and I pushed nothing.

  • Look pages: the feedback is the gate's automatic list. It inferred /, /docs, /explore and 25 more from src/web/layout.ts and src/web/pages/changelog.ts. That list is correct for this change, so I added no Looks: or No look: line. Only the ETag header changes and no page shows anything different.
  • Release hold: the note says production is rolling and changes to it should wait. This pull only deploys a preview, and a person merges it, so it doesn't touch production. It should stay unmerged until the release settles. The hold did not shape the change.

Change

  • src/http/etag.ts adds strongEtag, a quoted tag with no W/. holds still compares weakly.
  • src/web/layout.ts uses it for server-rendered pages, and src/web/pages/changelog.ts makes its own tag strong.
  • test/etag.test.ts checks nine pages: the tag is strong and survives br, HEAD carries the same tag, and the 304 keeps Cache-Control and Vary. It has no wall-clock bound because the repo's rules forbid one.

Not done

  • The edge's behaviour is not tested, since the tests can't see it. The check is the preview or production curl below.
  • A counts.test.ts test ("an issue listing costs the same for 5 rows as for 50") and flake.test.ts failed in an earlier check. counts.test.ts passes on its own and calls an API this pull doesn't touch. I didn't investigate either, so a repeat failure of those two needs a look.

Try it: run curl -sI -H 'accept-encoding: br' https://lab--14300--forge-trevato-apps.open-platform.sh/explore. The etag should not start with W/. Send it back in If-None-Match and the answer should be 304.

Changes requested — before the latest push

The preview at this head does not show the fix. GET /explore with accept-encoding br, gzip and identity all return 200 with public cache-control and NO etag header. The pull's own try-it (an etag not starting with W/) fails. Either the edge drops strong tags on encoded HTML too (so the cause is unproven and the premise is wrong), or the app is not tagging with the bearer header; find which, and prove it on the preview before landing. Also: (1) the issue asks for a weak ETag and the pull ships strong, which is a departure; a strong tag across br/gzip variants is a correctness trade, so justify it with preview evidence. (2) Box 1 asks for the route checklist in the pull; /offline is excluded, fine, but the list is by hand. (3) Box 5 (304 under 5 ms) is unmeasured; the assertion was removed. Assert a 304 has an empty body and no render work instead. (4) changelog.ts: strongOf is inserted between a doc comment and cacheHeaders, detaching the comment; move it above the comment, and reuse strongEtag logic rather than a regex duplicate. (5) The new test claims to prove a tag 'survives br' but the test server does no encoding, so it cannot fail for the stated reason. The 'signed-in never stored' test checks only that public is absent, and does not assert private. No docs/features or TUI door is touched, acceptable as header-only.

commented

The strong tag did not fix it, so I backed it out; src/ is now identical to the base. This pull no longer lands the fix. It adds tests only, and the missing ETag on / and /explore is still unsolved.

What the preview showed. At the previous head, /explore and /pricing returned no etag on the 200 under br, gzip or identity. The app does tag them: I hashed the /pricing body, sent that value as If-None-Match (both "tag" and W/"tag"), and got a 304 each time. So Cloudflare removes the ETag from these HTML responses, strong or weak, and the premise of my earlier commits was wrong. /llms.txt does keep a tag, but it arrived weak under identity and strong under br, so the edge changes tags too, and I can't say why.

Review points

  • Strong vs weak: the issue asked for weak, and the strong tag bought nothing. Both are gone, along with strongEtag and the strongOf regex in src/web/pages/changelog.ts, so the detached comment is moot.
  • "Survives br" claim: the test server does no encoding, so the claim was unfounded and I dropped it.
  • Clock assertion: replaced by checks that a 304 has an empty body and the same cache-control and vary as the 200.
  • Signed-in page: now asserted no-cache or private, not just not public.
  • Route checklist: the new test covers /, /explore, /changelog, /docs, /pricing, /apps, /security, /privacy and /terms. Each must have a weak tag, a HEAD that matches, and an empty 304. The list is still by hand.
  • Not done: I haven't found why the edge strips the header. That needs a Cloudflare rule, a platform setting or a different validator, and none of those can be changed from this repository.

etag.test.ts and changelog-cache.test.ts pass (25 tests) and tsc --noEmit is clean. I did not run the full suite.

To check what I found, run curl -sI -H 'accept-encoding: identity' -H "authorization: Bearer $FORGE_TOKEN" https://lab--14300--forge-trevato-apps.open-platform.sh/explore. It shows no etag, while an If-None-Match with the body's hash still gets a 304.

Changes requested — before the latest push

The pull adds tests only; src/ is identical to the base. On the preview, GET / and /explore return 200 with public cache-control and no etag header (also /pricing). So the issue's goal, a repeat visit answering 304, is not met where a person sees it. The tests prove the app tags and answers 304 in-process, which already worked before this pull.

Changes needed in one round:

  1. Prove on this preview why the edge drops the tag, and make the pull fix it or say plainly it cannot. Options: a validator the edge keeps (Last-Modified, or an ETag on a path the edge passes) or an owner-side platform rule filed as an issue. A tests-only pull cannot close #14277.
  2. Box 1 asks for the route checklist in the pull. The list is a hand-kept array in the test. Derive it from the route table or add a test that fails when a public page route is missing from it, and name /offline and the signed-in routes as exclusions.
  3. Box 5, a 304 under 5 ms, is unmeasured. Assert the 304 does no render work, for example that the render function is not called, rather than a clock.
  4. The signed-in test accepts no-cache OR private, so it does not pin the contract. Assert the exact header the page sends, and also assert that a signed-in request with a matching If-None-Match gets a 200, not a 304.
  5. The comment says 'see the pull'. Put the finding into the test comment or docs so it does not depend on a pull description.

No door or docs change is needed (header-only).

commented

Signed-out pages now carry a Last-Modified header next to their weak ETag. This turns a repeat visit into a 304 through the edge, which strips the ETag from HTML. I checked it on this pull's preview with brotli: /, /explore, /pricing and /changelog each answer a matching If-Modified-Since with 304 and no body.

Why the edge drops the tag (measured 2026-11-19 on production and this preview). It depends on content type, not on the path or whether the tag is weak or strong:

  • ETag dropped: every text/html answer, including /changelog, which has its own strong-style tag.
  • ETag kept: text/plain (/llms.txt), JSON (/api/v1/repos) and Atom (/changelog.atom).
  • Last-Modified on HTML: passes through, and a 304 comes back (shown above).

The forge proxy is not the cause. It sent the pages through as they came (no toolbar injected). This finding is written into the comment above the page walk in test/etag.test.ts and into src/http/etag.ts unchanged, so it does not depend on this description.

What changed

  • src/http/page_stamp.ts (new) gives each URL the date its current bytes were first sent. When the bytes change, the date moves to a later second, so a page that goes A→B→A gets three dates and a held copy of B never matches. A restart or an evicted URL starts again at now, which costs a 200, never a stale 304. It keeps at most 2048 URLs.
  • unchanged() in src/http/etag.ts: a request that sends If-None-Match is judged by the tag, which wins over the date (RFC 9110 §13.1.3). Otherwise it is judged by If-Modified-Since.
  • page() in src/web/layout.ts sends both validators on every page it tags. /changelog's own early 304 now carries the same date.

Review points

  1. Proven above; the pull fixes it rather than filing a platform rule.
  2. The walked list is now built from HEADS in src/http/route_marks.ts: every open page that answers a stranger with 200 HTML must carry both validators and answer either one with an empty 304. Admin and signed-in routes are not open, so they are not walked. The named exclusions are:
    • /offline: the service worker's fallback page.
    • /status: the standalone health page, fresh for 10 s by max-age.

    A guard test checks that the main pages are in the list. Each crawler-limit check sees a fresh minute, so the walk cannot pass by quietly getting 429s.

  3. I disagree that this one can be done in this round. A 304 here still renders the page, because the tag is the hash of the rendered bytes. I tried answering repeat visits from the existing write-clock cache in http/repeat_reads.ts and backed it out:
    • Signed-out pages read in-memory state the write clock cannot see. With the shortcut, test/changelog-cache.test.ts "the tag changes when a release lands" fails.
    • The extra clock read put /pricing at 11 SQL statements against a budget of 10.

    The safe form is to compute each page's tag from its inputs before rendering, as /changelog already does. That is filed as #14411, and #14277 now waits for it. On the preview, the / and /explore answers that a 304 skips report about 2–4 ms of app time.

  4. The signed-in test now asserts exactly cache-control: no-cache, no etag and no last-modified. A signed-in request sending a matching If-None-Match and If-Modified-Since gets a 200. The earlier no-cache|private check was too loose, and that test was wrong, not the code.
  5. Done: see the comments named above.

test/page-stamp.test.ts covers same-second changes and A→B→A.

Checks run: bun scripts/gate.ts --fix, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. scripts/fast.ts named 2609 files for this change, too broad to run here, so I picked the files that cover it and all 216 tests pass: etag, page-stamp, changelog-cache, changelog-etag, changelog, explore-etag, page-budget, home-cache, standards-etag, apps-proxy-etag and signed-out-home.looks.

Try it:

lm=$(curl -sI <preview>/explore | grep -i ^last-modified | cut -d' ' -f2-)
curl -s -o /dev/null -w '%{http_code}\n' -H "if-modified-since: $lm" <preview>/explore

The second command prints 304.

The desk hold "production is rolling" is about production. This pull changes nothing there until it is merged.

🤖 Generated with Claude Code

commented

<!-- look-pages cc4a622b2ed3 --> Look pages for cc4a622, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/changelog.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Server-rendered pages now answer a repeat visit with a 304. Until now they couldn't, because the edge strips ETag from text/html responses. Pages now carry a Last-Modified stamp (src/http/page_stamp.ts) beside the ETag. When a visit comes back with If-Modified-Since, the page answers 304 Not Modified instead of rendering again. This applies to every page that goes through src/web/layout.ts, and to the changelog page. Earlier commits tried a strong tag on the changelog; that was backed out because the preview showed the edge strips that too. The tests from that attempt stay.

The last push failed the gate's "pipeline first" rule: the pull adds a feature and doesn't touch the pipeline. That push now has an empty commit with a Direction: line saying why this goes ahead of the pipeline items: it fixes how pages are served rather than adding a new surface (#14277). No code changed in that commit.

The gate and bun x tsc --noEmit pass. scripts/fast.ts named 2381 test files for this diff, far too many to run in the sandbox, so I stopped that run and ran the covering tests by hand: page-stamp, stamped, etag, every -etag and changelog suite, and the direction suites. That is 541 tests in 71 files: 540 pass, 1 skipped, none fail. The full check runs on this push.

The gate also inferred that 31 pages (/, /docs, /explore and 28 more) need a look, because they import src/web/layout.ts. That's information only and doesn't fail the gate; I added no Looks: or No look: override.

To try it on the preview, run curl -sI against /explore and note the last-modified header. Send that value back in If-Modified-Since and the answer is 304 Not Modified.

🤖 Generated with Claude Code

commented

<!-- look-pages 5570f502b14e --> Look pages for 5570f50, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/changelog.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Server-rendered pages now answer a repeat visit with a 304 by date, because the edge strips ETag from text/html. Pages that go through page() in src/web/layout.ts now send Last-Modified beside their weak ETag. A request whose If-Modified-Since matches gets 304 Not Modified; when the request also sends If-None-Match, that wins (RFC 9110 §13.1.3), via unchanged() in src/http/etag.ts. The changelog page answers its own 304 before rendering, and that 304 now carries the same date.

src/http/page_stamp.ts sets the date. The forge doesn't know when a page's rows last changed, so the stamp is when this forge first sent these exact bytes for this URL. Whenever the bytes change, the stamp moves to a later second, even within the same second or when a page changes and changes back (A, B, A). A restart or an evicted URL starts again at now, which costs a full 200 but never gives a stale 304. Stamps are kept per forge in a WeakMap keyed by ctx.app (as askingOf(app) does, #12569) and capped at 2048 URLs, oldest dropped. That fixes the last check's module-state failure: a shared module-level Map is no longer allowed.

On the look-pages feedback: no page renders a byte differently, only response headers change. So the commit carries a No look: line instead of looking at the 28 pages that import the layout.

  • Other failures in the last check: pull-check-again and attention (the first-view budget) pass here, so I take them as flakes when the shards are loaded. The ask-to-reach failure was the test failing to listen on a port. The check's own log reports llms-budget red on main as well.
  • Tests run: the gate and bun x tsc --noEmit pass. page-stamp, module-state, stamped, etag, explore-etag, changelog, changelog-etag and changelog-cache pass (207 tests), as do pull-check-again and attention. scripts/fast.ts named 2381 test files for this diff, too many to run in the sandbox, so I picked the files above by hand.
  • Try it on the preview: curl -sI /explore, then send its last-modified value back as If-Modified-Since; the answer is 304.

🤖 Generated with Claude Code

Approved

Earlier asks: (1) edge cause proven and fixed with Last-Modified, measured in the test comment and etag.ts: met. (2) list derived from HEADS with /offline and /status named as exclusions, plus a guard test: met. (3) no render-skip: answered with #14411, non-blocking. (4) signed-in test pins exact no-cache, no validators, 200 on matching validators: met. (5) finding is in the comments: met. Preview: /explore answers 304 to its Last-Modified in If-Modified-Since; / carries the date and public cache headers. Non-blocking: the stamp is per-process, so a restart costs a 200, never a stale 304 - acceptable.

commented

Left #14277 open: 1 unticked item remain (Server-timing for the 304 is under 5 ms).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs