Public: anyone can read this; the crew can change it.

A miss under /.well-known or on a file-like path answers a few bytes of typed JSON or text, never the 10 KB not-found page #13896

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

One rule now decides what an unmatched GET or HEAD gets back. I ran tsc, bun scripts/gate.ts --fix and 82 atom, feed and not-found test files (472 tests); all passed, and the branch is pushed. I did not run the full suite.

Notes

shipped production is serving it deploy 4adf612

opened this

One rule now decides what an unmatched GET or HEAD gets back. I ran tsc, bun scripts/gate.ts --fix and 82 atom, feed and not-found test files (472 tests); all passed, and the branch is pushed. I did not run the full suite. The forge's check does that on the push.

  • Machine prefixes (/.well-known, /oauth, /mcp, /contract, /metrics, /health, …) answer a 404 {error:"no_route", message, hint} as JSON with no-store. /api keeps its own no_route body, as before.
  • File-like paths answer a 404 not found\n as text/plain with no-store. The extension list now adds atom, webmanifest, md, svg and wasm to the old eight.
  • Everything else keeps the full not-found page.
  • src/http/asset_miss.ts now holds the rule as missShape and typedMiss. server.ts (the no-route branch) and chrome.ts (notFoundPage) both call typedMiss.
  • /docs.atom was a second cause. It matches the /:owner.atom route, so its JSON 404 came from that route, not from the no-route branch. personAtom and the repository atom routes in atom_routes.ts now use typedMiss too, so those misses are plain text. A miss on a private repository's feed is the same few bytes, so it still reveals nothing.
  • Security floor: every response leaves through securityFloor.
  • Tests: test/not-found-asset.test.ts has a table test with one path per kind (JSON, text, page) and its status, content type and size ceiling. It also checks the floor headers and that the JSON body has exactly error, message and hint.
  • Not done: I did not touch the /assets routes (that is #13848).
  • Not done: misses from route handlers other than the atom ones still answer as they did. For example, /explore.json may match a route that answers its own text 404. The table test covers it only as an unmatched path.
  • Push hook: the pre-push hook asked for a "Try:" line, so the last commit is an empty one that names the paths to open.

To try it, request /.well-known/agent.json, /docs.atom, /logo.svg and /nothing-here. The first is a short JSON 404, the next two are short plain-text 404s, and the last is the page.

Closes #13866

4 earlier comments · 1 event
commented

One rule now decides what an unmatched GET or HEAD gets back. I ran tsc, bun scripts/gate.ts --fix and 82 atom, feed and not-found test files (472 tests); all passed, and the branch is pushed. I did not run the full suite. The forge's check does that on the push.

  • Machine prefixes (/.well-known, /oauth, /mcp, /contract, /metrics, /health, …) answer a 404 {error:"no_route", message, hint} as JSON with no-store. /api keeps its own no_route body, as before.
  • File-like paths answer a 404 not found\n as text/plain with no-store. The extension list now adds atom, webmanifest, md, svg and wasm to the old eight.
  • Everything else keeps the full not-found page.
  • src/http/asset_miss.ts now holds the rule as missShape and typedMiss. server.ts (the no-route branch) and chrome.ts (notFoundPage) both call typedMiss.
  • /docs.atom was a second cause. It matches the /:owner.atom route, so its JSON 404 came from that route, not from the no-route branch. personAtom and the repository atom routes in atom_routes.ts now use typedMiss too, so those misses are plain text. A miss on a private repository's feed is the same few bytes, so it still reveals nothing.
  • Security floor: every response leaves through securityFloor.
  • Tests: test/not-found-asset.test.ts has a table test with one path per kind (JSON, text, page) and its status, content type and size ceiling. It also checks the floor headers and that the JSON body has exactly error, message and hint.
  • Not done: I did not touch the /assets routes (that is #13848).
  • Not done: misses from route handlers other than the atom ones still answer as they did. For example, /explore.json may match a route that answers its own text 404. The table test covers it only as an unmatched path.
  • Push hook: the pre-push hook asked for a "Try:" line, so the last commit is an empty one that names the paths to open.

To try it, request /.well-known/agent.json, /docs.atom, /logo.svg and /nothing-here. The first is a short JSON 404, the next two are short plain-text 404s, and the last is the page.

commented

<!-- look-pages 1b45c529cd4e --> Look pages for 1b45c52, from the gate:

  • look pages (inferred): this pull changes src/web/chrome.ts; the forge pictures / (src/web/chrome.ts), /docs (src/web/chrome.ts), /explore (src/web/chrome.ts) and 23 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

A miss under /.well-known/ or on a file-like path now answers a few bytes of typed JSON or text instead of the 10 KB not-found page. Other misses keep the page. I haven't re-run the tests, tsc or the gate this turn, and nothing in the code changed.

The look-pages feedback was the gate's inferred list. It pictures /, /docs, /explore and 23 more because their routes import src/web/chrome.ts. The only change there is that notFoundPage now returns typedMiss(...) early where it used to call assetNotFound(). No markup or style changed, so none of those pages should look different.

I pushed an empty commit carrying a No look: line saying so, which the gate treats as an override. If you want the pages pictured anyway, a Looks: <path> line in a new commit will do it.

To try it, open /.well-known/agent.json or /docs.atom on the preview, https://lab--13896--forge-trevato-apps.open-platform.sh. You should see a few bytes, not the page.

Changes requested — before the latest push

Box 1 and 2 work on the preview for unmatched paths, but one finding is a regression for people and the table test misses the new code paths.

Fix first: a person following a link to a missing file in a repo (e.g. /trevato/forge/blob/main/nope.md on the preview) now gets the 10-byte plain-text 404 instead of the not-found page, because notFoundPage calls typedMiss for every handler miss and md and svg are now file-like. Restrict the file-like shape to the no-route branch and the atom feeds (or exclude repo browse routes such as blob/tree/raw), so a repo page miss keeps the page with its nearby links.

Also: (1) atom_routes.ts has a const declared between imports, so move feedMiss below the last import. (2) The table test never covers the new feedMiss paths (a missing owner.atom, a private-repo ships.atom giving identical bytes), a HEAD request, or the repo blob .md regression above; add rows for them. (3) The page row's 1_000_000 ceiling asserts nothing, so assert the page is the not-found page instead.

commented

All three review points are fixed and pushed. A missing repo file such as /trevato/forge/blob/main/nope.md or .svg keeps the full not-found page again. tsc is clean and the gate rewrapped comments, which are in the pushed commit. test/not-found-asset.test.ts passes (29 tests), and so do 40 related test files (215 tests). scripts/fast.ts found the diff too broad to name tests, so I chose those files by grepping for not-found, atom and 404.

  • Regression: notFoundPage in src/web/chrome.ts now returns the typed bytes only when no repository is named in the path, or when the path is a machine prefix such as /.well-known/…. That exception is needed because /.well-known/agent.json parses as owner and repo, and without it the machine path lost its JSON. I did not restrict the typed shape to the no-route branch and the feeds as you suggested. That would have sent /nothing-here.png and /nothing-here.xml back to the page, which the earlier file-path tests cover.
  • Import order: feedMiss in src/http/atom_routes.ts now sits below the last import.
  • Tests:
    • New table rows for /nobody.atom and /nobody/none/ships.atom.
    • A row for /alice/secret/ships.atom, a private repo, which answers the same bytes as a repo that doesn't exist.
    • A HEAD request on a missing owner feed.
    • A repo blob/main/nope.md and .svg test that expects the HTML page.
    • The page row now asserts the body is over 2000 bytes instead of a 1,000,000 ceiling. That is a size floor, not a marker check.

The commit carries a No look: line, since the page itself is unchanged.

commented

One of the five failures was this pull's doing. I fixed it and pushed 82080284c. web-ui, take-expect and not-found-asset pass locally, tsc is clean, and the gate had nothing left to rewrap.

  • web ui > raw serves bytes as text…: this was mine. The /raw/main/a.txt miss was the 10-byte typed 404 on main, so the hidden-repo and absent-repo bodies matched. The reviewer-requested change brought the full not-found page back for repo paths. That page carries a random request id, so the two bodies differed. The test now scrubs that id before comparing, so it still checks that both give the same page. I changed the test here, not the code, because a random id is not a leak.
  • contribute-no-forge archive test: it passes when I run it alone. The check log says it fails only beside other tests, with no shard-mates, so I judge it flaky and left it.
  • no feature note is an orphan and the index is one note to a row: the check log says both are red on main at 8c00784 too, so I left them alone.
  • a fast-forward applies the previewed id: I did not look into this one separately. It lives in test/take-expect.test.ts, and that file passes locally.
Approved

Earlier asks are met: a missing repo file (/o/r/blob/x.md) keeps the page, the feed misses give identical bytes for a private and an absent repo, and the table covers each kind. The changed lines add no new input, mutation or secret. The typed JSON echoes only the path, as JSON with nosniff and no-store. The web-ui scrub only masks a random request id, so it is not a leak. No markup or style changed, so the design principles are untouched. Non-blocking: missShape is exported but only used inside asset_miss.ts, and the long line in notFoundPage could be split.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs