Public: anyone can read this; the crew can change it.

A test walks every public GET route signed out and fails on a 5xx, a slow answer or a missing security header #13881

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

This adds one new test, test/open-routes-floor.test.ts, which checks the security floor on every open GET route at once. It lists the GET routes from both routers that markOf in src/http/routemarks.ts marks open; 506 patterns today.

Notes

shipped production is serving it deploy 4adf612

opened this

This adds one new test, test/open-routes-floor.test.ts, which checks the security floor on every open GET route at once. It lists the GET routes from both routers that markOf in src/http/route_marks.ts marks open; 506 patterns today. It seeds alice with a public repository notes and issue #1, and fills path parameters from those. Parameters with nothing seeded behind them (:team, :env, :file, :slug, :token…) get a well-formed value, so those routes still have to send back a not-found with the floor. Each route is requested with no cookie and redirect: "manual", and must:

  • answer below 500;
  • carry content-security-policy, x-content-type-options: nosniff and referrer-policy;
  • send its headers within 1000 ms (200 ms with the 5x slack), with one retry before it counts as a failure.

Streams are not read: their headers are the answer. The walk takes about 2.8 s, and the slowest answer here was 131 ms.

What to look at:

  • Requests go through serve(app) and a real fetch on the loopback port, not forgeHttp(app).handle. The floor is set at the server's seam (server.ts answered → securityFloor). My first draft used the in-process handler, and almost every route came back without the three headers. So this test would fail if the floor were dropped from that seam.
  • A route nothing can fill is a failure, never a silent skip. Its pattern is listed with "nothing fills :param". UNFILLED is where such a route goes with its reason, and a test fails if an entry there is no longer a registered open GET route. The list is empty today because every parameter has a value.
  • The failure messages and the fill logic are tested on their own, so the check cannot pass by doing nothing.

Not done: admin routes, which test/route-marks.test.ts already covers; non-GET methods; and following redirects to their targets. The 3xx answer itself is checked.

To try it: bun test test/open-routes-floor.test.ts. bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. I couldn't record the issue plan: the keep_plan tool isn't available in this session.

🤖 Generated with Claude Code

Closes #13874

commented

This adds one new test, test/open-routes-floor.test.ts, which checks the security floor on every open GET route at once. It lists the GET routes from both routers that markOf in src/http/route_marks.ts marks open; 506 patterns today. It seeds alice with a public repository notes and issue #1, and fills path parameters from those. Parameters with nothing seeded behind them (:team, :env, :file, :slug, :token…) get a well-formed value, so those routes still have to send back a not-found with the floor. Each route is requested with no cookie and redirect: "manual", and must:

  • answer below 500;
  • carry content-security-policy, x-content-type-options: nosniff and referrer-policy;
  • send its headers within 1000 ms (200 ms with the 5x slack), with one retry before it counts as a failure.

Streams are not read: their headers are the answer. The walk takes about 2.8 s, and the slowest answer here was 131 ms.

What to look at:

  • Requests go through serve(app) and a real fetch on the loopback port, not forgeHttp(app).handle. The floor is set at the server's seam (server.ts answered → securityFloor). My first draft used the in-process handler, and almost every route came back without the three headers. So this test would fail if the floor were dropped from that seam.
  • A route nothing can fill is a failure, never a silent skip. Its pattern is listed with "nothing fills :param". UNFILLED is where such a route goes with its reason, and a test fails if an entry there is no longer a registered open GET route. The list is empty today because every parameter has a value.
  • The failure messages and the fill logic are tested on their own, so the check cannot pass by doing nothing.

Not done: admin routes, which test/route-marks.test.ts already covers; non-GET methods; and following redirects to their targets. The 3xx answer itself is checked.

To try it: bun test test/open-routes-floor.test.ts. bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. I couldn't record the issue plan: the keep_plan tool isn't available in this session.

🤖 Generated with Claude Code

commented

I added test/open-routes-floor.test.ts. Signed out, it asks every GET route that src/http/route_marks.ts marks open, from both routers. Path parameters are filled in from a seeded public repository (alice/notes, issue 1). Each answer must be below 500, carry content-security-policy, x-content-type-options: nosniff and referrer-policy, and send its headers within 1 s (200 ms with 5× slack). A route that fails gets one retry, so a slow first request doesn't fail the run.

The first push failed the forge's check because the whole walk ran as one test, which took 2.17 s against the 2 s per-test budget. The walk is now one test per route. A separate test checks that more than 100 routes are covered and that every route parameter has a fill value or a listed reason in UNFILLED. The routes are asked one at a time, never in parallel, so the timing check stays honest. Locally, the file runs 510 tests in about 4 s. The 20 s per-file budget still applies, and I did not add the file to SLOW.

The check also failed one other test: test/boot-production.test.ts ("the engine's Worker starts at boot…"). This pull only adds a test file, and that suite passes here, so the failure looks specific to that check shard.

bun scripts/gate.ts --fix and bun x tsc --noEmit are clean.

To try it: bun test test/open-routes-floor.test.ts

🤖 Generated with Claude Code

Approved

Approved. One new test file walks all open GET routes from both routers through serve() with no cookie, checks status <500, the three floor headers and the 5x slack with one retry, and fails on any parameter nothing fills. It adds no product surface, so there is nothing for a phone or keyboard to hit. No README or dependency change. The helper tests keep the checker from passing by doing nothing. Next, two things I would do: (1) add a seeded 404 and a redirect assertion so a route answering 404 for everything it was filled with is flagged, because status below 500 alone lets a mis-filled route pass; (2) fix the garbled name of the first test ("nothing fills is left unlisted") and consider also walking with an Accept: application/json ask for the API router.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs