Public: anyone can read this; the crew can change it.

One test walks every route the server registers and fails on any response without the security floor #10397

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

Adds `test/security-sweep.test.ts`: one test file that reads the route table from both routers (977 entries: 460 GET, 446 POST, 34 DELETE, 25 PUT, 12 PATCH) and runs three passes against one seeded server. The sweep found no response missing the floor, so `src/http/respond.ts` is unchanged.

shipped production is serving it deploy a5a2512

opened this

Adds test/security-sweep.test.ts: one test file that reads the route table from both routers (977 entries: 460 GET, 446 POST, 34 DELETE, 25 PUT, 12 PATCH) and runs three passes against one seeded server. The sweep found no response missing the floor, so src/http/respond.ts is unchanged.

  • Anonymous, every registered (verb, route) once. It also asks /healthz, /readyz and an unknown studio path and API path. Every answer must carry Content-Security-Policy, X-Content-Type-Options and Referrer-Policy. The test asserts it actually met a 2xx, a 3xx, a 404, another 4xx and an event stream, so it can't pass on empty coverage. Mutations are sent from another origin carrying the seeded account's real username and password, and none may return a forge_session cookie, which catches login CSRF on any route. POST /signin and POST /api/v1/sessions already refuse this with a 403.
  • A wrong verb on every API pattern. The answer must be a 405 with the floor, and its Allow header must equal apiRouter.allowed(path). A wrong verb on a studio path gets the not-found page; that is checked once, with PATCH /.
  • Every mutating route, cross-origin, with a live session cookie. Each must answer 403 (the originGate refusal) and set no cookie, and afterwards the session must still be valid.

Every Set-Cookie seen in any pass must be HttpOnly. A real same-origin sign-in, through both the form and the API, must set forge_session HttpOnly.

Exception lists: each entry carries its reason, and the tests check the entries both ways.

  • FLOOR_EXCEPTIONS is empty, because the probes and assets also leave through the one door (answered) that applies the floor.
  • READABLE_COOKIES is empty.
  • NO_405 names /api/v1, /api/v1/ and /metrics, which answer a wrong verb with 404 by design.

I checked that each guarantee is really tested: removing Referrer-Policy from securityFloor, short-circuiting originGate, and dropping HttpOnly from sessionCookie each turn tests red.

Timing — the "under 5 s" item is not enforced: the passes took about 3 s when the file ran alone, but about 5 s when run with the other fast-tier files. Most of that is the first render of each page module (cold 2.7 s, warm 1.1 s for the 460 GETs). Hard 5 s timeouts timed out in the batch run, so the timeouts are flake guards instead (6 s, 2 s, 2 s, 1 s), in line with the "count work, not a clock" principle. What keeps the cost in check is the shape: one request per (verb, route) per pass.

Not done: the anonymous pass fills only the owner, repository, username and issue number from the seed; other parameters get a placeholder, so those routes answer their miss. Deeper GET filling, signed in and out, stays in test/security-floor-routes.test.ts, which this doesn't replace. I couldn't call the keep_plan tool from this session, so the issue has no plan posted.

Try it: bun test test/security-sweep.test.ts. bun scripts/gate.ts, bun x tsc --noEmit and the 18 files scripts/fast.ts names (219 tests) pass.

🤖 Generated with Claude Code

Closes #10386

commented

Adds test/security-sweep.test.ts: one test file that reads the route table from both routers (977 entries: 460 GET, 446 POST, 34 DELETE, 25 PUT, 12 PATCH) and runs three passes against one seeded server. The sweep found no response missing the floor, so src/http/respond.ts is unchanged.

  • Anonymous, every registered (verb, route) once. It also asks /healthz, /readyz and an unknown studio path and API path. Every answer must carry Content-Security-Policy, X-Content-Type-Options and Referrer-Policy. The test asserts it actually met a 2xx, a 3xx, a 404, another 4xx and an event stream, so it can't pass on empty coverage. Mutations are sent from another origin carrying the seeded account's real username and password, and none may return a forge_session cookie, which catches login CSRF on any route. POST /signin and POST /api/v1/sessions already refuse this with a 403.
  • A wrong verb on every API pattern. The answer must be a 405 with the floor, and its Allow header must equal apiRouter.allowed(path). A wrong verb on a studio path gets the not-found page; that is checked once, with PATCH /.
  • Every mutating route, cross-origin, with a live session cookie. Each must answer 403 (the originGate refusal) and set no cookie, and afterwards the session must still be valid.

Every Set-Cookie seen in any pass must be HttpOnly. A real same-origin sign-in, through both the form and the API, must set forge_session HttpOnly.

Exception lists: each entry carries its reason, and the tests check the entries both ways.

  • FLOOR_EXCEPTIONS is empty, because the probes and assets also leave through the one door (answered) that applies the floor.
  • READABLE_COOKIES is empty.
  • NO_405 names /api/v1, /api/v1/ and /metrics, which answer a wrong verb with 404 by design.

I checked that each guarantee is really tested: removing Referrer-Policy from securityFloor, short-circuiting originGate, and dropping HttpOnly from sessionCookie each turn tests red.

Timing — the "under 5 s" item is not enforced: the passes took about 3 s when the file ran alone, but about 5 s when run with the other fast-tier files. Most of that is the first render of each page module (cold 2.7 s, warm 1.1 s for the 460 GETs). Hard 5 s timeouts timed out in the batch run, so the timeouts are flake guards instead (6 s, 2 s, 2 s, 1 s), in line with the "count work, not a clock" principle. What keeps the cost in check is the shape: one request per (verb, route) per pass.

Not done: the anonymous pass fills only the owner, repository, username and issue number from the seed; other parameters get a placeholder, so those routes answer their miss. Deeper GET filling, signed in and out, stays in test/security-floor-routes.test.ts, which this doesn't replace. I couldn't call the keep_plan tool from this session, so the issue has no plan posted.

Try it: bun test test/security-sweep.test.ts. bun scripts/gate.ts, bun x tsc --noEmit and the 18 files scripts/fast.ts names (219 tests) pass.

🤖 Generated with Claude Code

Approved

A test-only change, and it does what the issue asks. One sweep reads both routers and sends each (verb, route) once, anonymously. It asserts the floor on 2xx, 3xx, 404, other 4xx and the event stream, and it fails if one of those kinds is never met. It sends cross-origin mutations with the seeded credentials and asserts no session comes back. It checks 405 and Allow for wrong verbs. It sends every mutation cross-origin with a live session cookie and expects 403, no Set-Cookie, and the session still valid. It checks HttpOnly on every cookie seen, plus a real same-origin sign-in through both the form and the API. The exception tables are checked both ways, so a stale entry fails. Nothing here needs a TUI screen or a docs/features page, and nothing was added to README. No new dependency. I did not run the preview or the tests myself; this verdict rests on reading the diff and the pull description.

Two things I would do next:

  1. Enforce the time budget. Box 5 is not met: the file gives 6 s, 2 s, 2 s and 1 s flake guards, and the author measured about 5 s when it runs alongside other fast-tier files. Either cut the cost, for example by skipping the cold first render of heavy page modules for routes that answer the same way, or take the sweep out of the fast tier. Fold it under the per-file time budget in #1242 instead of leaving it a comment.
  2. Fill deeper parameters (pulls, runs, apps) for the mutating and non-GET routes. Today they only answer their placeholder miss. The sweep proves the floor on a miss and does not yet prove it on a hit.
commented

Left #10386 open: 1 unticked item remain (The sweep runs under 5 s.).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs