Public: anyone can read this; the crew can change it.
Sentinel: shared:01m23vygyffx5hg2f5vm8r74sr:3vwjk71z5u is past its budget: trevato/forge: "a page that answers 200 names its canonical address, equal to og:url; a 404 names none" failed on 3 pul… #8238
Check · Review · Land · Live done — Landed and live in production.
The canonical-address test no longer depends on how loaded the check machine is. I couldn't reproduce the failure here, so the cause below is an inference from reading the code. `test/forge-webspec.test.ts` passes (3 runs before the change, 1 after), `tsc` is clean and `bun scripts/gate.ts` printed nothing in its last lines. I did not run the full suite, so no other test file was exercised.
shipped production is serving it deploy b070674
The canonical-address test no longer depends on how loaded the check machine is. I couldn't reproduce the failure here, so the cause below is an inference from reading the code. test/forge-webspec.test.ts passes (3 runs before the change, 1 after), tsc is clean and bun scripts/gate.ts printed nothing in its last lines. I did not run the full suite, so no other test file was exercised.
Cause. Since #8134, a request with no User-Agent and no session is counted as a crawler against a per-minute budget. That budget is divided by 4 whenever the event loop is running late (BEHIND_DIVISOR in src/http/crawl.ts). The webspec tests send many requests with no User-Agent to one app, so on a loaded machine the later ones can get a 429. The test then fails on expect(status).toBe(200).
Change. The tests in test/forge-webspec.test.ts now send a browser User-Agent on every request. That covers the canonical-address test, the webspec walks, and the two page-exists checks. The through helper adds it only when a request has none. No production code changed.
Not done. I did not exempt in-process or test requests from the crawl budget, because that would change production behaviour.
To try it. Run bun test test/forge-webspec.test.ts. The sentinel cause shared:01m23vygyffx5hg2f5vm8r74sr:3vwjk71z5u should clear once the check passes on this push.
🤖 Generated with Claude Code
Test-only change; every request in test/forge-webspec.test.ts now carries a browser User-Agent, so the crawl budget (#8134/#8129) cannot 429 them under load. No production code touched, which is the right call. I checked that no http.handle/Request call in the file is left without it. Cause is inferred, not reproduced, but it fits crawl.ts (no UA counts as a crawler, budget divided by 4 when behind). Two things next: 1) give the test app a crawl-clock seam (setCrawlClock) so the file cannot depend on machine load at all; 2) fold the four repeated header spreads into one helper like through does, so a new request cannot forget it.
The canonical-address test no longer depends on how loaded the check machine is. I couldn't reproduce the failure here, so the cause below is an inference from reading the code.
test/forge-webspec.test.tspasses (3 runs before the change, 1 after),tscis clean andbun scripts/gate.tsprinted nothing in its last lines. I did not run the full suite, so no other test file was exercised.Cause. Since #8134, a request with no
User-Agentand no session is counted as a crawler against a per-minute budget. That budget is divided by 4 whenever the event loop is running late (BEHIND_DIVISORinsrc/http/crawl.ts). The webspec tests send many requests with noUser-Agentto one app, so on a loaded machine the later ones can get a 429. The test then fails onexpect(status).toBe(200).Change. The tests in
test/forge-webspec.test.tsnow send a browserUser-Agenton every request. That covers the canonical-address test, the webspec walks, and the two page-exists checks. Thethroughhelper adds it only when a request has none. No production code changed.Not done. I did not exempt in-process or test requests from the crawl budget, because that would change production behaviour.
To try it. Run
bun test test/forge-webspec.test.ts. The sentinel causeshared:01m23vygyffx5hg2f5vm8r74sr:3vwjk71z5ushould clear once the check passes on this push.🤖 Generated with Claude Code
Closes #8233