Public: anyone can read this; the crew can change it.

The sitemap timing test is a coin flip under load #6194

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The sitemap timing test no longer asserts a raw 15 ms. `test/sitemap.test.ts` now passes locally (2 pass, 0 fail). I ran only that file, not the type check, and haven't seen the forge's check verdict on the push.

shipped production is serving it deploy 309b41e

opened this

The sitemap timing test no longer asserts a raw 15 ms. test/sitemap.test.ts now passes locally (2 pass, 0 fail). I ran only that file, not the type check, and haven't seen the forge's check verdict on the push.

  • Best of five: the test now clears the listing cache and rebuilds /sitemap.xml five times, keeping the lowest summed Server-Timing phase time.
  • Budget helper: that best time goes through expectBudget from test/helpers/budget.ts with a 15 ms budget. The baseline is baselineMs, a /healthz measurement taken in the same run. Under load the limit stretches with the baseline, up to at most 60 ms. With FORGE_TEST_QUIET=1 the absolute 15 ms applies.
  • Unchanged: the content checks (100 public repos, no private ones) and the presence of the repos, render and etag timing phases are as before.
  • Left out: I didn't touch the ETag/304 test, and I didn't check other tests for raw timing assertions.

To try it, run bun test test/sitemap.test.ts.

Closes #6193

Approved

Uses the repo's own expectBudget/baselineMs helper (#4429) and takes the best of 5 cold builds (listings forgotten each round), so one load spike no longer fails the test while a real regression still does. Test-only, single purpose, no new dependency. Next: (1) the test title still says "under 15 ms" though the limit is now load-relative; reword it. (2) Consider a sitemap-specific assertion that the cache really is cold each round, so best-of-5 cannot silently measure a warm path.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs