Public: anyone can read this; the crew can change it.

The forge holds 500 anonymous code pages a second, with people still fast #9461

Parked opened this in answer to
Closed without merging

This covers two of the six "Done when" items: commit-pinned pages are served without a git walk after the first read, and a repeatable load test now exists. The other four are not done. Nothing was run against a live copy of the forge or at 500 pages/s.

Notes

stopped parked by the crew deploy da11d51

opened this

This covers two of the six "Done when" items: commit-pinned pages are served without a git walk after the first read, and a repeatable load test now exists. The other four are not done. Nothing was run against a live copy of the forge or at 500 pages/s.

What changed

  • Blob pages: a stranger's page for a file at a full commit id on a public repository is now kept after the first read. The next read of that page spawns no git. Tree pages already did this.
  • Raw files: the bytes of a small /raw file (up to 1 MiB) at a full commit id are kept the same way. Larger files are not kept, so a few big ones can't fill the cache.
  • Who is served from the cache: only anonymous readers of public repositories get the cached blob page. Signed-in readers, branch names, tags and ?fragment=1 all still read git. The caches are in src/web/pages/tree_cache.ts, used from src/web/pages/repo.ts.
  • Test: a new case in test/anonymous-deep.test.ts moves the repository's objects directory away after the first read. The blob page and raw bytes still answer, and a path never read still returns 404. The test fails without the change.
  • Load test: bench/load.ts is an open-loop anonymous reader of pinned tree, blob and raw pages at a set rate. It prints the rate served, the refused count, p50 and p95 as JSON. --out writes the result into bench/results/.
  • Regression verdict: --compare <last.json> exits 1 when the served rate falls more than 20% from the last kept result. The judging code is src/perf/load_verdict.ts, with a hand-calculated test in test/load-verdict.test.ts. How to run it is in docs/features/load-test.md.

Checks: tsc --noEmit, scripts/gate.ts, the design lint and the touched tests pass (anonymous-deep, load-verdict, blob-window, blob-long-file, media). scripts/fast.ts named 1237 test files for this change, so I ran only the tests near it, not those 1237. I did not run bench/load.ts itself.

Not done

  • Profile: no profile of tree, blob, raw and media under production's shape is attached. That needs production data and a machine this sandbox can't reach.
  • Open questions: I did not find why production is far slower than the throwaway copy. The 0.7.384 ceiling that keys on event-loop lateness and should also read in-flight deep pages is untouched. The profile candidates (synchronous logging, prepare, toLocaleString, slow_sql callers) are untouched.
  • Main-thread claim: nothing here keeps a render from holding the main thread, so signed-in pages staying under 200 ms at p95 while strangers read at full rate is unmeasured.
  • 500/s target: the target of nothing refused and p95 under 100 ms, with and without the edge cache, is unmeasured.
  • Release check: the 20% comparison is not wired into the release check (scripts/cut-release.ts), and bench/results/ is empty. No baseline exists until someone runs the test from a separate machine against a copy.
  • Media pages: /media is unchanged; it already stats through the pooled reader and answers revalidations without reading bytes.

To try it: run bun bench/load.ts --url <copy> --repo owner/name --rate 500 --seconds 20 --version <v> --out bench/results/<v>.json. Run bun test test/anonymous-deep.test.ts test/load-verdict.test.ts for the tests.

🤖 Generated with Claude Code

Closes #9405

3 earlier comments
commented

This covers two of the six "Done when" items: commit-pinned pages are served without a git walk after the first read, and a repeatable load test now exists. The other four are not done. Nothing was run against a live copy of the forge or at 500 pages/s.

What changed

  • Blob pages: a stranger's page for a file at a full commit id on a public repository is now kept after the first read. The next read of that page spawns no git. Tree pages already did this.
  • Raw files: the bytes of a small /raw file (up to 1 MiB) at a full commit id are kept the same way. Larger files are not kept, so a few big ones can't fill the cache.
  • Who is served from the cache: only anonymous readers of public repositories get the cached blob page. Signed-in readers, branch names, tags and ?fragment=1 all still read git. The caches are in src/web/pages/tree_cache.ts, used from src/web/pages/repo.ts.
  • Test: a new case in test/anonymous-deep.test.ts moves the repository's objects directory away after the first read. The blob page and raw bytes still answer, and a path never read still returns 404. The test fails without the change.
  • Load test: bench/load.ts is an open-loop anonymous reader of pinned tree, blob and raw pages at a set rate. It prints the rate served, the refused count, p50 and p95 as JSON. --out writes the result into bench/results/.
  • Regression verdict: --compare <last.json> exits 1 when the served rate falls more than 20% from the last kept result. The judging code is src/perf/load_verdict.ts, with a hand-calculated test in test/load-verdict.test.ts. How to run it is in docs/features/load-test.md.

Checks: tsc --noEmit, scripts/gate.ts, the design lint and the touched tests pass (anonymous-deep, load-verdict, blob-window, blob-long-file, media). scripts/fast.ts named 1237 test files for this change, so I ran only the tests near it, not those 1237. I did not run bench/load.ts itself.

Not done

  • Profile: no profile of tree, blob, raw and media under production's shape is attached. That needs production data and a machine this sandbox can't reach.
  • Open questions: I did not find why production is far slower than the throwaway copy. The 0.7.384 ceiling that keys on event-loop lateness and should also read in-flight deep pages is untouched. The profile candidates (synchronous logging, prepare, toLocaleString, slow_sql callers) are untouched.
  • Main-thread claim: nothing here keeps a render from holding the main thread, so signed-in pages staying under 200 ms at p95 while strangers read at full rate is unmeasured.
  • 500/s target: the target of nothing refused and p95 under 100 ms, with and without the edge cache, is unmeasured.
  • Release check: the 20% comparison is not wired into the release check (scripts/cut-release.ts), and bench/results/ is empty. No baseline exists until someone runs the test from a separate machine against a copy.
  • Media pages: /media is unchanged; it already stats through the pooled reader and answers revalidations without reading bytes.

To try it: run bun bench/load.ts --url <copy> --repo owner/name --rate 500 --seconds 20 --version <v> --out bench/results/<v>.json. Run bun test test/anonymous-deep.test.ts test/load-verdict.test.ts for the tests.

🤖 Generated with Claude Code

commented

I added the missing "Try:" line the gate asked for and pushed. Nothing else changed.

The failing gate rule "say how to try it on the preview" wants a commit message with a Try: line whenever a pull touches src/web/pages/repo.ts or src/web/pages/tree_cache.ts. I couldn't amend the earlier commit, because force pushes are refused. So I added an empty commit, addd13c6, whose message carries the line.

The line reads: open /trevato/forge/tree/main twice on the preview and compare the second response time, then a commit-pinned blob page.

I did not rerun gate.ts, tsc or the tests before pushing. The only change is a commit message, so no code differs from the version whose check ran. The check on this push will show whether the gate now passes.

This pull keeps commit-pinned blob and raw pages after the first read. It also adds a repeatable load test that gives a 20% regression verdict (#9405).

commented

The failing test looks like a one-off timeout, not something my change caused. I haven't changed any code, and I pushed an empty commit (da11d51e) to make the forge run the check again.

  • The failure: "the runtime meters what it runs > deploy, sleep, wake and stop each open or close an interval" hit its 90-second timeout in shard 19.
  • Why I think it's a flake: run alone here, test/meter.test.ts passes all 8 tests in 2.8 seconds. My pull only touches src/web/pages/repo.ts and src/web/pages/tree_cache.ts, which that test doesn't reach.
  • Not verified: I didn't find out why it hung in the shard. If it fails again, I'll look at what that shard was running alongside it.
commented

Parked by the crew: the check stayed red through 2 automatic tries

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs