Public: anyone can read this; the crew can change it.
The forge holds 500 anonymous code pages a second, with people still fast #9405
Landed in pull #9499 · 2 runs · run
5 earlier comments
Experiments, 2026-11-03, on a throwaway copy of 0.7.385 (8 CPUs, shallow 300-commit copy of this repository, no crew, memory disk). Not production, and far less data than production, so read these as best case.
Capacity, code pages on random pinned commits:
| Asked | Served/s | p50 | p95 | A person's home, p95 |
|---|---|---|---|---|
| 10/s | 10 | 39 ms | 70 ms | 29 ms |
| 30/s | 28 | 24 ms | 49 ms | 26 ms |
| 100/s | 89 | 5.2 s | 10.2 s | 7.3 s |
| 300/s | 88 | 14.5 s | 27.4 s | 26.6 s |
It saturates near 90 pages a second, and a person waits behind the strangers once it does.
Open questions these raise:
- The copy handles 30/s at 49 ms p95, while production averaged 22 s at about 31/s. Something production has makes it far slower; find what before fixing pages.
- The ceiling in 0.7.384 didn't fire at 100 anonymous pages/s here: no 429s, 3.9 s p50, a person's p95 4.4 s. It keys on event-loop lateness, and here the wait isn't on the loop. It should also read in-flight deep pages.
Profile, 30 s at 30/s, share of samples: #getInternalBuffer 13.9%, writeSync 8.8%, spawn 7.8%, prepare 6.9%, toLocaleString 6.1%, callers (src/kernel/slow_sql.ts:31) 4.1%. Candidates: logging synchronously on the main thread, preparing statements afresh, Intl formatting per call, and the slow-SQL diagnostic itself. Not confirmed as causes; profile production's shape before acting.
Chaos:
- SIGKILL: back in 1.0 s, 20 of 20 acknowledged writes present,
quick_checkok. The 516 load errors are likely the load generator sharing the killed container's network, not an outage. - Freeze 10 s: healthy 0.1 s after resuming.
- 59 git children killed under load: no failed requests, p95 61 ms.
- Disk full: stayed up and a write still returned 201. That likely means SQLite had room already, so write failure on a full disk is still untested.
Parked by the crew: the check stayed red through 2 automatic tries (pull #9461 closed). Planner: split this smaller, or rewrite what done looks like; a person can also say what to try.
Reviewed pull #9499 at its head: ticked Commit-pinned pages are served without a git walk after the first read.. left open: A repeatable load test, run from a machine that isn't the owner's Mac against a copy of the forge, reports the rate and latency it reached, and its result is kept for each release. — The tool exists and is tested, but no run has been made and no result is kept per release (#9504).; A profile of a tree, blob, raw and media page names where each one's time goes under load, and is attached here before any fix is chosen. — No profile attached (#9503).; Rendering a code page can't hold the main thread: signed-in pages stay under 200 ms at p95 while strangers read at full rate. — Rendering is not moved off the main thread and nothing is measured.; At 500 anonymous code pages a second, nothing is refused and p95 is under 100 ms, with and without the edge cache. — Not measured at 500/s.; A release fails its check if the tested rate falls more than 20% from the last one. — The verdict exists in bench/load.ts --compare but is not wired into the release check (#9504)..
Left open after pull #9499 landed: 5 unticked items remain (A repeatable load test, run from a machine that isn't the owner's Mac against a copy of the forge, reports the rate and latency it reached, and its result is kept for each release.; A profile of a tree, blob, raw and media page names where each one's time goes under load, and is attached here before any fix is chosen.; Rendering a code page can't hold the main thread: signed-in pages stay under 200 ms at p95 while strangers read at full rate.; …). The planner files each as its own issue and closes this one; a person may close it, tick what is done, or say what is left. Until then the crew does not pick it again.
The open boxes are split. Box 1 (repeatable load test, result kept per release) and box 5 (a release fails on a drop of more than 20%) are #9504. Box 2 (profile of tree, blob, raw and media pages) is #9503. Box 3 (signed-in pages under 200 ms p95 while strangers read) is #9513, which waits for the profile. Box 4 (500/s, nothing refused, p95 under 100 ms, with and without the edge cache) is #9514, which waits for #9513 and #9504. Box 4 (commit-pinned pages served without a git walk after the first read) shipped in #9499.
Correction to the box numbers above. In the issue body as written: load test → #9504; profile → #9503; main thread under 200 ms → #9513; commit-pinned pages (ticked, shipped in #9499); 500/s under 100 ms → #9514; release fails on a drop of more than 20% → #9504.
The owner, 2026-11-03: "i really want a focus on speed, efficiency and scalability."
A flood of about 31 anonymous code pages a second took 91% of the forge's answering time. Those pages averaged about 22 s, home reached 7.9 s, and the foundry stopped answering, while the node sat at a load of about 1.7 with most of its memory free. The limit shipped in 0.7.384 sheds this traffic; it doesn't make the pages cheap.
Done when: