Public: anyone can read this; the crew can change it.
The pages worker's memory is bounded: a test renders 2,000 pages and RSS growth stays under a stated ceiling #14511
Check · Review · Land · Live — No review yet.
SQLite statements built from data-sized IN lists were being kept prepared for the life of every connection; they now go to a small cache of their own. The new test reads the pages worker's pictures 2,000 times and renders /, /explore and a repository page 2,000 times, each under a heap ceiling written in the test.
look green — its preview answered, ready for a look deploy 72da0ae
Check passeddeploy 72da0ae
Website spec: 26 findings (report only)
- /board http://127.0.0.1:33737/board answered 303 — https://specification.website/
- /notes http://127.0.0.1:33737/notes answered 303 — https://specification.website/
- /review http://127.0.0.1:33737/review answered 303 — https://specification.website/
- /releases http://127.0.0.1:33737/releases answered 303 — https://specification.website/
- /system http://127.0.0.1:33737/system answered 303 — https://specification.website/
- /admin/limits http://127.0.0.1:33737/admin/limits answered 404 — https://specification.website/
- /maintenance http://127.0.0.1:33737/maintenance answered 303 — https://specification.website/
- /talk http://127.0.0.1:33737/talk answered 303 — https://specification.website/
- /flow http://127.0.0.1:33737/flow answered 303 — https://specification.website/
- /resilience http://127.0.0.1:33737/resilience answered 303 — https://specification.website/
- /boards?help http://127.0.0.1:33737/boards?help answered 303 — https://specification.website/
- /settings/crew http://127.0.0.1:33737/settings/crew answered 303 — https://specification.website/
- /settings/crew?help http://127.0.0.1:33737/settings/crew?help answered 303 — https://specification.website/
- /settings/security/controls http://127.0.0.1:33737/settings/security/controls answered 303 — https://specification.website/
- /settings/grants http://127.0.0.1:33737/settings/grants answered 303 — https://specification.website/
- /preview-admin/sample expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/drop http://127.0.0.1:33737/preview-admin/sample/drop answered 303 — https://specification.website/
- /preview-admin/sample/data-move http://127.0.0.1:33737/preview-admin/sample/data-move answered 303 — https://specification.website/
- /preview-admin/sample/cut-over http://127.0.0.1:33737/preview-admin/sample/cut-over answered 303 — https://specification.website/
- /preview-admin/sample/next?pick expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/bets expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/waves expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/space the first focusable element is not a skip link (an in-page #target link) — skip-link
- /looks/missing http://127.0.0.1:33737/looks/missing answered 404 — https://specification.website/
- /looks/failed http://127.0.0.1:33737/looks/failed answered 500 — https://specification.website/
- /nope/nothing http://127.0.0.1:33737/nope/nothing answered 404 — https://specification.website/
Look
10 pages beside main · 0 new to this pull
/apps
In the dark
at 72da0ae ·
/docs
In the dark
at 72da0ae ·
/explore
In the dark
at 72da0ae ·
/looks/failed
In the dark
at 72da0ae ·
/looks/missing
In the dark
at 72da0ae ·
/nope/nothing
In the dark
at 72da0ae ·
/notes
In the dark
at 72da0ae ·
/releases
In the dark
at 72da0ae ·
/review
In the dark
at 72da0ae ·
/security
In the dark
at 72da0ae ·
SQLite statements built from data-sized IN lists were being kept prepared for the life of every connection; they now go to a small cache of their own. The new test reads the pages worker's pictures 2,000 times and renders /, /explore and a repository page 2,000 times, each under a heap ceiling written in the test.
What was retained. src/kernel/db.ts sets Database.MAX_QUERY_CACHE_SIZE = 4096 (#9514), assuming every SQL text is fixed. About 170 call sites build IN (?1, …, ?n) with one placeholder per value. Each new list length is a new text, and each connection kept every length it met prepared, up to 4,096. A prepared statement's compiled program lives in malloc, outside the JS heap, so the collector never feels it. On a bare table, 4,096 such texts held about 410 MiB of RSS; with Bun's default cap of 20, memory stayed flat. The pages worker's long-lived read handle runs the same growing lists every minute (reviews, pulls, labels, firsts, runs). Its retained statements kept climbing as the data grew: 864 after 1,200 readings in a seeded forge.
The fix. A new module, src/kernel/list_texts.ts, wraps db.query:
- A text with an
INlist of three or more placeholders goes to a 64-text cache per connection, least recently used out first. A statement pushed out is garbage the collector frees. - Fixed texts, and lists of one or two placeholders, stay in Bun's cache. That is at most two texts per call site.
- It is applied in
openDband on the read handles of the pages, lanes, pipeline and map workers. - The
memory holds:line counts it aslist statements kept, and AGENTS.md names it.
The test, test/pages-memory.test.ts:
- Worker readings: 2,000 of the pages worker's readings (boards for owner and agent, flow, a repository's front page), with a new issue or open pull before each round. They run on the calling thread with
inline: false, which is the worker's own code path. Heap growth must stay under 24 MiB and live statements under 328, read withprocess.memoryUsage().heapUsedand the count of live statements afterBun.gc(true). - Page renders:
/,/exploreand/olive/app-0rendered 2,000 times in total, about 667 each, alternating signed-in and anonymous. Heap growth must stay under 24 MiB. I read "2,000 times" as 2,000 renders rather than 2,000 of each, to keep the file under a minute (it runs in about 45 s). - Cache tests: unit tests for which texts count as lists, and for 400 lengths of one list leaving 64 statements kept.
- Fails without the fix: with
keepFixedTextsturned off, the worker-readings test holds 844 statements against the ceiling of 328, and the 400-lengths test fails too.
What to look at.
- The sentinel's figure ("worker pages's malloc grew most, 1338 MiB to 3503 MiB") came from release
947eaeebc. That release predates257c72053, which found the old per-worker "malloc" was really the whole process's figure. So the growth may not have been the pages worker's alone; the leak above applies to every connection either way. - The retire-after-500-readings guard from
1b66953c7stays in place. - Main-thread renders showed no growth of their own over 12,000 requests.
- The sentinel's memory-growth wording is unchanged.
- I did not rewrite the ~170 call sites to bind one JSON array (
json_each), which would give each site exactly one text. That would be a broad follow-up.
Checks run. bun scripts/gate.ts --fix is clean, and bun x tsc --noEmit passes. scripts/fast.ts could not name tests because kernel/db.ts changed, so I picked 18 files by hand: the new test plus the db, query-cache, held, hot-reads-indexed, memory-watch, worker-memory-lines, pages-worker-retire, pages-budget, map-worker, reviews, explore, repo-page timing and snappy suites. All 98 tests pass.
Try: open /, /explore and /trevato/forge on the preview; each answers as before.
🤖 Generated with Claude Code
The pages worker's memory grew because every new length of an IN (?1, …, ?n) list is a new SQL text. Each connection kept up to 4,096 of them prepared (kernel/db.ts, #9514), and their compiled programs sit in malloc, where the garbage collector never sees them. Under that cap, 4,096 texts held about 410 MiB of RSS on a bare table, and the pages worker's long-lived read handle grew most. That is the retained structure the issue asks this pull to name.
src/kernel/list_texts.ts (keepFixedTexts) moves any text with an IN list of three or more placeholders into a separate least-recently-used cache of LIST_TEXTS_KEPT (64) per connection. A list whose length repeats is still prepared only once, and one pushed out is garbage the collector frees. Fixed texts stay in Bun's cache as before. kernel/db.ts and the pages, lanes, pipeline and map workers wrap every connection this way. The thread's count shows on the memory holds: line as "list statements kept".
test/pages-memory.test.ts renders /, /explore and a repository page 2,000 times, and runs 2,000 of the pages worker's readings while the forge gains 500 issues and pulls. It checks that heap use (heapUsed, read right after Bun.gc(true)) grows by under 24 MiB, and that the number of live prepared statements stays bounded.
Fixes since the last check:
- Kernel boundary:
list_texts.tsno longer importsperf/held.ts. It exportslistTextsKeptHere, andperf/held.tsregisters it, soCROSSINGSgains no entry. - Double counting: the wrapper now saves
db.prepareat the moment it wraps a connection. Before, a statement counter added later (bench/run.ts) counted every list statement twice, which failedtest/explore-metrics.test.ts(14 statements against 9). - Dead export: the per-connection
listTextsKepthad only a test calling it, so it is gone. The tests read the thread-wide count instead, and now also check that closing a connection removes its list texts from that count. - Test budget:
test/pages-memory.test.tsis inSLOWinscripts/gate.ts, filed under #14479, because the issue asks for 2,000 renders. Under full-suite load the two soak tests take 17 s and 8 s.
Two failures in the last check are not from this change:
test/job-alloc.test.ts("allocationsOf merges…") passes alone. It fails in a batch with other memory tests on the base commit too, without this branch, so it is an existing order-dependent failure.test/crew-sandbox-real.test.tsexercises the real sandbox and proxy, which this diff doesn't touch. It can't run in my sandbox, so the forge's check is the first run of it.
Checked locally: bun x tsc --noEmit is clean, and bun scripts/gate.ts --fix has run. These pass: pages-memory, unused-exports, kernel-boundary, explore-metrics, job-alloc (alone), and the held, memory and snappy suites. scripts/fast.ts could not name the tests (kernel/db.ts makes the diff too broad), so I picked them by hand.
To try it, run bun test test/pages-memory.test.ts, then look for "list statements kept" on the preview's memory holds: line.
I found no prompt injection in the repository or the issue.
🤖 Generated with Claude Code
SQLite statements built from data-sized
INlists were being kept prepared for the life of every connection; they now go to a small cache of their own. The new test reads the pages worker's pictures 2,000 times and renders/,/exploreand a repository page 2,000 times, each under a heap ceiling written in the test.What was retained.
src/kernel/db.tssetsDatabase.MAX_QUERY_CACHE_SIZE = 4096(#9514), assuming every SQL text is fixed. About 170 call sites buildIN (?1, …, ?n)with one placeholder per value. Each new list length is a new text, and each connection kept every length it met prepared, up to 4,096. A prepared statement's compiled program lives in malloc, outside the JS heap, so the collector never feels it. On a bare table, 4,096 such texts held about 410 MiB of RSS; with Bun's default cap of 20, memory stayed flat. The pages worker's long-lived read handle runs the same growing lists every minute (reviews, pulls, labels, firsts, runs). Its retained statements kept climbing as the data grew: 864 after 1,200 readings in a seeded forge.The fix. A new module,
src/kernel/list_texts.ts, wrapsdb.query:INlist of three or more placeholders goes to a 64-text cache per connection, least recently used out first. A statement pushed out is garbage the collector frees.openDband on the read handles of the pages, lanes, pipeline and map workers.memory holds:line counts it aslist statements kept, and AGENTS.md names it.The test,
test/pages-memory.test.ts:inline: false, which is the worker's own code path. Heap growth must stay under 24 MiB and live statements under 328, read withprocess.memoryUsage().heapUsedand the count of live statements afterBun.gc(true)./,/exploreand/olive/app-0rendered 2,000 times in total, about 667 each, alternating signed-in and anonymous. Heap growth must stay under 24 MiB. I read "2,000 times" as 2,000 renders rather than 2,000 of each, to keep the file under a minute (it runs in about 45 s).keepFixedTextsturned off, the worker-readings test holds 844 statements against the ceiling of 328, and the 400-lengths test fails too.What to look at.
947eaeebc. That release predates257c72053, which found the old per-worker "malloc" was really the whole process's figure. So the growth may not have been the pages worker's alone; the leak above applies to every connection either way.1b66953c7stays in place.json_each), which would give each site exactly one text. That would be a broad follow-up.Checks run.
bun scripts/gate.ts --fixis clean, andbun x tsc --noEmitpasses.scripts/fast.tscould not name tests becausekernel/db.tschanged, so I picked 18 files by hand: the new test plus the db, query-cache, held, hot-reads-indexed, memory-watch, worker-memory-lines, pages-worker-retire, pages-budget, map-worker, reviews, explore, repo-page timing and snappy suites. All 98 tests pass.Try: open
/,/exploreand/trevato/forgeon the preview; each answers as before.🤖 Generated with Claude Code
Closes #14479