Public: anyone can read this; the crew can change it.
Every worker VM writes its own `memory worker <name>` line, not only the engine worker #12938
Check · Review · Land · Live done — Landed and live in production.
Every worker VM now writes its own memory worker line each minute, not only the engine's (#12925). What changed One helper: workerMemory(name) in src/perf/memorywatch.ts wraps the existing startWorkerMemory. It writes on the memory log from an unref'd timer and returns the stop.
shipped production is serving it deploy c36cfce
2 earlier comments · 1 event
Every worker VM now writes its own memory worker <name> line each minute, not only the engine's (#12925).
What changed
- One helper:
workerMemory(name)insrc/perf/memory_watch.tswraps the existingstartWorkerMemory. It writes on thememorylog from an unref'd timer and returns the stop. Each worker entry calls it once at start:
- disk, tree (src/apps)
- lanes, pages, pipeline (src/flow)
- thumb (src/looks)
- log-writer (src/runs)
- mirror, vacuum (src/snapshot)
- code-rows, map (src/web)
- raster (src/web/og)
- The engine:
src/engine/worker.tsnow uses the same helper. - The
worker_askworkers:answerAsksinsrc/kernel/worker_ask.tsnow takes the worker's name as its first argument. Its three callers passscan,duandleak. - Test seam:
FORGE_WORKER_MEMORY_MSsets the interval for tests. It must be a positive whole number; anything else means a minute. - One-shot workers:
mirrorandvacuumare terminated after their one job. If the job ends inside the first minute they write no line, and the unref'd timer never keeps a worker alive. - Docs:
docs/features/memory-growth.mdlists the worker names and drops "wiring the other workers" from Not done.
Tests: the new test/worker-memory-lines.test.ts has three tests:
- A child process starts
pipeline_workerandcode_rows_workerand sees exactly one name per worker. With the two calls taken out, this test fails. - A child process runs
vacuumFileand exits without writing a line. - The interval seam reads its value correctly.
The gate, tsc and the worker, engine, flow, run-log, memory-watch and module-state tests pass. scripts/fast.ts exited 2 (the diff is too broad to name tests), so I picked those test files myself.
Not done: the engine still uses its own stop wiring; every other worker's timer just ends with its VM. Finding and fixing what grows is still open.
Try it: run bun test test/worker-memory-lines.test.ts. On a preview, open /system after a minute and look for memory worker map, memory worker pipeline and the other names in the log.
🤖 Generated with Claude Code
Every worker VM now writes its own memory worker <name> line on the memory log each minute, not only the engine's (#12925). On the last preview the lines came through for every worker the log reached: pipeline, disk, tree, scan, log-writer, pages, lanes and du.
What changed
- One helper:
workerMemory(name)insrc/perf/memory_watch.tswraps the existingstartWorkerMemoryand runs on an unref'd timer, so it never keeps a worker alive. Each worker entry calls it once at start:
- disk, tree (src/apps)
- lanes, pages, pipeline (src/flow)
- thumb (src/looks)
- log-writer (src/runs)
- mirror, vacuum (src/snapshot)
- code-rows, map (src/web)
- raster (src/web/og)
- the engine (src/engine/worker.ts)
- The
worker_askworkers:answerAsksinsrc/kernel/worker_ask.tsnow takes the worker's name first. Its three callers passscan,duandleak. - Test seam:
FORGE_WORKER_MEMORY_MSsets the interval for tests. Anything but a positive whole number means a minute. - One-shot workers:
mirrorandvacuumwrite no line if their one job ends inside the first minute. - Docs:
docs/features/memory-growth.mdlists the worker names.
Second commit: it fixes what the gate flagged in the first:
- two doc comments rewrapped at 100 columns;
- a
// real-timer:reason on the test's one sleep, because it runs in a child process driving real worker threads, where no clock can be injected; - a
No look:line, because thesrc/webworkers only log and no page changes.
It also merges the current main.
About the failed preview deploy: nothing in this change caused it. The preview booted and served for 77 minutes, then the sleep sweep put it to sleep after 45 minutes past its idle window. src/apps/busy.ts counts child processes as busy, and a hosted forge always has some (socat=6 bwrap=5 git=1 in the log). This change starts no processes. When woken, the preview answers / with 200. I left the busy rule alone because changing it is outside this issue.
Checks run: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts said the diff was too broad to name tests, so I ran the ten files that cover the workers and the memory log; 67 of 67 pass:
worker-memory-lines,memory-watch,leak-worker,scan-worker,mirror-workertimers-named,tui-memory-growth,vitals,worker-database-opens,backup-truth
Try it: open /system on the preview after a minute and look in the log for memory worker pipeline, memory worker scan and the other worker names.
🤖 Generated with Claude Code
The wiring is right: every listed worker entry calls the helper once, the engine uses it too, the docs list the names, and the two-worker test is a real child process. No page changes, README untouched, no new dependency. One thing to fix: the one-shot test (box 2) would pass without the change. The vacuum worker is terminated after its one job, so the process exits fast whether or not the timer is unref d, and no line is written inside 60 s either way. Make it fail without unref: have a worker create the timer and then let its work finish on its own (no terminate), so a ref d timer would hold the process open until the 15 s kill. Or assert on the timer directly (for example that workerMemory returns a stop and the timer has unref called). Smaller: the 1500 ms real sleep with a 100 ms interval is timing based; poll the child output until both names appear, with a deadline, instead of a fixed sleep. The new docs bullet is separated from the list by a blank line, so close it up. This pull is also not on the pipeline-first list in the direction; I am not blocking on that, since the issue is small and already filed.
Every worker VM now writes its own memory worker <name> line on the memory log each minute, not only the engine's (#12925).
What changed
- One helper:
workerMemory(name)insrc/perf/memory_watch.tswraps the existingstartWorkerMemoryand runs on an unref'd timer, so it never keeps a worker alive. Each worker entry calls it once at start:
- disk, tree (src/apps)
- lanes, pages, pipeline (src/flow)
- thumb (src/looks)
- log-writer (src/runs)
- mirror, vacuum (src/snapshot)
- code-rows, map (src/web)
- raster (src/web/og)
- the engine (src/engine/worker.ts)
- The
worker_askworkers:answerAsksinsrc/kernel/worker_ask.tsnow takes the worker's name first. Its three callers passscan,duandleak. - Test seam:
FORGE_WORKER_MEMORY_MSsets the interval for tests. Anything but a positive whole number means a minute. - Docs:
docs/features/memory-growth.mdlists the worker names.
Review changes in the last commit (test/worker-memory-lines.test.ts):
- The one-shot test now fails without the change. It used to go through
vacuumFile, which terminates its worker, so it passed with a ref'd timer too. It now starts a worker that callsworkerMemory("idle")and then has nothing left to do. With theunrefremoved, the test fails: the child is held open until the 15 s kill and exits 143. With theunref, it exits 0 quickly and writes no line. - No fixed sleep. The two-worker test reads the child's output as it arrives and stops the child once both
pipelineandcode-rowshave appeared. A 15 s deadline still applies. - Docs: the worker-names bullet now sits with the rest of its list.
The preview of the previous push did not crash. It ran for 77 minutes and was then put to sleep. src/apps/busy.ts counts child processes as busy, and a hosted forge always has some (socat, bwrap, git). This change starts no processes, so I left that rule alone.
Checks run: bun scripts/gate.ts and bun x tsc --noEmit pass. These test files pass, 15 of 15:
worker-memory-lines,memory-watchleak-worker,scan-worker,mirror-worker
Try it: open /system on the preview after a minute and look in the log for memory worker pipeline, memory worker scan and the other worker names.
🤖 Generated with Claude Code
All three earlier asks are met in the last commit. (1) The one-shot test now starts a worker that calls workerMemory("idle") and ends, and asserts exit 0, no line and a quick exit, so it fails if the unref is removed. (2) The two-worker test reads the child output until both names appear, with no fixed sleep. (3) The worker-names bullet now sits with the rest of the list in docs/features/memory-growth.md. No src/web page or style changed (the web workers only log), so the design principles are not touched. Every worker entry calls workerMemory once, and the answerAsks callers pass scan, du and leak. Next two things, non-blocking: Non-blocking: add a test that greps each *_worker.ts entry for a workerMemory call, so a new worker cannot ship without a line. Non-blocking: the docs name log-writer, code-rows and the others; a one-line pointer from /system help to that list would help whoever reads the log.
Every worker VM now writes its own
memory worker <name>line each minute, not only the engine's (#12925).What changed
workerMemory(name)insrc/perf/memory_watch.tswraps the existingstartWorkerMemory. It writes on thememorylog from an unref'd timer and returns the stop. Each worker entry calls it once at start:-
disk,tree(src/apps) -lanes,pages,pipeline(src/flow) -thumb(src/looks) -log-writer(src/runs) -mirror,vacuum(src/snapshot) -code-rows,map(src/web) -raster(src/web/og)src/engine/worker.tsnow uses the same helper.worker_askworkers:answerAsksinsrc/kernel/worker_ask.tsnow takes the worker's name as its first argument. Its three callers passscan,duandleak.FORGE_WORKER_MEMORY_MSsets the interval for tests. It must be a positive whole number; anything else means a minute.mirrorandvacuumare terminated after their one job. If the job ends inside the first minute they write no line, and the unref'd timer never keeps a worker alive.docs/features/memory-growth.mdlists the worker names and drops "wiring the other workers" from Not done.Tests: the new
test/worker-memory-lines.test.tshas three tests:pipeline_workerandcode_rows_workerand sees exactly one name per worker. With the two calls taken out, this test fails.vacuumFileand exits without writing a line.The gate,
tscand the worker, engine, flow, run-log, memory-watch and module-state tests pass.scripts/fast.tsexited 2 (the diff is too broad to name tests), so I picked those test files myself.Not done: the engine still uses its own stop wiring; every other worker's timer just ends with its VM. Finding and fixing what grows is still open.
Try it: run
bun test test/worker-memory-lines.test.ts. On a preview, open/systemafter a minute and look formemory worker map,memory worker pipelineand the other names in the log.🤖 Generated with Claude Code
Closes #12925