Memory by owner, the heap across a stall, memory growth (#12901)

Since Bun 1.4.4 the serving process has held several GiB after hours. This change makes the growth readable; it does not stop it (see "Not done").

Where it shows

- fell: a collection ran means the heap shrank by more than a tenth since the sample, so the collector ran inside that stall. Bun 1.4.4 reports no collector time, so this is inferred, not measured. - did not fall: no collection seen means the stall was not the collector's.

What to do when the need shows

  1. The need names the VM and the part. To see the curve, read the memory main and memory worker lines around the time named: heap up means JS objects, heap flat with RSS up means native memory, a worker line up means that worker.

Then read the memory holds: lines across the same minutes: a holder that grew with the heap is named there with its count.

  1. Check the stall lines for fell: a collection ran: stalls that are collections point at the heap.
  2. Restart the forge to release it, and report the lines with the issue.

The need clears by itself once RSS falls under twice the baseline. A process that slept through its first hour has no baseline and raises none.

How it is built

src/perf/memory_watch.ts owns the readings (heapSize() and memoryUsage(), constant time; never heapStats(), which held the loop ~280 ms a minute, #11118). vitals() takes the watch as watch, as it takes disk. Each worker entry calls workerMemory(<name>) once at start; FORGE_WORKER_MEMORY_MS shortens the minute for tests. The same minute timer that writes a worker's line posts its reading ({ name, heap, current }) on the forge-memory BroadcastChannel (tellReading); the serving VM's memory sampler listens (hearWorkers), validates each message (workerReadingOf: a lower-case name, not main, finite byte counts) and notes it. Asks carry nothing extra, and at most 64 workers are kept. Tests: test/memory-watch.test.ts, test/worker-memory-lines.test.ts, test/vitals.test.ts, test/tui-memory-growth.test.ts.

What grew, and what holds it now

Found on a copy of production's data with no network, production's own work on (apps, crew, checks, the picture keeper) and a replay of pages, API and MCP calls, event streams and pushes; object counts by type from bun:jsc heapStats() after a full collection, and heap snapshots, at 5, 20 and 60 minutes.

A new long-lived collection names itself in heldOf(app) (a forge's) or threadHeld (the thread's), with a bound and a test of its eviction. Named today: turn, requests in flight, event streams, request counters, route timings, map plans, map lanes, map layouts, and four the crew keeps per pull or per check that nothing sweeps yet (autonomy conflicted pulls, finishing said, checks retried, app links said): each is a short string per entry, and its line says when one is worth a sweep.

Tests: test/app-turn.test.ts, test/cache-holds.test.ts, test/forge-sleep.test.ts, test/held.test.ts.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs