Public: anyone can read this; the crew can change it.

One process, one thread by default: the serve/work split switches on only when the machine has the cores #12531

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

When FORGEENGINE is not set, the machine now picks the engine shape at boot. A machine with at least 4 CPUs and a memory limit of at least 1024 MiB (or no known limit) gets the serve/work split. Anything smaller keeps one thread.

Notes

shipped production is serving it deploy d73d768

opened this

When FORGE_ENGINE is not set, the machine now picks the engine shape at boot. A machine with at least 4 CPUs and a memory limit of at least 1024 MiB (or no known limit) gets the serve/work split. Anything smaller keeps one thread. An explicit FORGE_ENGINE=worker or FORGE_ENGINE=inline still wins. A hosted child, a read-only standby and a test boot stay on one thread whatever is set, as before.

  • src/engine/mode.ts: new machineChoice({ cpus, memoryBytes }), a pure function. The thresholds are SPLIT_MIN_CPUS and SPLIT_MIN_MEMORY. Its reason states the numbers, for example "FORGE_ENGINE is not set and 1 CPU is under the 4 the split needs". That reason is the text /system and the terminal already show.
  • Reading the machine: readMachine() takes the CPU count from os.availableParallelism(). For memory it takes cgroup v2's memory.max, capped at os.totalmem(); when memory.max is unreadable or says max, it uses os.totalmem() alone. parseMemoryMax turns max, an empty file or junk into "no limit".
  • src/main.ts: passes machine: readMachine() into engineChoice. Callers that pass no machine keep the old answer, one thread, so existing tests and callers are unaffected.
  • Docs: docs/features/engine-shape.md and the "Two threads" section of docs/ARCHITECTURE.md describe the new default.

What to look at: the thresholds. 4 CPUs allows one core for serving, one for the engine and two for runs and checks. At 1 GiB the Worker's ~20 MiB is under 2% of memory.

Not done: the choice is made once at boot and is not re-evaluated if the machine's limits change while it runs.

Tests: new cases in test/engine-shape.test.ts check that 1 CPU gives one thread and 8 CPUs give the split. They also cover:

  • the exact thresholds;
  • a 512 MiB limit keeping one thread;
  • FORGE_ENGINE winning in both directions;
  • a hosted child or test boot overriding the machine;
  • parsing memory.max.

test/engine-shape.test.ts, test/engine-split.test.ts and test/worker-database-opens.test.ts pass. bun scripts/gate.ts and bun x tsc --noEmit are clean. Touching src/main.ts makes scripts/fast.ts name about 2,000 test files, so I ran these three, the only ones that reference engine/mode, and left the rest to the forge's full check.

Try it: boot with FORGE_ENGINE unset and open /system → "Engine shape". The reason line names the machine's CPU count and memory limit.

🤖 Generated with Claude Code

Closes #12520

2 earlier comments · 1 event
commented

When FORGE_ENGINE is not set, the machine now picks the engine shape at boot. A machine with at least 4 CPUs and a memory limit of at least 1024 MiB (or no known limit) gets the serve/work split. Anything smaller keeps one thread. An explicit FORGE_ENGINE=worker or FORGE_ENGINE=inline still wins. A hosted child, a read-only standby and a test boot stay on one thread whatever is set, as before.

  • src/engine/mode.ts: new machineChoice({ cpus, memoryBytes }), a pure function. The thresholds are SPLIT_MIN_CPUS and SPLIT_MIN_MEMORY. Its reason states the numbers, for example "FORGE_ENGINE is not set and 1 CPU is under the 4 the split needs". That reason is the text /system and the terminal already show.
  • Reading the machine: readMachine() takes the CPU count from os.availableParallelism(). For memory it takes cgroup v2's memory.max, capped at os.totalmem(); when memory.max is unreadable or says max, it uses os.totalmem() alone. parseMemoryMax turns max, an empty file or junk into "no limit".
  • src/main.ts: passes machine: readMachine() into engineChoice. Callers that pass no machine keep the old answer, one thread, so existing tests and callers are unaffected.
  • Docs: docs/features/engine-shape.md and the "Two threads" section of docs/ARCHITECTURE.md describe the new default.

What to look at: the thresholds. 4 CPUs allows one core for serving, one for the engine and two for runs and checks. At 1 GiB the Worker's ~20 MiB is under 2% of memory.

Not done: the choice is made once at boot and is not re-evaluated if the machine's limits change while it runs.

Tests: new cases in test/engine-shape.test.ts check that 1 CPU gives one thread and 8 CPUs give the split. They also cover:

  • the exact thresholds;
  • a 512 MiB limit keeping one thread;
  • FORGE_ENGINE winning in both directions;
  • a hosted child or test boot overriding the machine;
  • parsing memory.max.

test/engine-shape.test.ts, test/engine-split.test.ts and test/worker-database-opens.test.ts pass. bun scripts/gate.ts and bun x tsc --noEmit are clean. Touching src/main.ts makes scripts/fast.ts name about 2,000 test files, so I ran these three, the only ones that reference engine/mode, and left the rest to the forge's full check.

Try it: boot with FORGE_ENGINE unset and open /system → "Engine shape". The reason line names the machine's CPU count and memory limit.

🤖 Generated with Claude Code

commented

When FORGE_ENGINE is not set, the machine now picks the engine shape at boot. A machine with at least 4 CPUs and a memory limit of at least 1024 MiB (or no known limit) runs the engine's work on a separate Worker thread, apart from serving. Anything smaller keeps one thread, because the Worker costs about 20 MiB and only pays off with a core to spare. An explicit FORGE_ENGINE=worker or FORGE_ENGINE=inline still wins. A hosted child, a read-only standby and a test boot stay on one thread whatever is set, as before.

  • src/engine/mode.ts: new machineChoice({ cpus, memoryBytes }), a pure function. The thresholds are SPLIT_MIN_CPUS and SPLIT_MIN_MEMORY. Its reason states the numbers, for example "FORGE_ENGINE is not set and 1 CPU is under the 4 the split needs". /system and the terminal already show that text.
  • Reading the machine: readMachine() takes the CPU count from os.availableParallelism(). For memory it reads cgroup v2's memory.max, capped at os.totalmem(). When memory.max can't be read or says max, it uses os.totalmem() alone. parseMemoryMax turns max, an empty file or junk into "no limit".
  • src/main.ts: passes machine: readMachine() into engineChoice. Callers that pass no machine get the old answer, one thread, so existing tests and callers behave as before.
  • Docs: docs/features/engine-shape.md and the "Two threads" section of docs/ARCHITECTURE.md describe the new default.
  • Gate fix: the first push failed the gate because three new comment lines in mode.ts were over 100 columns. They are rewrapped. My earlier local gate run passed only because I ran it before committing, and the gate reads committed changes.

What to look at: the thresholds. With 4 CPUs, one core serves, one runs the engine and two are left for runs and checks. At 1 GiB the Worker's ~20 MiB is under 2% of memory.

Not done: the choice is made once at boot and is not re-checked if the machine's limits change while it runs.

Tests: new cases in test/engine-shape.test.ts check that 1 CPU gives one thread and 8 CPUs give the split. They also cover:

  • the exact thresholds;
  • a 512 MiB limit keeping one thread;
  • FORGE_ENGINE winning in both directions;
  • a hosted child or a test boot overriding the machine;
  • parsing memory.max.

bun scripts/gate.ts and bun x tsc --noEmit pass. test/engine-shape.test.ts passes after the rewrap. test/engine-split.test.ts and test/worker-database-opens.test.ts were the other files that reference engine/mode; they passed before the rewrap, which only changed comments. Touching src/main.ts makes scripts/fast.ts name about 2,000 test files, so the rest is left to the forge's full check.

Try it: boot with FORGE_ENGINE unset and open /system → "Engine shape". The reason line gives the machine's CPU count and memory limit.

🤖 Generated with Claude Code

Changes requested — before the latest push

The pure choice, the thresholds and the explicit-FORGE_ENGINE override are right and tested, and the preview answers 200 (/system needs an admin session, so I could not read the reason line there). One thing blocks it: the code that decides in production is not tested, so that part of box 1 has no test that would fail without it.

Fix first: readMachine() and the cgroup read in src/engine/mode.ts have no test. This is what flips every unset-FORGE_ENGINE boot from one thread to the split, so the 'pure function of CPUs and memory' only holds on paper. Split the file read from the logic, e.g. memoryLimit(cgroupText: string | null, total: number): number | null, and test it: cap below total, cap above total (takes total), max (takes total), unreadable (takes total), total 0 (null). Also cover cpus: Math.max(1, ...).

Also: (1) src/main.ts wiring is untested. Add a test that engineChoice with no machine still answers one thread, since that is the guarantee for existing callers, and that the reason is what /system shows for a 1-CPU machine. (2) cgroup v1 (/sys/fs/cgroup/memory/memory.limit_in_bytes) is ignored, so a v1 host with a 512 MiB cap reads total RAM and splits. Either handle it or say in docs/features/engine-shape.md that only v2 is read. (3) availableParallelism() does not reflect a cgroup cpu.max quota; a container limited to 1 CPU on a big host reads as many CPUs and splits. Read cpu.max (quota period or max) and take the min, with a test, or state the limitation in the doc.

commented

When FORGE_ENGINE is not set, the machine now picks the engine shape at boot. A machine with at least 4 usable CPUs and a memory limit of at least 1024 MiB (or no known limit) runs the engine on a separate Worker thread. Anything smaller keeps one thread, because the Worker costs about 20 MiB and only pays off with a core to spare. An explicit FORGE_ENGINE=worker or FORGE_ENGINE=inline still wins. A hosted child, a read-only standby and a test boot stay on one thread whatever is set, as before.

src/engine/mode.ts

  • machineChoice({ cpus, memoryBytes }) is a pure function. The thresholds are SPLIT_MIN_CPUS and SPLIT_MIN_MEMORY. Its reason states the numbers, for example "FORGE_ENGINE is not set and 1 CPU is under the 4 the split needs". That is the text /system and the terminal show.
  • Following review, reading the machine is split into small pure parts:

- memoryLimit(cgroupText, total) takes the smaller of the cgroup's cap and the machine's memory. It takes the machine's memory when the cap says max, can't be read, or is cgroup v1's "unlimited" value. It answers null when neither is known. - cpuQuota(text) turns a quota and period into whole CPUs, at least 1. It answers null for max, -1 or junk. - usableCpus(parallelism, quota) holds availableParallelism() under the quota and never goes below 1.

  • readMachine(read, parallelism, total) reads cgroup v2 (memory.max, cpu.max) first. When those files are missing it falls back to cgroup v1 (memory.limit_in_bytes, cpu.cfs_quota_us and cpu.cfs_period_us). All three inputs can be swapped in, so the production read is tested without touching /sys. A container limited to 1 CPU and 512 MiB on a 64-core host now counts as 1 CPU and keeps one thread.

src/main.ts passes machine: readMachine() into engineChoice. Callers that pass no machine still get one thread, and a test now pins that.

Docs: docs/features/engine-shape.md names the cgroup files read on v2 and v1. The "Two threads" section of docs/ARCHITECTURE.md describes the new default.

Tests: test/engine-shape.test.ts covers:

  • 1 CPU giving one thread and 8 CPUs giving the split, and the exact thresholds;
  • FORGE_ENGINE winning in both directions;
  • every memoryLimit case from review: cap below the total, cap above it, max, unreadable, and a total of 0;
  • the quota parsing and the minimum of 1 CPU;
  • readMachine against fake v2 and v1 file sets, and against no cgroup files;
  • a caller with no machine getting one thread;
  • a 1-CPU machine's reason appearing on the rendered /system page.

test/engine-shape.test.ts, test/engine-split.test.ts and test/worker-database-opens.test.ts pass, and bun scripts/gate.ts and bun x tsc --noEmit are clean. Touching src/main.ts makes scripts/fast.ts name about 2,000 test files, so I ran only the files that cover the engine choice and left the rest to the forge's full check.

Not done: the choice is made once at boot and is not re-checked if the limits change while the forge runs. A cgroup's quota and cap are read only at its root (/sys/fs/cgroup), not by walking up nested cgroups.

Try it: boot with FORGE_ENGINE unset and, as an admin, open /system → "Engine shape". The reason line gives the CPU count and memory limit that decided it.

🤖 Generated with Claude Code

Approved

Both earlier asks are met. (1) The cap/total arithmetic is now the pure memoryLimit, tested for cap<total, cap>total, max, unreadable, total 0 and v1 unlimited. (2) cpu.max and the v1 quota files are read through cpuQuota and usableCpus, with tests, and a 1-CPU container on a 64-core host reads as small. The reason shows on /system and in the terminal, and an explicit FORGE_ENGINE wins. The how-to is in docs/features/engine-shape.md, not the README. Next, both optional: Non-blocking: walk up nested cgroups, since only the root /sys/fs/cgroup is read. Non-blocking: record the CPU and memory numbers on /system as structured fields, not only in the reason text.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs