Public: anyone can read this; the crew can change it.

The limits that bound a surge of new forges are constants and env values, not settings, and one of them is silent #13835

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

Births now go through one admission rule, and most of the surge limits are settings an admin can change without a restart. I only ran the spawn, limits, sleep and first-ten-minutes suites (they pass), plus tsc and the gate. The forge runs the full check on this push.

Notes

shipped production is serving it deploy 3dfbd46

opened this

Births now go through one admission rule, and most of the surge limits are settings an admin can change without a restart. I only ran the spawn, limits, sleep and first-ten-minutes suites (they pass), plus tsc and the gate. The forge runs the full check on this push.

What changed

  • Births: src/spawn/births.ts is new. A spawn needs three things:
    • fewer than spawn.births_at_once forges being made (default 8);
    • spawn.birth_memory_mb (1,536) free above forge.memory_floor_mb;
    • the disk above its existing floor.

    A spawn that doesn't fit waits in a first-in-first-out line for up to 15 s. Past that it gets 503 {error:"queued", place, aboutS, retryAfter, why} instead of a bare refusal. A birth keeps its place until the forge answers, or for 5 minutes at most. aboutS comes from a running mean of measured births, starting at 4 minutes.

  • Door: src/spawn/door.ts takes a place before it makes the forge. It gives the place back at once if no forge was made.
  • Settings (all on /admin/limits):
    • spawn.anon_per_day (100) and spawn.anon_global_per_day (2,000), which replace the constants in rate.ts.
    • spawn.timers_max (20), which replaces MAX_WATCHERS.
    • forges.awake_max (8) and forges.wakes_per_forge_hour (20). FORGE_CHILD_AWAKE_MAX and FORGE_CHILD_WAKES_PER_HOUR still apply while the setting is unset.
    • spawn.births_at_once and spawn.birth_memory_mb, the two new ones.
  • Untimed spawns: a spawn past the timer ceiling now logs "made but not timed". startWatch also takes an onSettled callback, which is how a place is given back when the forge answers.
  • Tests and docs: test/spawn-births.test.ts is new. The anonymous-spawn, live and release-prober tests now read the settings, and spawn-anon has a door-level 503 test. AGENTS.md and docs/features/forge-sleep.md describe the births state.

What I did not do

  • Idle time, NEW_FORGE_MS and held wake requests (wakeLimits) are still constants. I left them alone because the idle timing is tangled with the cell sweep.
  • No queue ticket. A spawn that waits 15 s and gets the 503 loses its place and rejoins at the back when it retries. The place is real while the request is held, and the 503 reports it.
  • Under test only, the births count applies only where a test sets it, and the memory reading is off unless a test provides one. The suite's probes never answer, so every spawn would otherwise hold its place for the full 5 minutes.
  • No "being made" page. The browser page for a queued spawn is unchanged; the 503 body and message are what carry the place.
  • No Retry-After header on the 503: the door has no way to set headers, so retryAfter is only in the JSON body.

To try it, set spawn.births_at_once to 1 on /admin/limits, then post two spawns at /api/v1/forges. The second waits up to 15 s, then answers 503 with place: 1 and aboutS.

🤖 Generated with Claude Code

Closes #13822

24 earlier comments · 3 events
commented

Births now go through one admission rule, and most of the surge limits are settings an admin can change without a restart. I only ran the spawn, limits, sleep and first-ten-minutes suites (they pass), plus tsc and the gate. The forge runs the full check on this push.

What changed

  • Births: src/spawn/births.ts is new. A spawn needs three things:
    • fewer than spawn.births_at_once forges being made (default 8);
    • spawn.birth_memory_mb (1,536) free above forge.memory_floor_mb;
    • the disk above its existing floor.

    A spawn that doesn't fit waits in a first-in-first-out line for up to 15 s. Past that it gets 503 {error:"queued", place, aboutS, retryAfter, why} instead of a bare refusal. A birth keeps its place until the forge answers, or for 5 minutes at most. aboutS comes from a running mean of measured births, starting at 4 minutes.

  • Door: src/spawn/door.ts takes a place before it makes the forge. It gives the place back at once if no forge was made.
  • Settings (all on /admin/limits):
    • spawn.anon_per_day (100) and spawn.anon_global_per_day (2,000), which replace the constants in rate.ts.
    • spawn.timers_max (20), which replaces MAX_WATCHERS.
    • forges.awake_max (8) and forges.wakes_per_forge_hour (20). FORGE_CHILD_AWAKE_MAX and FORGE_CHILD_WAKES_PER_HOUR still apply while the setting is unset.
    • spawn.births_at_once and spawn.birth_memory_mb, the two new ones.
  • Untimed spawns: a spawn past the timer ceiling now logs "made but not timed". startWatch also takes an onSettled callback, which is how a place is given back when the forge answers.
  • Tests and docs: test/spawn-births.test.ts is new. The anonymous-spawn, live and release-prober tests now read the settings, and spawn-anon has a door-level 503 test. AGENTS.md and docs/features/forge-sleep.md describe the births state.

What I did not do

  • Idle time, NEW_FORGE_MS and held wake requests (wakeLimits) are still constants. I left them alone because the idle timing is tangled with the cell sweep.
  • No queue ticket. A spawn that waits 15 s and gets the 503 loses its place and rejoins at the back when it retries. The place is real while the request is held, and the 503 reports it.
  • Under test only, the births count applies only where a test sets it, and the memory reading is off unless a test provides one. The suite's probes never answer, so every spawn would otherwise hold its place for the full 5 minutes.
  • No "being made" page. The browser page for a queued spawn is unchanged; the 503 body and message are what carry the place.
  • No Retry-After header on the 503: the door has no way to set headers, so retryAfter is only in the JSON body.

To try it, set spawn.births_at_once to 1 on /admin/limits, then post two spawns at /api/v1/forges. The second waits up to 15 s, then answers 503 with place: 1 and aboutS.

🤖 Generated with Claude Code

commented

I merged main into the branch and pushed. The gate is clean now, tsc is clean, and the spawn, live, anon, sleep, first-ten-minutes and limits tests pass.

  • Conflict: the only one was in src/spawn/host.ts, in wakeDue. main had rewrapped a comment, and my side had changed the awakeRefusal(...) call on the line below it. I kept main's comment and my call. The call passes parent.db, so forges.awake_max is honoured there. No other call site was missed.
  • New gate rule: main now refuses tests that sleep on a real timer. Three sleeps in test/spawn-births.test.ts tripped it, so I marked them // real-timer: with the reason, as the rule asks. This is in a second commit after the merge.

I ran only the spawn, limits and sleep suites listed above, not the full suite; the forge runs the full check on this push.

commented

This push fixes the four failures the check reported. I re-ran the affected tests locally and they pass, along with tsc and the gate. I didn't run the full suite; the forge's check does that on this push.

  • Dead code: birthCounts and provideMemoryForTest in src/spawn/births.ts are only called from tests, so they're now in scripts/unused-exports-allow.json.
  • MCP table: docs/MCP.md was stale for the new limits tool. I regenerated it with bun scripts/mcp-table.ts.
  • Timers: the three timers in births.ts were not wrapped in during. The look interval, the 5-minute cap and the hold deadline now each run inside it.
  • Slow test: "a surge past the births at once is told its place" waited out the real 15 s hold. The hold is now a module-level value with a provideHoldForTest seam (also allow-listed), and the test sets it to 50 ms and resets it afterwards. HOLD_MS is no longer exported.

Behaviour is unchanged in production: the hold is still 15 s.

commented

I merged main into the branch and pushed. The only conflict was in src/spawn/door.ts, and tsc and the gate were clean after the merge. I re-ran the spawn-anon, spawn-births, spawn-live, timers-named, unused-exports and mcp-table tests (43 pass, 0 fail); the forge's check runs the full suite on this push.

The conflict was where a new forge's apps boot:

  • From main: the boot call is kept in up, so the reach proof waits for the forge to come up.
  • From this branch: the watch is given onSettled: place.release, which gives the birth place back when the forge settles.

I kept both. The line now reads up = bootHostedChild(...), and startWatch(...) still passes onSettled: place.release.

commented

Two tests failed on the check. I fixed one and left the other alone. I pushed the fix, but I haven't seen the forge's new check verdict.

  • Job-slices test fixed: births-look, births-cap and births-hold are the during jobs that src/spawn/births.ts runs. test/job-slices.test.ts neither timed them nor listed them in UNTIMED. I added them to UNTIMED with reasons: each runs only while a hosted forge is being born, and the fixture has none. That test passes now, and bun x tsc --noEmit and bun scripts/gate.ts --fix came back clean.
  • Fast-forward test not fixed: "a fast-forward applies the previewed id: a parent that moved after it is not taken" is in test/take-expect.test.ts, which passes 5 of 5 when I run it alone. It failed under shard load in the check, and I made no change for it. If it fails again on the next check, it needs its own look.
commented

I merged main into this branch, resolved the three conflicts, and pushed. The spawn, sleep, wake, limits, births and job-slices tests pass (798 pass, 12 skip, 0 fail), and tsc and scripts/gate.ts --fix came back clean. I did not run the whole suite. scripts/fast.ts said the diff was too broad to name tests, so I ran the spawn-related ones. The forge runs the full check on the push.

  • AGENTS.md: both additions stay. The #13822 births paragraph is followed by the src/crew/beat.ts paragraph from main.
  • src/spawn/sleep.ts:
    • The limits imports are merged: LIMITS comes from main, and limitIsSet, limitValue and LimitKey from this branch.
    • awakeCeiling and wakesPerHour keep this branch's settings-aware versions, so /admin/limits still overrides the environment.
    • main's per-owner wake budget (wakeBudget, budgetRefusal, waitWords) comes through unchanged.
  • src/spawn/host.ts: wakeGate uses main's per-owner budget key and refusal text. The forges.wakes_per_person_hour limit is read from parent.db. awakeRefusal and rates.refusal still get process.env and parent.db, so this branch's limits apply there.

The forge.test and localhost network denials in the test output are the sandbox blocking outbound connections. The suite still passed.

Changes requested — before the latest push

Changes needed (first round, all in one):

  1. HIGH, denial of service: takePlace in src/spawn/births.ts pushes every waiting spawn onto an unbounded queue, each holding a request open for 15 s with a promise and a timer. The anonymous door is open to strangers, and the anon daily count is only noted after a forge is made, so a loop of spawns while births are full piles up held requests with no bound, and strangers also starve signed-in people in the same FIFO. Fix: cap the queue length (a setting or a constant, e.g. 4x births_at_once) and answer past it at once with the 503 queued body; let signed-in callers go ahead of anonymous ones or keep a separate small anon queue. Add a test that fails without the cap.
  2. MEDIUM, leak to strangers: the 503 message returned to anonymous callers includes the machine free memory in MiB, the floor, and the raw floorRefusal disk text (src/spawn/births.ts whyNot, src/spawn/door.ts). Return only place, aboutS, retryAfter and the short why code to anonymous callers; keep the figures in the log or admin views.
  3. A place can leak: in src/spawn/door.ts, startWatch returns false when timers are full, so onSettled never runs and the birth stays counted for 5 minutes, and an exception between spawnForge and startWatch (e.g. in bootHostedChild) does the same. Release the place when startWatch returns false, and in a catch/finally on that path. Test it.
  4. Production code branches on NODE_ENV==="test" (whyNot skips the birth count unless a test set it; memoryReader is off under test). That means the default path is never exercised by the suite. Replace with an injected seam (like provideMemoryForTest) rather than an environment check, and make the suite probes release their place.
  5. Direction: the owner put pipeline work first (#6534, #6611, #5276, #3316, #6217, ...); this is not on that list. Say in the pull which pipeline issue it unblocks, or confirm the issue was explicitly prioritised.
  6. The limits are on /admin/limits but I see no terminal path named; confirm the terminal limits screen lists the new keys from LIMITS (add a test if it is hand-listed), and that the 503 queued state is reachable/visible there too.

I could not exercise the preview beyond reading the diff; the pull is otherwise tidy (docs/features, AGENTS.md, no README change, no new dependency).

commented

I fixed items 1–4 and 6, with one gap in item 6. Item 5 is answered below. The spawn, sleep, wake, limits, births and child tests pass (882 pass, 14 skip, 0 fail), and tsc and the gate are clean.

  • 1. Unbounded queue: takePlace in src/spawn/births.ts now takes { holdMs, anon }. At most 4 × spawn.births_at_once spawns wait, and strangers hold at most births_at_once of those places. A spawn past either cap is answered at once with its place and the 503 queued body. Signed-in callers are placed ahead of waiting strangers. The two new tests in test/spawn-births.test.ts fail without the caps.
  • 2. Leak to strangers: the 503 message to an anonymous caller is now only place, about-seconds and retry. The MiB, floor and disk text appear only for signed-in callers. test/spawn-anon.test.ts asserts a stranger's message has none of them.
  • 3. Leaked place: in src/spawn/door.ts, startWatch and bootHostedChild now sit in a try/finally that releases the place unless the watch started. The new test in test/spawn-anon.test.ts fills the one timer, spawns, and checks inFlight is 0.
  • 4. NODE_ENV branches: both are gone from births.ts; memory defaults to freemem. test/helpers/preload.ts now sets roomy memory and a new useWatchProbeByDefault seam in door.ts, so suite children answer at once and release their place. Without that the real births count stalled the suite on its own never-answering children. The default path now runs under test.
  • 5. Direction: I can't confirm this was prioritised. The issue (#13822) is about the limits that bound a surge of new forges, and it names no pipeline issue. A person should say whether it stands before it lands.
  • 6. Terminal:
    • The terminal limits screen, the web page and the API all call listLimits, which maps over LIMIT_KEYS = Object.keys(LIMITS). Nothing is hand-listed, so the new keys appear. test/spawn-limits-listed.test.ts pins the seven new-forge keys, and its second assertion is close to a tautology.
    • I did not add the queue state (births in flight, spawns waiting) to the admin or terminal view. birthCounts exists for it. I left that as a follow-up.

AGENTS.md describes the new bounds.

commented

The gate failure is fixed and pushed; I haven't seen the forge's check re-run on it yet.

What was wrong: an earlier bun scripts/gate.ts --fix had rewrapped two comments in test/helpers/preload.ts that this pull didn't write. The gate rule "the diff is the change" refuses that. I put both comments back on their original single lines, so the diff to that file is now only the 8 lines this pull added.

Checks run: bun scripts/gate.ts now passes. It still prints a note that those two original comment lines are over 100 columns. The note says to leave them, because the nightly rewrap handles them on main. I didn't run tsc or the tests, since the only change is comment text in a test helper.

commented

This push fixes the three failures from the check. I re-ran only the affected tests locally: they pass, and bun x tsc --noEmit and bun scripts/gate.ts --fix came back clean. I did not run the full suite.

  • Dead-code test: useWatchProbeByDefault in src/spawn/door.ts is called only from test/helpers/preload.ts, which the scanner doesn't count. It is now listed in scripts/unused-exports-allow.json, next to provideMemoryForTest and useReachByDefault, which are seams of the same kind.
  • Flow clog tests: test/brief-hot.test.ts left its repo-hot pull files in the process-wide hot-file memory. When it shared a shard with test/flow-clog-page.test.ts, /flow counted 4 pulls on src/a.ts instead of 3. It now clears them in afterAll. The flow-clog-page and brief-hot pair fails without the cleanup and passes with it.
  • Preload comments: bun scripts/gate.ts --fix rewrapped the long comments in test/helpers/preload.ts. I committed that, so it undoes the earlier "keep the preload diff" revert for those lines.

I chose not to change the births code itself, since none of the failures came from it. The check's over-budget test timings in spawn-move-cell and others are not enforced and I left them alone.

To try it, run bun test test/flow-clog-page.test.ts test/brief-hot.test.ts test/unused-exports.test.ts.

commented

This push fixes the failed "the diff is the change" gate rule: test/helpers/preload.ts now differs from main only by the lines this pull added.

An earlier gate.ts --fix had rewrapped two existing comments there, the ones above useReachByDefault and servedWindow.ms. I put both back to their original single-line form, so the diff is now just the two seam setups added at the end of the file (provideMemoryForTest and useWatchProbeByDefault).

I ran bun scripts/gate.ts and bun x tsc --noEmit before pushing, and both came back clean. The gate prints one note that the restored comments run past 100 columns. It is only a note, and the nightly rewrap fixes it on main, so don't run --fix here: it would rewrap those two lines again. I did not rerun any tests, since this change only restores comment lines.

commented

I merged main into the branch and pushed. The two conflicts, in src/spawn/host.ts and src/spawn/sleep.ts, are resolved.

Both conflicts came from the same cause. main now has the awake ceiling as the capacity setting children.awake, and awakeRefusal reads it through maxAwake(db). This pull had added a forges.awake_max limit for the same thing.

  • forges.awake_max: removed it from src/settings/limits.ts, src/spawn/sleep.ts, the limits-listed test, AGENTS.md and docs/features/forge-sleep.md. The docs now point at children.awake. FORGE_CHILD_AWAKE_MAX is still the environment default, as it was before.
  • host.ts: wakeGate keeps main's roomFor check, which puts the longest-idle forge to sleep to make room. The wakes-per-forge check next to it still reads the forges.wakes_per_forge_hour limit this pull added. main's parent's-own-questions exemption is also kept.
  • forges.wakes_per_forge_hour: stays as a limit on /admin/limits. The environment variable FORGE_CHILD_WAKES_PER_HOUR applies only while it is unset.

bun scripts/gate.ts and bun x tsc --noEmit were clean. scripts/fast.ts would have run 2971 of 3613 test files (the diff is too broad to name), so I ran the spawn, sleep, awake-room, limits and births suites by hand: 486 pass, 9 skip, 0 fail. The skips are the unix-socket suites, which the sandbox skips and the forge's check runs.

commented

The check's docs/MCP.md failure is fixed and pushed. I couldn't reproduce the other three failures, so those are untested against a fix.

The fix: docs/MCP.md was stale for the limits tool. I regenerated the table with bun scripts/mcp-table.ts, which changed one line. test/mcp-table.test.ts now passes (8 of 8) and tsc --noEmit is clean.

Not reproduced: the two "each skipped pull is logged…" tests and the "an admin presses r on the feed screen" terminal test pass here. I ran test/finishing-skips.test.ts and test/features-tui.test.ts (12 tests, all green). My change touches neither area, so I made no code change for them. If they fail again on the next check run, the log will say whether they depend on shard order or timing.

Changes requested — before the latest push

Changes needed, all in this round.

  1. FIRST, correctness: whyNot in src/spawn/births.ts compares one freemem() reading to spawn.birth_memory_mb, and its comment says births in flight are 'already counted in what is free'. They are not: a birth takes its ~1.5 GiB over the minutes until it answers. drain admits several births in one loop on the same reading, and a burst of takePlace calls does the same. So the memory rule does not bound a surge; only the count does. Subtract inFlight * birth_memory_mb (or the not-yet-taken share) from what is left, fix the comment, and add a test where memory fits one birth and the second waits with why=memory even though births_at_once is higher.
  2. A person on a phone who hits the queue gets raw JSON: the browser page for a queued spawn is unchanged, and there is no Retry-After header. A 503 'queued' must carry Retry-After (the door can set headers through the response it returns, or the spawn route can add it) and the page path must show the place and 'try again in N s' as words, not a JSON body. Add a test for the header and for the page wording.
  3. Both doors: the queue state (birthCounts: births in flight, spawns waiting) is exported only for tests and allow-listed as unused. Show it on /admin/limits and on the terminal limits screen next to the new limits, so an admin who sees strangers queued can see why. Drop the allow-list entry once it is used. test/spawn-limits-listed.test.ts is near tautological (its second assertion compares LIMIT_KEYS to LIMITS); replace it with a test of the terminal screen showing the new keys and the queue line.
  4. Direction: this is not on the owner's pipeline-first list and the pull names no pipeline issue it unblocks. State in the pull which pipeline item it serves, or a person must confirm it before it lands.
  5. Test seams in production modules (provideHoldForTest, provideMemoryForTest, useWatchProbeByDefault) are allow-listed as unused exports. Acceptable as a pattern in this repo, but keep them to one seam each and do not widen it.

The security points from the first review (queue cap, stranger-safe message, leaked place on untimed spawns, no NODE_ENV branch) look fixed and tested. I did not run the preview beyond reading the diff and the screenshots list.

commented

Turns a surge of new forges into a queue with a place and a wait instead of a refusal. The limits that bound it (spawn.births_at_once, spawn.birth_memory_mb, spawn.timers_max, spawn.anon_per_day, spawn.anon_global_per_day, forges.wakes_per_forge_hour) become settings on /admin/limits. Before, they were constants or environment values. A spawn that has no timer is now logged. This round fixes the points from the second review:

  • Memory rule now limits a surge (src/spawn/births.ts): whyNot subtracts inFlight × spawn.birth_memory_mb from the free memory it reads. Before, drain or a quick run of takePlace calls could start several births using one memory reading. The misleading comment is gone. New test in test/spawn-births.test.ts: there is memory for one birth and births_at_once is 8. The second spawn waits with why: "memory", and when the first birth is released only one of the two waiting spawns starts. This test fails without the fix.
  • A queued spawn says how long to wait:
    • doorHeaders in src/spawn/door.ts gives the 503 a Retry-After header with the same seconds as the body's retryAfter. Before, the JSON floor in json() always sent 60. POST /api/v1/forges uses it, and so does the /forges/new form.
    • The home start form now has a queued refusal (src/web/start_refusal.ts). It sends you back to / with a fixed sentence: "…yours is number N in line. Try again in S s…". The place and the wait are read from the query only when they are whole numbers within bounds; anything else gets the plain sentence.
    • The terminal start screen shows the same sentence.
    • The door's own message now says "try again in S s".
    • Tests: the header check is in test/spawn-anon.test.ts; the page wording and a forged query are in test/home-start-forge.test.ts.
  • Queue state on both doors: listLimits (src/actions/limits.ts) now includes births: birthCounts(app). /admin/limits (#births-now) and the terminal limits screen show "New forges now: N being made, M waiting their turn." next to the limits. The MCP limits tool returns the same field. birthCounts is removed from the unused-exports allow-list. test/spawn-limits-listed.test.ts now drives the terminal limits screen: it checks the six new keys and the births line, both idle and with one stranger waiting. test/limits.test.ts checks the line on the web page.
  • Test seams: still one each (provideHoldForTest, provideMemoryForTest, useWatchProbeByDefault), and none of them widened.
  • Direction: the issue has no pillar on the map. It serves P4 Forge in a link (#590): on a day the link is shared, every stranger who opens it causes a birth, and this is what keeps that day from taking down the machine. I have not linked it to the umbrella myself, so a person needs to confirm this before it lands.

Not done:

  • The memory rule now counts the full share for every birth in flight, even one that has already taken part of it. That is cautious by design: it may make someone wait a little early, but it never starts more births than the memory allows.
  • The MCP spawn tool still returns its refusal as a body only; MCP has no headers.

Tests and checks: run here: bun x tsc --noEmit, bun scripts/gate.ts --fix, bun scripts/design-lint.ts, and 20 test files covering births, the anonymous queue, start, new-forge, limits (web, terminal, MCP, API) and unused exports. They gave 341 pass, 0 fail. scripts/fast.ts named 2,972 files, too broad to run here, so the files were picked by hand.

Try it on the preview: as an admin, open /admin/limits and read the line above the limits table. Set spawn.births_at_once to 1, then start two forges from / close together. The second comes back to the home page with its place and wait in a sentence. The API answers the same case with a 503 and a matching Retry-After.

🤖 Generated with Claude Code

commented

A surge of new forges now waits in a queue and is told its place and how long to wait, instead of being refused. The limits that bound it are settings on /admin/limits: spawn.births_at_once, spawn.birth_memory_mb, spawn.timers_max, spawn.anon_per_day, spawn.anon_global_per_day and forges.wakes_per_forge_hour. Before, they were constants or environment values. A spawn past spawn.timers_max is still made, without a timer, and is now logged.

Memory rule (src/spawn/births.ts): whyNot subtracts inFlight × spawn.birth_memory_mb from the free memory it reads. A forge takes its memory over the minutes until it answers, so one reading could otherwise let several births start at once. A test gives memory for one birth with births_at_once at 8. The second spawn waits with why: "memory", and when the first is released only one waiting spawn starts. This test fails without the fix.

Queued answer:

  • doorHeaders in src/spawn/door.ts sets Retry-After to the same seconds as the body's retryAfter, on POST /api/v1/forges and the /forges/new form. Before, the 503 always said 60.
  • The home start form, the /agents form and the terminal start screen show one sentence with the place in line and the seconds to wait (startRefusalWords in src/web/start_refusal.ts).
  • The place and the wait are read back from the query only as whole numbers within bounds; anything else gets the plain "being made right now" sentence.

Queue on the limits screens: listLimits returns births (births in flight and spawns waiting). /admin/limits, the terminal limits screen and the MCP limits tool show "New forges now: N being made, M waiting their turn." birthCounts is off the unused-exports allow-list. test/spawn-limits-listed.test.ts now drives the terminal screen and checks the new keys and that line.

Merge with main:

  • AGENTS.md keeps both new notes, the births one and waiting_on_me.
  • src/http/api/forges.ts takes main's listedKept alongside doorHeaders.
  • src/web/pages/start-forge.ts keeps main's /agents refusal path, and that path also shows a queued start's place and wait.

Direction: this serves P4 Forge in a link (#590): a shared link must not take down the machine. The issue has no pillar on the map, so a person should confirm that before it lands.

Not done:

  • Every birth in flight counts its full memory share, even one that has already taken part of it. That can make someone wait a little early, but never starts more births than memory allows.
  • The MCP spawn tool returns its refusal in the body only; MCP has no headers.

Checks: the type check and bun scripts/gate.ts --fix are clean after the merge. 13 hand-picked test files covering start, /agents, spawn, births and limits passed: 103 pass, 0 fail. scripts/fast.ts named 2,972 files, too many to run here.

Try it: set spawn.births_at_once to 1 on /admin/limits, then start two forges from / close together. The second comes back with its place and wait in words. The API answers the same case with a 503 and a matching Retry-After.

🤖 Generated with Claude Code

commented

<!-- look-pages 85cc1c662a6c --> Look pages for 85cc1c6, from the gate:

  • look pages (inferred): "Looks: /admin/limits" is not a page of app.looks in forge.json; this pull changes src/web/pages/admin-limits.ts, src/web/pages/new-forge.ts, src/web/pages/start-forge.ts and 2 more; the forge pictures / (src/web/pages/start-forge.ts), /docs (src/web/pages/start-forge.ts), /explore (src/web/pages/start-forge.ts) and 24 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

A surge of new forges now waits in a queue and is told its place and about how long to wait, instead of being refused. The limits that bound a surge are settings on /admin/limits: spawn.births_at_once, spawn.birth_memory_mb, spawn.timers_max, spawn.anon_per_day, spawn.anon_global_per_day and forges.wakes_per_forge_hour. Before, they were constants or environment values. A forge made past spawn.timers_max is not timed, and the log now says so.

Memory rule (src/spawn/births.ts): each birth in flight subtracts its full spawn.birth_memory_mb from the free memory reading. A forge takes that memory over the minutes until it answers, so one reading used to let several births start at once. The test gives memory for one birth with spawn.births_at_once at 8; the second spawn waits with why: "memory". It fails without the fix.

Queued answer:

  • doorHeaders in src/spawn/door.ts sets Retry-After to the same seconds as the body's retryAfter. This applies to POST /api/v1/forges and the /forges/new form; before, the header always said 60.
  • The home start form, the /agents form and the terminal start screen say the place in line and the seconds to wait in one sentence (startRefusalWords in src/web/start_refusal.ts).
  • The page reads the place and wait back from the query only as whole numbers within bounds. Anything else gets the plain "being made right now" sentence.

Queue on the limits screens: listLimits returns births, the forges being made and the spawns waiting. /admin/limits, the terminal limits screen and the MCP limits tool show "New forges now: N being made, M waiting their turn." birthCounts is off the unused-exports allow-list. test/spawn-limits-listed.test.ts now drives the terminal screen and checks the new keys and that line.

Merge with main:

  • AGENTS.md keeps both new notes, births and waiting_on_me.
  • src/http/api/forges.ts takes main's listedKept alongside doorHeaders.
  • src/web/pages/start-forge.ts keeps main's /agents refusal path, which now also says a queued start's place and wait.
  • test/statement.test.ts is back as main has it. gate --fix had rewrapped a comment there during the merge, and that is what failed the last check.

Direction: this serves P4 Forge in a link (#590): a shared link must not take down the machine. The issue has no pillar on the map, so a person should confirm that before it lands.

Not done:

  • A birth that has already taken part of its memory still counts its full share. That can make someone wait a little early, but it never starts more births than memory allows.
  • The MCP spawn tool gives its refusal in the body only, because MCP has no headers.

Checks: the type check and bun scripts/gate.ts are clean. Of the tests, I ran the statement, home start, births and limits-screen files after the last change: 28 pass, 0 fail. The wider set from the merge (start, /agents, spawn, births, limits) was 103 pass, 0 fail. scripts/fast.ts named 2,972 files, too many to run here.

Try it: set spawn.births_at_once to 1 on /admin/limits, then start two forges from / close together. The second comes back with its place and wait in words. The API answers the same case with a 503 and a matching Retry-After.

🤖 Generated with Claude Code

commented

<!-- look-pages e349d70f6078 --> Look pages for e349d70, from the gate:

  • look pages (inferred): "Looks: /admin/limits" is not a page of app.looks in forge.json; this pull changes src/web/pages/admin-limits.ts, src/web/pages/new-forge.ts, src/web/pages/start-forge.ts and 2 more; the forge pictures / (src/web/pages/start-forge.ts), /docs (src/web/pages/start-forge.ts), /explore (src/web/pages/start-forge.ts) and 24 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The limits that bound a surge of new forges are now settings on /admin/limits instead of constants and environment values. One rule decides whether a new forge may start, using the machine's own figures. A spawn that runs without a timer used to be silent and is now logged.

  • Admission (births.ts): a new forge starts only when a place is free under spawn.births_at_once and the memory left covers birth_memory_mb. Each start already under way counts its full birth_memory_mb against that memory. Before this, one reading of free memory could let several starts through on memory only one of them would get.
  • Waiting line: it holds at most 4 × births_at_once. People who are not signed in may use only a quarter of it, and signed-in people go first. If no timer starts for a start, its place is given back. The machine's memory figures are not shown to people who are not signed in.
  • Queued answer: the API and the new-forge form send Retry-After set to the seconds named in the body (before, the JSON fallback always said 60). The start form on the home page and the terminal start screen say in words the place in line and the wait. They read only whole numbers from the query.
  • Counts: /admin/limits, the terminal limits screen and the MCP limits tool show how many forges are starting and how many are waiting, next to the limits that decide it.
  • Looks: /admin/limits is now in both app.looks and app.looksAsOwner in forge.json. The gate had refused this pull's Looks: /admin/limits line because app.looks did not list the page. It is an admin page like /system, so it is pictured as the owner. test/looks-owner.test.ts now checks that it answers the preview admin with no credential in the page, and the list of non-top-level looks in test/forge-looks.test.ts names it.

I did not add a new door for the counts: they sit on the existing limits screen. The how-to is in docs/features/forge-sleep.md.

bun scripts/gate.ts and bun x tsc --noEmit pass. test/looks-owner.test.ts, test/forge-looks.test.ts and test/refresh-meta.test.ts pass (65 tests). The rest of the suite is left to the forge's check on this push.

To try it on the preview: open /admin/limits as an admin and read the line above the limits table. Then POST /api/v1/forges more times than spawn.births_at_once allows and read Retry-After on the queued answer.

🤖 Generated with Claude Code

Approved — before the latest push

Earlier asks are met. (1) Memory rule: whyNot now subtracts inFlight × birth_memory_mb, with a test. (2) Queued answer: Retry-After comes from doorHeaders, and the home, /agents, new-forge and terminal start screens say the place and wait in words, with the query read as bounded whole numbers only. (3) Both doors: the births line is on /admin/limits, the terminal limits screen and the MCP limits tool, and the terminal screen is tested. (4) Direction: the pull names P4 (#590); a person should confirm it as the pull says. (5) Seams: one each, not widened. The latest change adds /admin/limits to app.looks and looksAsOwner, with tests. The preview answers 200. I could not diff against 0dbc1125fbb6 because the commit is not in the clone, so I judged the change as given. Non-blocking: the docs/features how-to sits in forge-sleep.md rather than its own page, which is acceptable.

commented

<!-- look-pages e74cd400ebb7 --> Look pages for e74cd40, from the gate:

  • look pages (named): this pull changes src/web/pages/admin-limits.ts, src/web/pages/new-forge.ts, src/web/pages/start-forge.ts; a commit names /admin/limits
commented

The limits that bound a surge of new forges are now settings on /admin/limits instead of constants and environment values. One rule decides whether a new forge may start, using the machine's own figures. A spawn that runs without a timer used to be silent and is now logged.

  • Admission (src/spawn/births.ts): a new forge starts only when a place is free under spawn.births_at_once and the memory left covers birth_memory_mb. Each start already under way counts its full birth_memory_mb against that memory. Before, one reading of free memory could let several starts through on memory only one of them would get.
  • Waiting line: it holds at most 4 × births_at_once. People who are not signed in may fill only a quarter of it, and signed-in people go first. If a forge starts but no timer is set for it, its place is given back. The machine's memory figures are not shown to people who are not signed in.
  • Queued answer: the API and the new-forge form send Retry-After set to the seconds named in the body (before, the JSON fallback always said 60). The home start form and the terminal start screen say in words the place in line and the wait. They read only whole numbers from the query.
  • Counts: /admin/limits, the terminal limits screen and the MCP limits tool show how many forges are starting and how many are waiting, next to the limits that decide it.
  • Looks: /admin/limits is in both app.looks and app.looksAsOwner in forge.json. It is an admin page like /system. test/looks-owner.test.ts checks it answers the preview admin with no credential in the page, and test/forge-looks.test.ts lists it among the looks below the top level.

The last check (run 01m7hykfdzev30n0bpqnzgcqk8) failed on tests this pull does not touch. These 107 tests pass on the merge with current main, and three copies of the first three below also passed when run four at a time:

  • attention, tui-crawlers, media:
    • All pass here, after the merge and when run four at a time.
    • The one failure whose assertion survives in the log is the media test. It got "The forge is busy…" back: src/http/crawl.ts sends that when signed-out requests have waited too long, which happens on a loaded runner. That runner also had about 13% disk free.
    • The log's start is cut off, so the attention and crawler assertions are not in it.
  • Preview-seed test in looks-owner: over its 2 s budget (2.41 s in the check). It runs about 2 s on main too: 1.98 s and 2.26 s here, against 1.50 s and 1.38 s on this branch.

I checked that this pull's preload additions add no import cost: births.ts and door.ts add 0 ms once the modules main already preloads are in. This push merges current main, so the check runs on a fresh base. Load-related red runs like these are what #14187 is working on. I did not add the seed test to SLOW or change any of those assertions.

bun scripts/gate.ts --fix and bun x tsc --noEmit pass.

To try it on the preview, open /admin/limits as an admin and read the line above the limits table. Then POST /api/v1/forges more times than spawn.births_at_once allows and read Retry-After on the queued answer.

🤖 Generated with Claude Code

commented

<!-- look-pages 46fcd33a8617 --> Look pages for 46fcd33, from the gate:

  • look pages (named): this pull changes src/web/pages/admin-limits.ts, src/web/pages/new-forge.ts, src/web/pages/start-forge.ts; a commit names /admin/limits
commented

I fixed one real cause of the failed check and pushed, but I couldn't reproduce most of the seven failures, so I can't promise the next check is green.

What I fixed: test/spawn-births.test.ts reset the disk reader with provideDiskForTest(undefined) after each test. That handed every later test file in the same shard the runner's real disk, which was 13.6% free (under the 15% floor), so tests that start forges, agents or cells could be refused. It now restores the suite's roomy reading (roomyDisk, now exported from test/helpers/roomy-disk.ts) and deletes the data directories it creates. A new test fails if the real reading leaks again.

Not fixed:

  • Six failures: web-ui, shared-red, hardening, the memory sampler, proveHosted and the preview seed's 2.07 s timing all pass locally, alone and grouped with the spawn tests. A low disk forced in front of web-ui and shared-red didn't break them either. The log stops before those shards' details, so I couldn't find their cause.
  • One failure is already on main: the crew-concurrency test in test/capacity.test.ts fails about one run in five when grouped with other files, with main's preload as well as this pull's.
  • Same reset elsewhere: test/maintain-evidence.test.ts and test/no-alarm-banner.test.ts also call provideDiskForTest(undefined). They're outside this pull, so I left them.

Gate and tsc pass. The covering files (spawn, limits, capacity and the named failing files) pass apart from that intermittent crew-concurrency test. I ran named files only; the forge runs the full suite on this push.

---

Births now go through one admission rule, and most of the limits on a surge of new forges are settings an admin can change on /admin/limits without a restart.

What changed

  • Births: src/spawn/births.ts is new. A spawn needs three things:
    • fewer than spawn.births_at_once forges being made at once (default 8);
    • spawn.birth_memory_mb (1,536) free above forge.memory_floor_mb, counting what births already in flight will still take;
    • the disk above its existing floor.

    A spawn that doesn't fit waits in a first-in-first-out line for up to 15 s, with signed-in people ahead of strangers. The line holds at most four times the births limit, and strangers may fill only a quarter of it. Past the wait, the spawn gets 503 {error:"queued", place, aboutS, retryAfter, why} with a Retry-After header. Strangers aren't shown the machine's figures. A birth keeps its place until the new forge answers, or 5 minutes at most. aboutS comes from a running average of measured births, starting at 4 minutes.

  • Door: src/spawn/door.ts takes a place before making the forge and gives it back at once if no forge was made or no watch started. The start page shows a queued spawn as words.
  • Settings (all on /admin/limits, which also shows births in flight and spawns waiting, as does the terminal's limits screen):
    • spawn.anon_per_day (100) and spawn.anon_global_per_day (2,000) replace the constants in rate.ts.
    • spawn.timers_max (20) replaces MAX_WATCHERS. A spawn past it is now logged as made but not timed, so it is no longer silent.
    • forges.awake_max (8) and forges.wakes_per_forge_hour (20). FORGE_CHILD_AWAKE_MAX and FORGE_CHILD_WAKES_PER_HOUR still apply while the setting is unset.
    • spawn.births_at_once and spawn.birth_memory_mb, the two new ones.
  • Test preload: gives the suite 64 GiB of free memory for births and makes new child forges answer their first check at once. Without that, every spawn in the suite would hold its place for the full 5 minutes.
  • /admin/limits in the forge's screenshots: it is now captured as the owner.

What I did not do

  • Idle time, NEW_FORGE_MS and held wake requests (wakeLimits) are still constants, because the idle timing is tangled with the cell sweep.
  • No queue ticket: a spawn that gets the 503 rejoins at the back when it retries.
  • No "being made" page: the queued state is carried by the 503 and the start page's words.

To try it: set spawn.births_at_once to 1 on /admin/limits, then post two spawns to /api/v1/forges. The second waits up to 15 s, then answers 503 with place: 1, aboutS and a Retry-After header.

🤖 Generated with Claude Code

Closes #13822

commented

Merged main into this branch again. Only src/spawn/host.ts conflicted, in two lines of wakeGate, and the fix keeps both sides:

  • From main: wakingResponse no longer takes parent, so both of its calls now begin wakingResponse(request, name, …).
  • From this branch: the new-forge limit still passes process.env, parent.db to rates.refusal, so the configured limits still bound a surge of new forges.

bun x tsc --noEmit is clean. bun scripts/gate.ts --fix changed nothing. scripts/fast.ts said the diff was too broad to name tests, so I picked 12 files that cover the spawn gate, waking, new-forge timing and the limits pages (spawn-births, spawn-limits-listed, spawn-anon, spawn-live, spawn-host, spawn-gate-e2e, spawn-person-limits, wake-budget, waking-plain, limits, admin-limits-words, home-start-forge). All 74 tests in them pass. Tests that need unix sockets or cells are skipped in this sandbox; the forge's check runs them on this push.

To try it, open the preview at https://lab--13835--forge-trevato-apps.open-platform.sh, check that /admin/limits lists the new-forge limits, and start forges past the limit to see the refusal page.

🤖 Generated with Claude Code

commented

<!-- look-pages 2d9ed8f961e0 --> Look pages for 2d9ed8f, from the gate:

  • look pages (named): this pull changes src/web/pages/admin-limits.ts, src/web/pages/new-forge.ts, src/web/pages/start-forge.ts; a commit names /admin/limits
commented

Nothing to change, so I made no commit and the branch stays at 2d9ed8f. The gate's "look pages" note is the satisfied form: it is named, not inferred or none.

Pull description:

The limits that bound a surge of new forges are now settings the owner can see and change on /admin/limits. Before, they were constants and env values, and one of them refused without a word. A queued spawn now says its place and wait as words, with a Retry-After header.

  • Admission (src/spawn/): new forges are let in by one rule based on the machine's figures. Forges still being made count against the memory they haven't taken yet. The waiting line has a bound. The machine's figures are kept from strangers, and a spawn that starts with no timer is logged.
  • Pages (src/web/pages/):
    • /admin/limits shows forges being made and spawns waiting at both limits.
    • The new-forge form and the home and /agents start form pass the queue's wait and Retry-After through on a refusal.
  • Merge: main was merged in. The conflict in wakeGate (src/spawn/host.ts) keeps main's shorter wakingResponse(request, …) and this branch's rates.refusal(…, process.env, parent.db).

Pictures: the forge pictures /admin/limits, light and dark, as owner and visitor. It is in app.looks and looksAsOwner, and commits e74cd400e and 47e093b6e name it in their Looks: /admin/limits lines.

I chose not to name a page for new-forge.ts and start-forge.ts. Their changes only show in the response to a refused POST, when a spawn is queued, and no GET page in app.looks can show that. test/home-start-forge.test.ts and test/spawn-births.test.ts cover those words and headers instead.

To try it: sign in as owner on the preview at https://lab--13835--forge-trevato-apps.open-platform.sh and open /admin/limits to see the new-forge limits. Start forges past the limit to see the queued reply, with its place and wait in words.

🤖 Generated with Claude Code

Approved

Approved. The earlier asks stand met: the memory rule counts births in flight, a queued 503 carries Retry-After and is worded on the home, agents and terminal start paths, and births in flight and waiting show on /admin/limits and the terminal limits screen. The queue is capped and strangers get no machine figures. A place is released if no timer starts. Changes since my last look are a merge with main and the test fix that restores the roomy disk reading after spawn-births, which is test-only. The commit e74cd400ebb7 is not in my clone, so I read the final diff rather than the incremental one. Non-blocking: the direction (pipeline first) does not name this pull. The pillar (P4, #590) needs a person to confirm it before it lands.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs