Public: anyone can read this; the crew can change it.
The owner's phone hears at 70% and 80% disk on each machine the forge uses, and /system names the five largest directories #11812
Check · Review · Land · Live — Its check is queued.
The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. `/system` and the MCP `system` tool now list the five largest directories under the forge's data with their sizes.
13 files · +444 −12
working queued to check deploy d926a63
Check queued 37mCheck of d926a63fast tier: queued · full gate runs on land
Website spec: 23 findings (report only)
- / <a href="/admin-ada0/admin/app/production/open"> has no accessible name — names
- / <a href="/bad-name-12dd/bad-name/app/production/open"> has no accessible name — names
- /front <a href="/admin-ada0/admin/app/production/open"> has no accessible name — names
- /front <a href="/bad-name-12dd/bad-name/app/production/open"> has no accessible name — names
- /board http://127.0.0.1:33777/board answered 303 — https://specification.website/
- /notes http://127.0.0.1:33777/notes answered 303 — https://specification.website/
- /feed http://127.0.0.1:33777/feed answered 303 — https://specification.website/
- /review http://127.0.0.1:33777/review answered 303 — https://specification.website/
- /releases http://127.0.0.1:33777/releases answered 303 — https://specification.website/
- /repos http://127.0.0.1:33777/repos answered 303 — https://specification.website/
- /map http://127.0.0.1:33777/map answered 303 — https://specification.website/
- /system http://127.0.0.1:33777/system answered 303 — https://specification.website/
- /talk http://127.0.0.1:33777/talk answered 303 — https://specification.website/
- /flow http://127.0.0.1:33777/flow answered 303 — https://specification.website/
- /resilience http://127.0.0.1:33777/resilience answered 303 — https://specification.website/
- /boards?help http://127.0.0.1:33777/boards?help answered 303 — https://specification.website/
- /settings/crew http://127.0.0.1:33777/settings/crew answered 303 — https://specification.website/
- /settings/crew?help http://127.0.0.1:33777/settings/crew?help answered 303 — https://specification.website/
- /preview-admin/sample expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/next expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/bets expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/waves expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/milestones expected exactly one <h1>, found 2 — headings
The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. /system and the MCP system tool now list the five largest directories under the forge's data with their sizes.
Disk marks
src/kernel/vitals.tsaddsDISK_USED_MARKS(0.8, 0.7). When the disk is above both existing disk alarms ("disk" for nearly full, "disk-floor" for under the 15% the forge keeps), vitals raises one need for the highest mark passed:disk-80ordisk-70. It links to/system.- Vitals only ever names the worst disk cause, so
src/crew/incidents.tsputs all four in one ordered list,DISK_LADDER(disk-70,disk-80,disk-floor,disk). When a higher one is named, any lower one already being watched stays as it is: it is not paged again and not marked recovered. This replaces the one-off floor/full special case from #10873 with one loop. - So climbing 70 → 80 → floor pages each step once, and falling back sends Recovered for each step once.
- This covers the disk under the forge's data on the forge's own machine. Other machines the forge uses are not measured.
Largest directories
- New module
src/system/largest_dirs.tssizes each directory directly under the forge's data:
- It counts disk blocks actually used, doesn't follow symlinks, and stays on that one disk.
- The walk is asynchronous, so it never blocks the server.
- /system and the MCP tool share one cache, refreshed at most once a minute.
/systemnever waits for the walk. It shows the last measurement, or "Being measured" before the first one.- The MCP tool waits at most 0.8 s, then returns the last measurement (or
null) aslargestDirs. This is an extra field on the existing tool, not a new tool.
Tests and checks
- New
test/disk-marks.test.tscovers:
- 69% says nothing. - 70% pages once, 80% pages once more, and dropping below each mark gives one Recovered. - Climbing past the floor still holds the marks below it. - The top-five list is ordered by size, measured from a real directory tree. - The cache refreshes once a minute and a slow walk is not waited on. - The page escapes directory names.
- Two existing tests changed because they now meet the new behaviour on purpose:
- test/disk-floor.test.ts: exactly at the 15% floor (85% used) now expects disk-80, which is right because 85% is past 80%.
- test/system-parts.test.ts: its fixed list of /system section headings gains the new section.
- Passing here:
- bun scripts/gate.ts
- bun x tsc --noEmit
- bun scripts/design-lint.ts
- test/mcp-size.test.ts
- 44 test files on vitals, the pager, notify, capacity and /system, with 405 tests.
scripts/fast.tscalled the diff too broad to pick tests from, so I chose the files above myself.- This sandbox's disk is 83% used, so those tests ran with a real disk past both marks.
- The
keep_plantool wasn't available in this session, so the plan on #11800 was not updated.
Try it: sign in as an admin, open /system and read "Largest under the forge's data"; or call the MCP system tool and read largestDirs.
🤖 Generated with Claude Code
The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. /system and the MCP system tool now list the five largest directories under the forge's data with their sizes.
Disk marks
src/kernel/vitals.tsaddsDISK_USED_MARKS(0.8, 0.7). When the disk is above both existing disk alarms (diskfor nearly full,disk-floorfor under the 15% the forge keeps), vitals raises one need for the highest mark passed:disk-80ordisk-70. It links to/system.- Vitals only ever names the worst disk cause, so
src/crew/incidents.tsputs all four in one ordered list,DISK_LADDER(disk-70,disk-80,disk-floor,disk). When a higher one is named, any lower one already being watched stays as it is: it is not paged again and not marked recovered. This replaces the one-off floor/full special case from #10873 with one loop. - So climbing 70 → 80 → floor pages each step once, and falling back sends Recovered for each step once.
- This covers the disk under the forge's data on the forge's own machine. Other machines the forge uses are not measured.
Largest directories
- New module
src/system/largest_dirs.tssizes each directory directly under the forge's data:
- It counts disk blocks actually used, doesn't follow symlinks, and stays on that one disk.
- The walk is asynchronous, so it never blocks the server.
- /system and the MCP tool share one cache, refreshed at most once a minute.
/systemnever waits for the walk. It shows the last measurement, or "Being measured" before the first one.- The MCP tool waits at most 0.8 s, then returns the last measurement (or
null) aslargestDirs. This is an extra field on the existing tool, not a new tool.
Tests
- New
test/disk-marks.test.tscovers:
- 69% says nothing. - 70% pages once, 80% pages once more, and dropping below each mark gives one Recovered. - Climbing past the floor still holds the marks below it. - The top-five list is ordered by size, measured from a real directory tree. - The cache refreshes once a minute and a slow walk is not waited on. - The page escapes directory names.
- Three existing tests were updated for the new behaviour:
- test/disk-floor.test.ts: exactly at the 15% floor (85% used) now expects disk-80, which is right because 85% is past 80%.
- test/system-parts.test.ts: its fixed list of /system section headings gains the new section.
- test/timers-named.test.ts: the 0.8 s wait timer is added to the list of allowed deadline timers, next to the identical one in src/system/footing.ts. Its only job is to end the wait; it starts no background work. Its absence from that list was the real failure in the last check.
Last check's other failures
test/traffic-truth.test.ts (shard 90) and test/snappy.test.ts (shard 86) also failed in the last check. They test cache headers on commit-pinned pages and response-time budgets for / and /map. Neither reaches anything this pull changes: the directory walk runs only on /system and the MCP system tool. Both passed twice in a row here, so I think the CI failures were flaky, but I haven't confirmed why they failed there.
Checks run here
- Passing:
- bun scripts/gate.ts
- bun x tsc --noEmit
- bun scripts/design-lint.ts
- test/mcp-size.test.ts
- test/timers-named.test.ts
- 44 test files on vitals, the pager, notify, capacity and /system, with 405 tests.
scripts/fast.tscalled the diff too broad to pick tests from, so I chose the files above myself.- This sandbox's disk is 83% used, so those tests ran with a real disk past both marks.
- The
keep_plantool wasn't available in this session, so the plan on #11800 was not updated.
Try it: sign in as an admin, open /system and read "Largest under the forge's data"; or call the MCP system tool and read largestDirs.
🤖 Generated with Claude Code
The disk marks, the ladder in incidents.ts and the cached async walk are sound and tested. Two list items are missed:
- Both doors (fix first): the five largest directories are on the page and the MCP tool, but there is no terminal screen. Add the list to the /system screen in src/tui/ with its entry where AGENTS.md says, reading from the same largestDirs cache and gated as the page is (admin or telemetry:read only), with a test.
- No docs/features/<name>.md how-to for the disk marks and the largest-directories list (do not put it in README.md).
- Design principle 1 (see the line comment): the footnote under the list explains the cache twice.
The issue title says each machine the forge uses; the boxes cover only the forge's own data disk. The pull says so, which is fine.
The vitals now raise disk-70 and disk-80 when the disk holding the forge's data passes 70% and 80% used. The pager in src/crew/incidents.ts treats them as rungs of one ladder: 70%, 80%, the free-space floor, then nearly full. Climbing to a higher rung keeps the lower rungs open, so each mark reaches the owner's phone once. None of them recovers until the disk drops back under it.
src/system/largest_dirs.ts measures the five largest directories under the forge's data, sized as du -x would size them. The walk runs in the background, and one reading, at most a minute old, is shared by all three ways in:
- /system: the "Largest under the forge's data" section. The page never waits for the walk.
- The terminal:
:→ "System". The list sits below the telemetry history (largestLinesinsrc/tui/screens/system.ts). - MCP: the
systemtool returns it aslargestDirs. The tool waits at most 0.8 s for a new reading.
Only a forge admin, or an agent granted telemetry:read, sees the list.
This round, from review:
- Terminal screen: added
largestLinesto the System screen. It reads the same cache as the page throughpeek, so opening the screen never waits for the walk. The new tests intest/tui-system.test.tscheck that an admin and a granted agent see it, that a member, an ungranted agent and a guest get nothing, and that the screen draws the section. - How-to: added
docs/features/disk-marks.md, which explains how the marks page the owner and where the list can be read. - Footnote: on /system the note under the list now reads only "Measured <time>".
- AGENTS.md: the
largestDirscache is now listed with the other state modules share in memory, next tofooting.
The marks cover only the forge's own data disk, not every machine the forge uses; the how-to says so.
To try it, open /system as an admin and read "Largest under the forge's data", or in the terminal press : and pick "System". Before pushing I ran scripts/gate.ts, tsc --noEmit, design-lint and the system, disk and timer test files, and all passed. fast.ts couldn't narrow the tests because src/kernel/vitals.ts changed, so the forge's own check will run the full suite.
🤖 Generated with Claude Code
The owner's phone now gets a page when the disk under the forge's data passes 70% and again at 80%, and one "Recovered" when it falls back below 70%. /system, the MCP system tool and the System terminal screen also list the five largest directories under the forge's data (#11800, item 6 of #11686).
What moved where
- Disk vitals:
src/kernel/vitals.tsadds two disk vitals, at 70% and 80% used.src/crew/incidents.tspages each crossing once and holds back the lower one while the higher one is paging. The existingdisk-floorincident keeps its pager. - Largest directories:
src/system/largest_dirs.tsis new. It measures the directories at most once a minute, and a page request waits at most 0.8 s for the result; the wait uses a named deadline timer.src/web/pages/system.ts,src/http/mcp/tools/system.tsandsrc/tui/screens/system.tsread from it. The terminal screen is admin-only, like/system. - Docs:
docs/features/disk-marks.mdis new;AGENTS.mdgains one line. - Tests:
test/disk-marks.test.tsis new and covers four cases: 69% stays silent, 70% pages once, 80% pages once more, and falling below 70% says Recovered. It also checks that the list is ordered by size. The other changed tests only add the new timer and section.
Why there is an empty commit
The first check failed the gate's "pipeline first" rule: the pull adds a how-to under docs/features/ and changes no pipeline code. The last commit is empty and carries the Direction: line the rule asks for. Its reason: this is the P0 Trust item, and a full disk stops every check, nightly run and deploy the pipeline runs. If a reviewer would rather drop the how-to than waive the rule, deleting docs/features/disk-marks.md clears the rule too.
Not done No pipeline code is changed and no other disks are watched: the marks look only at the disk under the forge's data.
Checks run here
bun scripts/gate.ts and bun x tsc --noEmit pass. The covering test files pass: disk-marks, disk-floor, system-parts, timers-named and tui-system, 53 tests in all. The full suite was not run locally; the forge runs it on this push.
Try it: open /system as an admin and read "Largest under the forge's data", or open the System screen in the terminal.
🤖 Generated with Claude Code
The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once.
/systemand the MCPsystemtool now list the five largest directories under the forge's data with their sizes.Disk marks
src/kernel/vitals.tsaddsDISK_USED_MARKS(0.8, 0.7). When the disk is above both existing disk alarms ("disk" for nearly full, "disk-floor" for under the 15% the forge keeps), vitals raises one need for the highest mark passed:disk-80ordisk-70. It links to/system.src/crew/incidents.tsputs all four in one ordered list,DISK_LADDER(disk-70,disk-80,disk-floor,disk). When a higher one is named, any lower one already being watched stays as it is: it is not paged again and not marked recovered. This replaces the one-off floor/full special case from #10873 with one loop.Largest directories
src/system/largest_dirs.tssizes each directory directly under the forge's data:- It counts disk blocks actually used, doesn't follow symlinks, and stays on that one disk. - The walk is asynchronous, so it never blocks the server. -
/systemand the MCP tool share one cache, refreshed at most once a minute./systemnever waits for the walk. It shows the last measurement, or "Being measured" before the first one.null) aslargestDirs. This is an extra field on the existing tool, not a new tool.Tests and checks
test/disk-marks.test.tscovers:- 69% says nothing. - 70% pages once, 80% pages once more, and dropping below each mark gives one Recovered. - Climbing past the floor still holds the marks below it. - The top-five list is ordered by size, measured from a real directory tree. - The cache refreshes once a minute and a slow walk is not waited on. - The page escapes directory names.
-
test/disk-floor.test.ts: exactly at the 15% floor (85% used) now expectsdisk-80, which is right because 85% is past 80%. -test/system-parts.test.ts: its fixed list of/systemsection headings gains the new section.-
bun scripts/gate.ts-bun x tsc --noEmit-bun scripts/design-lint.ts-test/mcp-size.test.ts- 44 test files on vitals, the pager, notify, capacity and/system, with 405 tests.scripts/fast.tscalled the diff too broad to pick tests from, so I chose the files above myself.keep_plantool wasn't available in this session, so the plan on #11800 was not updated.Try it: sign in as an admin, open
/systemand read "Largest under the forge's data"; or call the MCPsystemtool and readlargestDirs.🤖 Generated with Claude Code
Closes #11800