Public: anyone can read this; the crew can change it.

The owner's phone hears at 70% and 80% disk on each machine the forge uses, and /system names the five largest directories #11812

opened this

Check · Review · Land · Live — Its check is queued. Next: reviewer, pushed since the last review.

Ready to land

The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. `/system` and the MCP `system` tool now list the five largest directories under the forge's data with their sizes.

13 files · +444 −12

working queued to check deploy d926a63

Check queued 37mCheck of d926a63fast tier: queued · full gate runs on land

Website spec: 23 findings (report only)
opened this

The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. /system and the MCP system tool now list the five largest directories under the forge's data with their sizes.

Disk marks

  • src/kernel/vitals.ts adds DISK_USED_MARKS (0.8, 0.7). When the disk is above both existing disk alarms ("disk" for nearly full, "disk-floor" for under the 15% the forge keeps), vitals raises one need for the highest mark passed: disk-80 or disk-70. It links to /system.
  • Vitals only ever names the worst disk cause, so src/crew/incidents.ts puts all four in one ordered list, DISK_LADDER (disk-70, disk-80, disk-floor, disk). When a higher one is named, any lower one already being watched stays as it is: it is not paged again and not marked recovered. This replaces the one-off floor/full special case from #10873 with one loop.
  • So climbing 70 → 80 → floor pages each step once, and falling back sends Recovered for each step once.
  • This covers the disk under the forge's data on the forge's own machine. Other machines the forge uses are not measured.

Largest directories

  • New module src/system/largest_dirs.ts sizes each directory directly under the forge's data:

- It counts disk blocks actually used, doesn't follow symlinks, and stays on that one disk. - The walk is asynchronous, so it never blocks the server. - /system and the MCP tool share one cache, refreshed at most once a minute.

  • /system never waits for the walk. It shows the last measurement, or "Being measured" before the first one.
  • The MCP tool waits at most 0.8 s, then returns the last measurement (or null) as largestDirs. This is an extra field on the existing tool, not a new tool.

Tests and checks

  • New test/disk-marks.test.ts covers:

- 69% says nothing. - 70% pages once, 80% pages once more, and dropping below each mark gives one Recovered. - Climbing past the floor still holds the marks below it. - The top-five list is ordered by size, measured from a real directory tree. - The cache refreshes once a minute and a slow walk is not waited on. - The page escapes directory names.

  • Two existing tests changed because they now meet the new behaviour on purpose:

- test/disk-floor.test.ts: exactly at the 15% floor (85% used) now expects disk-80, which is right because 85% is past 80%. - test/system-parts.test.ts: its fixed list of /system section headings gains the new section.

  • Passing here:

- bun scripts/gate.ts - bun x tsc --noEmit - bun scripts/design-lint.ts - test/mcp-size.test.ts - 44 test files on vitals, the pager, notify, capacity and /system, with 405 tests.

  • scripts/fast.ts called the diff too broad to pick tests from, so I chose the files above myself.
  • This sandbox's disk is 83% used, so those tests ran with a real disk past both marks.
  • The keep_plan tool wasn't available in this session, so the plan on #11800 was not updated.

Try it: sign in as an admin, open /system and read "Largest under the forge's data"; or call the MCP system tool and read largestDirs.

🤖 Generated with Claude Code

Closes #11800

commented

The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. /system and the MCP system tool now list the five largest directories under the forge's data with their sizes.

Disk marks

  • src/kernel/vitals.ts adds DISK_USED_MARKS (0.8, 0.7). When the disk is above both existing disk alarms ("disk" for nearly full, "disk-floor" for under the 15% the forge keeps), vitals raises one need for the highest mark passed: disk-80 or disk-70. It links to /system.
  • Vitals only ever names the worst disk cause, so src/crew/incidents.ts puts all four in one ordered list, DISK_LADDER (disk-70, disk-80, disk-floor, disk). When a higher one is named, any lower one already being watched stays as it is: it is not paged again and not marked recovered. This replaces the one-off floor/full special case from #10873 with one loop.
  • So climbing 70 → 80 → floor pages each step once, and falling back sends Recovered for each step once.
  • This covers the disk under the forge's data on the forge's own machine. Other machines the forge uses are not measured.

Largest directories

  • New module src/system/largest_dirs.ts sizes each directory directly under the forge's data:

- It counts disk blocks actually used, doesn't follow symlinks, and stays on that one disk. - The walk is asynchronous, so it never blocks the server. - /system and the MCP tool share one cache, refreshed at most once a minute.

  • /system never waits for the walk. It shows the last measurement, or "Being measured" before the first one.
  • The MCP tool waits at most 0.8 s, then returns the last measurement (or null) as largestDirs. This is an extra field on the existing tool, not a new tool.

Tests and checks

  • New test/disk-marks.test.ts covers:

- 69% says nothing. - 70% pages once, 80% pages once more, and dropping below each mark gives one Recovered. - Climbing past the floor still holds the marks below it. - The top-five list is ordered by size, measured from a real directory tree. - The cache refreshes once a minute and a slow walk is not waited on. - The page escapes directory names.

  • Two existing tests changed because they now meet the new behaviour on purpose:

- test/disk-floor.test.ts: exactly at the 15% floor (85% used) now expects disk-80, which is right because 85% is past 80%. - test/system-parts.test.ts: its fixed list of /system section headings gains the new section.

  • Passing here:

- bun scripts/gate.ts - bun x tsc --noEmit - bun scripts/design-lint.ts - test/mcp-size.test.ts - 44 test files on vitals, the pager, notify, capacity and /system, with 405 tests.

  • scripts/fast.ts called the diff too broad to pick tests from, so I chose the files above myself.
  • This sandbox's disk is 83% used, so those tests ran with a real disk past both marks.
  • The keep_plan tool wasn't available in this session, so the plan on #11800 was not updated.

Try it: sign in as an admin, open /system and read "Largest under the forge's data"; or call the MCP system tool and read largestDirs.

🤖 Generated with Claude Code

commented

The owner's phone now hears when the disk under the forge's data passes 70% used and again at 80%. Each crossing pages once, and going back under each mark sends Recovered once. /system and the MCP system tool now list the five largest directories under the forge's data with their sizes.

Disk marks

  • src/kernel/vitals.ts adds DISK_USED_MARKS (0.8, 0.7). When the disk is above both existing disk alarms (disk for nearly full, disk-floor for under the 15% the forge keeps), vitals raises one need for the highest mark passed: disk-80 or disk-70. It links to /system.
  • Vitals only ever names the worst disk cause, so src/crew/incidents.ts puts all four in one ordered list, DISK_LADDER (disk-70, disk-80, disk-floor, disk). When a higher one is named, any lower one already being watched stays as it is: it is not paged again and not marked recovered. This replaces the one-off floor/full special case from #10873 with one loop.
  • So climbing 70 → 80 → floor pages each step once, and falling back sends Recovered for each step once.
  • This covers the disk under the forge's data on the forge's own machine. Other machines the forge uses are not measured.

Largest directories

  • New module src/system/largest_dirs.ts sizes each directory directly under the forge's data:

- It counts disk blocks actually used, doesn't follow symlinks, and stays on that one disk. - The walk is asynchronous, so it never blocks the server. - /system and the MCP tool share one cache, refreshed at most once a minute.

  • /system never waits for the walk. It shows the last measurement, or "Being measured" before the first one.
  • The MCP tool waits at most 0.8 s, then returns the last measurement (or null) as largestDirs. This is an extra field on the existing tool, not a new tool.

Tests

  • New test/disk-marks.test.ts covers:

- 69% says nothing. - 70% pages once, 80% pages once more, and dropping below each mark gives one Recovered. - Climbing past the floor still holds the marks below it. - The top-five list is ordered by size, measured from a real directory tree. - The cache refreshes once a minute and a slow walk is not waited on. - The page escapes directory names.

  • Three existing tests were updated for the new behaviour:

- test/disk-floor.test.ts: exactly at the 15% floor (85% used) now expects disk-80, which is right because 85% is past 80%. - test/system-parts.test.ts: its fixed list of /system section headings gains the new section. - test/timers-named.test.ts: the 0.8 s wait timer is added to the list of allowed deadline timers, next to the identical one in src/system/footing.ts. Its only job is to end the wait; it starts no background work. Its absence from that list was the real failure in the last check.

Last check's other failures test/traffic-truth.test.ts (shard 90) and test/snappy.test.ts (shard 86) also failed in the last check. They test cache headers on commit-pinned pages and response-time budgets for / and /map. Neither reaches anything this pull changes: the directory walk runs only on /system and the MCP system tool. Both passed twice in a row here, so I think the CI failures were flaky, but I haven't confirmed why they failed there.

Checks run here

  • Passing:

- bun scripts/gate.ts - bun x tsc --noEmit - bun scripts/design-lint.ts - test/mcp-size.test.ts - test/timers-named.test.ts - 44 test files on vitals, the pager, notify, capacity and /system, with 405 tests.

  • scripts/fast.ts called the diff too broad to pick tests from, so I chose the files above myself.
  • This sandbox's disk is 83% used, so those tests ran with a real disk past both marks.
  • The keep_plan tool wasn't available in this session, so the plan on #11800 was not updated.

Try it: sign in as an admin, open /system and read "Largest under the forge's data"; or call the MCP system tool and read largestDirs.

🤖 Generated with Claude Code

Changes requested — before the latest push

The disk marks, the ladder in incidents.ts and the cached async walk are sound and tested. Two list items are missed:

  1. Both doors (fix first): the five largest directories are on the page and the MCP tool, but there is no terminal screen. Add the list to the /system screen in src/tui/ with its entry where AGENTS.md says, reading from the same largestDirs cache and gated as the page is (admin or telemetry:read only), with a test.
  2. No docs/features/<name>.md how-to for the disk marks and the largest-directories list (do not put it in README.md).
  3. Design principle 1 (see the line comment): the footnote under the list explains the cache twice.

The issue title says each machine the forge uses; the boxes cover only the forge's own data disk. The pull says so, which is fine.

commented

The vitals now raise disk-70 and disk-80 when the disk holding the forge's data passes 70% and 80% used. The pager in src/crew/incidents.ts treats them as rungs of one ladder: 70%, 80%, the free-space floor, then nearly full. Climbing to a higher rung keeps the lower rungs open, so each mark reaches the owner's phone once. None of them recovers until the disk drops back under it.

src/system/largest_dirs.ts measures the five largest directories under the forge's data, sized as du -x would size them. The walk runs in the background, and one reading, at most a minute old, is shared by all three ways in:

  • /system: the "Largest under the forge's data" section. The page never waits for the walk.
  • The terminal: : → "System". The list sits below the telemetry history (largestLines in src/tui/screens/system.ts).
  • MCP: the system tool returns it as largestDirs. The tool waits at most 0.8 s for a new reading.

Only a forge admin, or an agent granted telemetry:read, sees the list.

This round, from review:

  • Terminal screen: added largestLines to the System screen. It reads the same cache as the page through peek, so opening the screen never waits for the walk. The new tests in test/tui-system.test.ts check that an admin and a granted agent see it, that a member, an ungranted agent and a guest get nothing, and that the screen draws the section.
  • How-to: added docs/features/disk-marks.md, which explains how the marks page the owner and where the list can be read.
  • Footnote: on /system the note under the list now reads only "Measured <time>".
  • AGENTS.md: the largestDirs cache is now listed with the other state modules share in memory, next to footing.

The marks cover only the forge's own data disk, not every machine the forge uses; the how-to says so.

To try it, open /system as an admin and read "Largest under the forge's data", or in the terminal press : and pick "System". Before pushing I ran scripts/gate.ts, tsc --noEmit, design-lint and the system, disk and timer test files, and all passed. fast.ts couldn't narrow the tests because src/kernel/vitals.ts changed, so the forge's own check will run the full suite.

🤖 Generated with Claude Code

commented

The owner's phone now gets a page when the disk under the forge's data passes 70% and again at 80%, and one "Recovered" when it falls back below 70%. /system, the MCP system tool and the System terminal screen also list the five largest directories under the forge's data (#11800, item 6 of #11686).

What moved where

  • Disk vitals: src/kernel/vitals.ts adds two disk vitals, at 70% and 80% used. src/crew/incidents.ts pages each crossing once and holds back the lower one while the higher one is paging. The existing disk-floor incident keeps its pager.
  • Largest directories: src/system/largest_dirs.ts is new. It measures the directories at most once a minute, and a page request waits at most 0.8 s for the result; the wait uses a named deadline timer. src/web/pages/system.ts, src/http/mcp/tools/system.ts and src/tui/screens/system.ts read from it. The terminal screen is admin-only, like /system.
  • Docs: docs/features/disk-marks.md is new; AGENTS.md gains one line.
  • Tests: test/disk-marks.test.ts is new and covers four cases: 69% stays silent, 70% pages once, 80% pages once more, and falling below 70% says Recovered. It also checks that the list is ordered by size. The other changed tests only add the new timer and section.

Why there is an empty commit The first check failed the gate's "pipeline first" rule: the pull adds a how-to under docs/features/ and changes no pipeline code. The last commit is empty and carries the Direction: line the rule asks for. Its reason: this is the P0 Trust item, and a full disk stops every check, nightly run and deploy the pipeline runs. If a reviewer would rather drop the how-to than waive the rule, deleting docs/features/disk-marks.md clears the rule too.

Not done No pipeline code is changed and no other disks are watched: the marks look only at the disk under the forge's data.

Checks run here bun scripts/gate.ts and bun x tsc --noEmit pass. The covering test files pass: disk-marks, disk-floor, system-parts, timers-named and tui-system, 53 tests in all. The full suite was not run locally; the forge runs it on this push.

Try it: open /system as an admin and read "Largest under the forge's data", or open the System screen in the terminal.

🤖 Generated with Claude Code

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs