/system and the system MCP tool show the database's ten largest tables and indexes, its file size and its growth #12025

/system now has a "Database size" section, and the MCP system tool takes kind: "size". Both show the database file's bytes, the WAL's bytes, the free pages inside the file, its growth per day, and one table with the ten largest tables (bytes, rows) and the ten largest indexes (bytes, the table each one serves). No new tool and no ceiling raised. docs/MCP.md's table was regenerated: the tool's row grew by 7 bytes.

How sizes are measured. Bun's SQLite (3.53) is built without dbstat and without sqlite_dbpage. Even where dbstat exists, it walks every page on the thread that asks. So src/system/db_size.ts reads the file itself:

• It reads in 4 MiB asynchronous chunks, so the event loop is never held, and records each b-tree page's kind, cells and children.
• It then walks each b-tree down from its root page in sqlite_schema. A table's bytes include its overflow pages, a rowid table's rows are its leaf cells (so no count(*) scans), and freed pages count for no table.
• On a seeded file the per-tree page counts match dbstat exactly (checked with Python's sqlite), and every page is one tree's or free.
• The file is read as of its last checkpoint, which the page says. One walk is shared by /system and the MCP tool, and it runs at most once an hour, only when someone asks.

Growth. One migration adds db_size_day: the file's first reading of each UTC day, written on the existing two-minute tick, with 60 days kept. Growth per day is the file now against the oldest reading of the last seven days, and nothing is shown until the readings span 20 hours.

Vitals need. It links to /system and fires in two cases:

• db-size: the file is past 0x7fff0000 bytes, the most Bun's SQLite maps.
• db-growth: the file grows faster than 64 MiB a day (an empty database reaches the map's cap in 32 days at that rate).

src/kernel/db.ts describes production's database as about 4.5 GB, so db-size will fire there as soon as this lands. That is what the issue asks for.

Two existing tests changed:

• test/vitals.test.ts: the vitals' query count goes from 10 to 11, because the daily readings are one more read (at most 60 rows).
• test/timers-named.test.ts: the cache's 0.8 s wait is added to the allowed timers.

Choices:

• The vitals use only the file's size and the readings, never the walk.
• owned() in src/crew/incidents.ts does not list db-size or db-growth, so neither need pages the phone.
• There is no terminal-screen line. If wanted, that is a small follow-up next to largestLines.

Checks run here:

• bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts pass.
• test/db-size.test.ts is new: it seeds a database and checks the order, the rows, the overflow math worked by hand, growth, the needs, the cache, the MCP gate and the escaping of the section.
• test/mcp-size.test.ts, the vitals*, system-, disk-, health*, schema-*, offsite-lane, timers-named and hot-reads-indexed suites pass (529 tests in one run).
• That run had one failure: test/tui-system.test.ts › "the System command passes the session". It passes alone, with and without this change, and this change doesn't touch the terminal screens.
• The app boots, / answers 200, and the tick writes today's reading.

Screenshot. I couldn't take the 1280 px screenshot the issue asks for. Chromium is installed, but it needs a unix socket, which the sandbox refuses. The section's rendered HTML was checked from a live boot instead; the look on this pull's preview should supply the picture.

Not done. The keep_plan and estimate tools the issue names were not available in this session, so no plan was kept on the issue and no time was promised.

Try: open /system on the preview as its owner and read "Database size", or call the MCP system tool with {"kind":"size"}.

🤖 Generated with Claude Code

Closes #12010

/system now has a "Database size" section, and the MCP system tool takes kind: "size". Both show the database file's bytes, the WAL's bytes, the free pages inside the file, its growth per day, and one table with the ten largest tables (bytes, rows) and the ten largest indexes (bytes, the table each one serves). No new tool and no ceiling raised. docs/MCP.md's table was regenerated: the tool's row grew by 7 bytes.

How sizes are measured. Bun's SQLite (3.53) is built without dbstat and without sqlite_dbpage. Even where dbstat exists, it walks every page on the thread that asks. So src/system/db_size.ts reads the file itself:

• It reads in 4 MiB asynchronous chunks, so the event loop is never held, and records each b-tree page's kind, cells and children.
• It then walks each b-tree down from its root page in sqlite_schema. A table's bytes include its overflow pages, a rowid table's rows are its leaf cells (so no count(*) scans), and freed pages count for no table.
• On a seeded file the per-tree page counts match dbstat exactly (checked with Python's sqlite), and every page is one tree's or free.
• The file is read as of its last checkpoint, which the page says. One walk is shared by /system and the MCP tool, and it runs at most once an hour, only when someone asks.

Growth. One migration adds db_size_day: the file's first reading of each UTC day, written on the existing two-minute tick, with 60 days kept. Growth per day is the file now against the oldest reading of the last seven days, and nothing is shown until the readings span 20 hours.

Vitals need. It links to /system and fires in two cases:

• db-size: the file is past 0x7fff0000 bytes, the most Bun's SQLite maps.
• db-growth: the file grows faster than 64 MiB a day (an empty database reaches the map's cap in 32 days at that rate).

src/kernel/db.ts describes production's database as about 4.5 GB, so db-size will fire there as soon as this lands. That is what the issue asks for.

Two existing tests changed:

• test/vitals.test.ts: the vitals' query count goes from 10 to 11, because the daily readings are one more read (at most 60 rows).
• test/timers-named.test.ts: the cache's 0.8 s wait is added to the allowed timers.

Choices:

• The vitals use only the file's size and the readings, never the walk.
• owned() in src/crew/incidents.ts does not list db-size or db-growth, so neither need pages the phone.
• There is no terminal-screen line. If wanted, that is a small follow-up next to largestLines.

Checks run here:

• bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts pass.
• test/db-size.test.ts is new: it seeds a database and checks the order, the rows, the overflow math worked by hand, growth, the needs, the cache, the MCP gate and the escaping of the section.
• test/mcp-size.test.ts, the vitals*, system-, disk-, health*, schema-*, offsite-lane, timers-named and hot-reads-indexed suites pass (529 tests in one run).
• That run had one failure: test/tui-system.test.ts › "the System command passes the session". It passes alone, with and without this change, and this change doesn't touch the terminal screens.
• The app boots, / answers 200, and the tick writes today's reading.

Screenshot. I couldn't take the 1280 px screenshot the issue asks for. Chromium is installed, but it needs a unix socket, which the sandbox refuses. The section's rendered HTML was checked from a live boot instead; the look on this pull's preview should supply the picture.

Not done. The keep_plan and estimate tools the issue names were not available in this session, so no plan was kept on the issue and no time was promised.

Try: open /system on the preview as its owner and read "Database size", or call the MCP system tool with {"kind":"size"}.

🤖 Generated with Claude Code

/system now has a "Database size" section, and the MCP system tool takes kind: "size". Both show the database file's bytes, the WAL's bytes, the free pages inside the file, its growth per day, and one table with the ten largest tables (bytes, rows) and the ten largest indexes (bytes, the table each one serves). No new tool and no ceiling raised. docs/MCP.md's table was regenerated: the tool's row grew by 7 bytes.

How sizes are measured. Bun's SQLite (3.53) is built without dbstat and without sqlite_dbpage. Even where dbstat exists, it walks every page on the thread that asks. So src/system/db_size.ts reads the file itself:

• It reads in 4 MiB asynchronous chunks, so the event loop is never held, and records each b-tree page's kind, cells and children.
• It then walks each b-tree down from its root page in sqlite_schema. A table's bytes include its overflow pages, a rowid table's rows are its leaf cells (so no count(*) scans), and freed pages count for no table.
• On a seeded file the per-tree page counts match dbstat exactly (checked with Python's sqlite), and every page is one tree's or free.
• The file is read as of its last checkpoint, which the page says. One walk is shared by /system and the MCP tool, and it runs at most once an hour, only when someone asks.

Growth. One migration adds db_size_day: the file's first reading of each UTC day, written on the existing two-minute tick, with 60 days kept. Growth per day is the file now against the oldest reading of the last seven days, and nothing is shown until the readings span 20 hours.

Vitals need. It links to /system and fires in two cases:

• db-size: the file is past 0x7fff0000 bytes, the most Bun's SQLite maps.
• db-growth: the file grows faster than 64 MiB a day (an empty database reaches the map's cap in 32 days at that rate).

src/kernel/db.ts describes production's database as about 4.5 GB, so db-size will fire there as soon as this lands. That is what the issue asks for.

Two existing tests changed:

• test/vitals.test.ts: the vitals' query count goes from 10 to 11, because the daily readings are one more read (at most 60 rows).
• test/timers-named.test.ts: the cache's 0.8 s wait is added to the allowed timers.

Choices:

• The vitals use only the file's size and the readings, never the walk.
• owned() in src/crew/incidents.ts does not list db-size or db-growth, so neither need pages the phone.
• There is no terminal-screen line. If wanted, that is a small follow-up next to largestLines.
• The pull carries a Direction: line for the gate's "pipeline first" rule: it serves #11973, ahead of the pipeline items.

Checks run here:

• bun scripts/gate.ts against the merge base, bun x tsc --noEmit and bun scripts/design-lint.ts pass.
• test/db-size.test.ts is new: it seeds a database and checks the order, the rows, the overflow math worked by hand, growth, the needs, the cache, the MCP gate and the escaping of the section. It, vitals, mcp-size, system-parts and timers-named pass after the fixes for the failed check.
• Before those fixes, the vitals*, system-, disk-, health*, schema-*, offsite-lane and hot-reads-indexed suites also passed (529 tests in one run).
• That run had one failure: test/tui-system.test.ts › "the System command passes the session". It passes alone, with and without this change, and this change doesn't touch the terminal screens.
• The app boots, / answers 200, and the tick writes today's reading.

Screenshot. I couldn't take the 1280 px screenshot the issue asks for. Chromium is installed, but it needs a unix socket, which the sandbox refuses. The section's rendered HTML was checked from a live boot instead; the look on this pull's preview should supply the picture.

Not done. The keep_plan and estimate tools the issue names were not available in this session, so no plan was kept on the issue and no time was promised.

Try: open /system on the preview as its owner and read "Database size", or call the MCP system tool with {"kind":"size"}.

🤖 Generated with Claude Code

/system now has a Database size section, and the MCP system tool returns the same data with kind: "size". Both show the size of forge.db and its WAL, the free space inside the file, how much it grows each day, and the ten largest tables (bytes, rows) and ten largest indexes (bytes, which table each one serves). A badge appears when the file goes past DB_FILE_CAP or grows more than DB_GROWTH_MAX a day.

The page never waits for the measurement. The table and index sizes come from a scan of the file that runs at most once an hour, and a page view only starts a new scan in the background when the last one is older than that. Until the first scan finishes, the section says so in one line. A migration (20261114000200-db-size-day, pinned) stores one reading a day so growth has history to work from. The code is in src/system/db_size.ts, the section in src/web/pages/system.ts, and notes in docs/features/db-size.md.

Raised budget: /system's first-view block budget in docs/design/attention-budget.json goes from 67 to 70. The three extra blocks are the new section's heading, its one row of figures, and its one-line note while the first scan runs. The table itself sits below the first view. The reason is recorded in the budget file's note for system.

test/vitals.test.ts is off scripts/header-width-allow.json now that its header comment is wrapped.

bun scripts/gate.ts and bun x tsc --noEmit pass, as do test/attention.test.ts and test/header-width.test.ts. The last check also failed:

• test/pages-budget.test.ts and test/flake.test.ts: the forge found these red on main too and did not count them against this pull.
• test/spawn-stopped-restart.test.ts, the MCP system stalls test and the repeat-search test: I did not run these here.

To try it, open /system on the preview and scroll to Database size, or call the MCP system tool with {"kind": "size"}.

🤖 Generated with Claude Code

The database's size (the file, its WAL, the free space inside it, its growth a day, and its ten largest tables and indexes) now shows on /system, in the terminal's System screen, and in the MCP system tool with kind: "size".

Direction: #11973

All three read one shared measurement in src/system/db_size.ts, using the same telemetryReader gate. A request never waits for the measurement, apart from the MCP tool, which waits up to 0.8 s.

I/O cost. A measurement reads the whole forge.db once. On the production file that is about 4.5 GB of reads. It runs at most once an hour, and only in an hour when someone opens /system or the terminal's System screen, or calls the tool. The reads are asynchronous 4 MiB chunks, so the event loop is never held. A measurement that fails is logged as database size walk failed and tried again an hour later. Before this it failed silently.

This round, from review:

• Terminal: the System dialog gets one line under the largest directories: the file's size, its growth a day, and "d: its largest tables". d opens a scrolling Database size screen with everything else. Tests cover the gate, the short and full forms, and opening the screen with d. docs/features/db-size.md describes it.
• Facts as dl rows: file, WAL, free space and growth are now status-rows instead of pills. Index rows have no — filler: the name cell spans two columns, and the columns are Name, Rows, Bytes.
• Text said once: the footer is now "Measured …, at most once an hour." The waiting line is "Its tables are being measured; reload in a moment."
• The attention budget is raised 67 → 69, not reverted. /system was already exactly at 67 without this section, and dl rows cost more than pills: each dt and dd counts as a block, so the four rows measured 75. The section now follows the off-site backup section on the same page (#11805): the heading and one sentence stay on first view ("forge.db is X; grows Y a day", plus a badge when it's worth a look). The fact rows and the table sit behind a disclosure. Two blocks is the smallest a section with a heading can cost; the reason is in docs/design/attention-budget.json.

Still missing: the 1280 px screenshot. My sandbox has no browser, so I can't take one; it needs to come from the preview at https://lab--12025--forge-trevato-apps.open-platform.sh/system.

Checks: gate, tsc and design-lint pass, and so do the tests covering this change: db-size, tui-system, tui-screens, attention, header-width, timers-named, system-parts, the MCP system tests and the design tests. I couldn't run the whole suite here, because scripts/fast.ts reports the diff as too broad to narrow down.

To try it: open /system as an admin and expand Database size. In the terminal, open System and press d. Over MCP, call system with {"kind": "size"}.

🤖 Generated with Claude Code

The database's size (the file, its WAL, the free space inside it, its growth a day, and its ten largest tables and indexes) now shows on /system, in the terminal's System screen, and in the MCP system tool with kind: "size".

Direction: #11973

All three read one shared measurement in src/system/db_size.ts, using the same telemetryReader gate. A request never waits for the measurement, apart from the MCP tool, which waits up to 0.8 s.

I/O cost. A measurement reads the whole forge.db once. On the production file that is about 4.5 GB of reads. It runs at most once an hour, and only in an hour when someone opens /system or the terminal's System screen, or calls the tool. The reads are asynchronous 4 MiB chunks, so the event loop is never held. A measurement that fails is logged as database size walk failed and tried again an hour later.

Page. The heading and one sentence stay on first view: "forge.db is X; grows Y a day", with a badge when either is worth a look. Behind a disclosure are the facts as dl rows (file, WAL, free space inside, growth). Below them is one table: tables with rows and bytes, then indexes with the table each one serves and bytes. Index rows have no filler cells.

Terminal. The System dialog gets one line under the largest directories: the file's size, its growth a day, and "d: its largest tables". d opens a scrolling Database size screen with everything else. The dialog doesn't scroll, so it gets only that one line.

Attention budget. /system's first-view block budget goes from 67 to 69. The page was already at exactly 67 before this section, and a heading plus one sentence is the least a section can add. The off-site backup section on the same page does the same (#11805). The reason is recorded in docs/design/attention-budget.json.

Also: test/vitals.test.ts is off the header-width allow list, now that its header comment is wrapped. The last fix only rewraps two comments in src/tui/screens/system.ts to 100 columns, the line limit the gate checks.

Checks: gate, tsc and design-lint pass, and so do the tests covering this change: db-size, tui-system, tui-screens, attention, header-width, timers-named, system-parts, the MCP system tests and the design tests. I couldn't run the whole suite here, because scripts/fast.ts reports the diff as too broad to narrow down.

Still missing: the 1280 px screenshot of /system. My sandbox has no browser, so I can't take one; it needs to come from the preview at https://lab--12025--forge-trevato-apps.open-platform.sh/system.

To try it: open /system as an admin and expand Database size. In the terminal, open System and press d. Over MCP, call system with {"kind": "size"}.

🤖 Generated with Claude Code