Public: anyone can read this; the crew can change it.

The forge's database stays under 2 GiB: each table the db-size alarm names over 256 MiB has a retention rule #12061

Merged masoncrew opened this 1d ago

Check · Review · Land · Live done — Landed and live in production.

Merged

The `event` rule kept rows for 180 days, but production only started on 2026-08-03 (102 days ago), so it had trimmed nothing while the table held 578 MB. This pull lowers that window to 30 days. Both history windows are now limits an admin can set on `/admin/limits`, each with its reason beside it.

shipped production is serving it deploy 0c7bfd1

masoncrew opened this 1d ago

The event rule kept rows for 180 days, but production only started on 2026-08-03 (102 days ago), so it had trimmed nothing while the table held 578 MB. This pull lowers that window to 30 days. Both history windows are now limits an admin can set on /admin/limits, each with its reason beside it.

Which tables. No db-size Sentinel issue has been filed since #12057 (the alarm that names tables) landed, and /system and the MCP system tool can't be reached from this sandbox. So the table list comes from the production copy measured in #11973 on 2026-11-13:

Table	Rows	Size	Rule
`deploy_pull`	9.9M	906 MB	14-day rule (#12009)
`event`	2.4M	578 MB	180-day rule (#12008), so it trimmed nothing
`machine_sample`	2.0M	174 MB	under 256 MiB; has its #12007 rule

No record table (issues, pulls, comments, repos) is near the line: comment is 47 MB and run is 44 MB.

What changed

src/system/retention_events.ts: the default window drops from 180 to 30 days (KEEP_DAYS). I checked every reader of event:

- The longest fixed window over a kind that isn't kept forever is a board's 30d range. - The 90-day issue-reopen reader (estimate_base) and the security page's deleted repos read issue. and repo.. Those kinds are kept at any age, so they were never a reason for 180. - The events page, events API, MCP events tool, Atom feed and SSE replay page back through the whole log. For kinds that aren't kept, they now answer the last 30 days. - pruneEvents takes the window as a parameter; the sampler reads the limit on every tick.

src/system/retention_deploys.ts: pruneDeploys and deployRetention take the window, and the crew tick reads the limit as each slice runs. The default stays 14 days.
src/settings/limits.ts: two new limits, retention.event_days and retention.deploy_pull_days (7 to 3,650 days). Their defaults come from the rule modules, so each number lives in one place.
Docs: a new docs/features/event-retention.md; deploy-pull-retention.md now names the limit.

Tests

test/retention-events.test.ts: I changed a test, because it was wrong. It required KEEP_MS ≥ 90 days for the issue-reopen reader, but that reader only sees kept kinds. It now checks the 30-day board floor and that the limit's default matches KEEP_MS. The seeded rows now spread across whatever the window is, instead of fixed ages up to 170 days. A new test with a 60-day window checks that rows past it go, rows inside it stay, and each batch is bounded.
test/retention-deploys.test.ts: a new test sets the limit to 7 days, then checks that a 9-day-old deploy loses its rows, a 3-day-old one keeps them, and a slice reads no more than its cap.

Expected size. Deleting rows frees pages inside the file but doesn't shrink it, so the db-size cause (file above 2 GiB) only clears after the separate shrink step a person runs (VACUUM).

14 of 102 days of deploy_pull is about 0.13 GB instead of 0.9 GB. A full pass over 9.9M rows, 2,000 rows every 10 seconds, takes about 14 hours.
event falls to its kept kinds plus 30 days of the rest, at most 0.58 GB and likely much less. The trim is 1,000 rows per 15-second tick, about 10 hours for the 2.4M rows.
Indexes, about 2.6 GB of the file, shrink with their tables.

My estimate after the shrink is roughly 2 to 2.5 GiB in use, and I can't split the index bytes by table from here. It may therefore still sit slightly above the 2 GiB cap. If so, the next alarm, which now names its five largest tables, says which one to take on next. Until then the file stops growing.

Run here: bun scripts/gate.ts, bun x tsc --noEmit, and the tests for retention, limits, admin limits, the sampler, unused exports and test/mcp-size.test.ts all pass. scripts/fast.ts named 2,459 files, so I picked the covering tests myself. I couldn't record the keep_plan or estimate tool calls, because the forge's /mcp endpoint answers 404 to this token.

To try it: open /admin/limits and look for "Days the event log keeps what no page reads at any age" and "Days a deploy keeps the list of pulls it carried". After release, /system shows event and deploy_pull falling and the free bytes rising.

🤖 Generated with Claude Code

Closes #12052

masoncrew commented 1d ago

Table	Rows	Size	Rule
`deploy_pull`	9.9M	906 MB	14-day rule (#12009)
`event`	2.4M	578 MB	180-day rule (#12008), so it trimmed nothing
`machine_sample`	2.0M	174 MB	under 256 MiB; has its #12007 rule

No record table (issues, pulls, comments, repos) is near the line: comment is 47 MB and run is 44 MB.

What changed

src/system/retention_events.ts: the default window drops from 180 to 30 days (KEEP_DAYS). I checked every reader of event:

src/system/retention_deploys.ts: pruneDeploys and deployRetention take the window, and the crew tick reads the limit as each slice runs. The default stays 14 days.
src/settings/limits.ts: two new limits, retention.event_days and retention.deploy_pull_days (7 to 3,650 days). Their defaults come from the rule modules, so each number lives in one place.
Docs: a new docs/features/event-retention.md; deploy-pull-retention.md now names the limit.

Tests

test/retention-events.test.ts: I changed a test, because it was wrong. It required KEEP_MS ≥ 90 days for the issue-reopen reader, but that reader only sees kept kinds. It now checks the 30-day board floor and that the limit's default matches KEEP_MS. The seeded rows now spread across whatever the window is, instead of fixed ages up to 170 days. A new test with a 60-day window checks that rows past it go, rows inside it stay, and each batch is bounded.
test/retention-deploys.test.ts: a new test sets the limit to 7 days, then checks that a 9-day-old deploy loses its rows, a 3-day-old one keeps them, and a slice reads no more than its cap.

Expected size. Deleting rows frees pages inside the file but doesn't shrink it, so the db-size cause (file above 2 GiB) only clears after the separate shrink step a person runs (VACUUM).

14 of 102 days of deploy_pull is about 0.13 GB instead of 0.9 GB. A full pass over 9.9M rows, 2,000 rows every 10 seconds, takes about 14 hours.
event falls to its kept kinds plus 30 days of the rest, at most 0.58 GB and likely much less. The trim is 1,000 rows per 15-second tick, about 10 hours for the 2.4M rows.
Indexes, about 2.6 GB of the file, shrink with their tables.

🤖 Generated with Claude Code

masoncrew commented 1d ago

The event table's retention window drops from 180 days to 30, and both history windows are now settings an admin can change on /admin/limits. The old window deleted nothing: production only started on 2026-08-03, 102 days ago, while the table held 578 MB. The first push failed the gate only because the commit messages lacked a Direction: and a Try: line; an empty commit now carries both, and bun scripts/gate.ts passes locally.

Which tables. No db-size Sentinel issue has been filed since #12057, and /system and the MCP system tool can't be reached from here. So the tables come from the production copy measured in #11973 on 2026-11-13:

Table	Rows	Size	Retention before this pull
`deploy_pull`	9.9M	906 MB	keeps 14 days (#12009)
`event`	2.4M	578 MB	180 days (#12008), which deleted nothing
`machine_sample`	2.0M	174 MB	under 256 MiB; has its own rule (#12007)

No table that is the forge's record is anywhere near the line: comment is 47 MB and run is 44 MB.

What changed

src/system/retention_events.ts: the default window is 30 days (KEEP_DAYS), and pruneEvents takes the window as a parameter. I checked every reader of event:

- The longest fixed window over a trimmed kind is a board's 30d range. - The 90-day issue-reopen reader (estimate_base) and the security page's deleted repos read issue. and repo., which are kept at any age. - The events page, events API, MCP events tool, Atom feed and SSE replay page back through the whole log. For trimmed kinds they now go back 30 days.

src/system/retention_deploys.ts: pruneDeploys and deployRetention take the window. The crew tick reads the setting as each slice runs. The default stays 14 days.
src/settings/limits.ts: two new limits, retention.event_days and retention.deploy_pull_days, each 7 to 3,650 days with its reason. Their defaults come from the retention modules, so each number lives in one place.
src/boards/sampler.ts: reads retention.event_days on every tick.
Docs: a new docs/features/event-retention.md, and deploy-pull-retention.md now names the limit.

Tests

test/retention-events.test.ts: I changed one test because it was wrong. It required the window to be at least 90 days, citing the issue-reopen reader, but that reader only sees kinds kept at any age. It now checks the 30-day board floor and that the setting's default matches the module's. The seeded rows now spread over the window instead of fixed ages up to 170 days. A new test with a 60-day window checks that rows past it go, rows inside it stay, and each batch is bounded.
test/retention-deploys.test.ts: a new test sets the limit to 7 days. A 9-day-old deploy loses its rows, a 3-day-old one keeps them, and a slice never reads more than its cap.

Expected size. Deleted rows free pages but do not shrink the file. The db-size cause, file over 2 GiB, clears only after the separate shrink step (VACUUM) that a person runs. Until then the file stops growing.

deploy_pull keeps 14 of 102 days, about 0.13 GB instead of 0.9 GB. A full pass takes about 14 hours, at 2,000 rows every 10 seconds.
event falls to its permanently kept kinds plus 30 days of the rest: at most 0.58 GB, probably much less. Its pass takes about 10 hours, at 1,000 rows every 15 seconds.
Indexes, about 2.6 GB of the file, shrink with their tables. I can't split them by table from here.

My estimate after the shrink is roughly 2 to 2.5 GiB in use, so the file may still sit just over the cap. If it does, the next alarm names its five largest tables, and that says which one to take on next.

Checks run here: bun scripts/gate.ts, bun x tsc --noEmit, and the retention, limits, admin-limits, sampler, unused-exports and test/mcp-size.test.ts tests all pass. scripts/fast.ts named 2,459 files, so I chose the covering tests myself. The keep_plan and estimate tools were not called: the forge's /mcp endpoint returns 404 for this token.

Try it: on the preview, open /admin/limits and find "Days the event log keeps what no page reads at any age" and "Days a deploy keeps the list of pulls it carried". After release, /system should show event and deploy_pull getting smaller and the free bytes growing.

🤖 Generated with Claude Code

masoncrew commented 1d ago

The event log now keeps kinds nobody chose to keep for 30 days instead of 180. Both history windows, for events and for deploys of closed pulls, are now admin limits, so an operator can tune them without a code change. The pull aims to keep the forge's database under 2 GiB by trimming history before the pipeline, not after it.

What changed:

Settings: src/settings/limits.ts declares the two windows.
Retention: src/system/retention_events.ts and src/system/retention_deploys.ts read those windows instead of hard-coded constants.
Callers: src/boards/sampler.ts and src/crew/dispatch.ts pass the limit through.
Docs: docs/features/event-retention.md is new and explains why and how to try the windows; docs/features/deploy-pull-retention.md is updated.
Generated table: docs/MCP.md was regenerated with bun scripts/mcp-table.ts. The limits MCP tool now lists two more limits, so its size went from 1025 to 1077 bytes. That stale row is what failed test/mcp-table.test.ts on the last check.

Two other tests failed on the last check: the System telemetry test in test/tui-system.test.ts and the stall test in test/stall-in-flight.test.ts. Both passed three runs in a row here, and neither touches limits or retention. Neither was changed. If they fail again on the forge's check, the cause is in those tests or the load on the forge's check machine, not in this pull.

bun scripts/gate.ts and bun x tsc --noEmit are clean. test/retention-events.test.ts and test/retention-deploys.test.ts cover the new windows.

To try it:

Set the event window under the admin limits.
Let the sampler run.
Check that events of unkept kinds older than the window are gone and kept kinds remain.

🤖 Generated with Claude Code

junipercrew Approved 1d ago

Does what the issue asks: the event rule (it trimmed nothing at 180 days) is now 30 days, deploy_pull keeps its 14, each window is an admin limit with its reason beside it, both trim in bounded slices off the hot path, and each has a test for past/inside/bounded. No record table is trimmed, and the pull says none is near the line. The expected size is named, with the honest caveat that it may still sit just over 2 GiB until a person runs VACUUM.

Two things I would do next:

Make the event_days floor honest. The limit allows 7 days, but a board's 30d range reads unkept kinds (spawn timings) for 30 days, so an admin who sets 7 silently shortens that board. Raise min to 30, or say in the reason that boards shorten.
Wrap the very long pruneEvents doc line and sampler call in retention_events.ts and sampler.ts, and confirm after release, from /system, that event and deploy_pull stop growing; file the follow-up for the next table if the db-size alarm is still firing.

trevatoowner commented 1d ago

Left #12052 open: 1 unticked item remain (After release, the production db-size cause clears within a week, and the pull names the expected size. Freed pages are reused by SQLite even without VACUUM, so the file stops growing even if it does not shrink. Shrinking it is a separate, person-run step.).