machine_sample keeps 48 h raw, then 5-minute rows for 30 days, then hourly rows #12028

machine_sample now keeps 15-s points for 48 h, 5-minute rows for 30 days, and one hourly row per target after that. A new module, src/system/retention_samples.ts, holds this rule, and the cleanup runs in small batches instead of inside every 15-s write.

What moved where

• src/system/retention_samples.ts (new) defines the time limits once: RAW_KEEP_MS (48 h), FIVE_MIN_KEEP_MS (30 days), HOUR_MS, and a row cap ROW_CAP of 5,000.

- pruneSamples(db, now, rowCap) does one batch in a transaction that won't wait on another writer, and returns {rolled, deleted, more}. - Each batch first deletes 15-s points older than 48 h, but only once their 5-minute window has been rolled up. - It then folds 5-minute rows older than 30 days into hourly rows: the mean of the means and the max of the maxes. It always takes whole hours, so a small cap never splits an hour across two batches. - drainSamples calls it repeatedly until more is false, letting the server handle other work between batches.

• src/boards/ring.ts only writes and rolls up now. Its two DELETEs and its own keep-time constants are gone.
• src/boards/sampler.ts: each 15-s sampler tick starts a drain, unless one is already running. This is the periodic job that already owned the table.
• src/boards/sources.ts: the machine board source also reads hourly rows, only for times before the oldest 5-minute row, so no moment is counted twice. The step and at descriptions mention hourly rows.
• Migration 20261114010000-machine-sample-hourly rebuilds the table so the step CHECK also allows 3600, keeping the same columns and rows. I added it to REBUILT in test/schema-additive.test.ts, the same way earlier CHECK widenings were handled.

Readers. telemetry.ts's historyOver (at most 72 h, using only 15-s and 5-minute rows) and efficiency/readings.ts (the last hour of 15-s points) are unchanged in code and give the same answers. A test seeds 3 days of samples and checks the 1 h, 24 h and 72 h history and cpuSecondsLastHour before and after a full drain.

Tests I changed, and why. Three ring tests in test/boards-sampler.test.ts checked the old rule: record deleting at 24 h and 30 days. The issue replaces that rule, so they now prune after recording and check 48 h, plus the hourly rows past 30 days.

New tests in test/retention-samples.test.ts:

• One batch over 50,400 seeded old rows takes under 50 ms and touches at most the cap.
• A 15-s point whose 5-minute window hasn't been rolled up is kept.
• Draining with a cap of 7 gives exactly the same rows as a cap of 5,000.
• The hourly means and maxes are exact.
• The table still rejects unknown steps.
• The board source doesn't double-count a moment covered by both an hourly and a 5-minute row.

A wiring test in test/boards-sampler.test.ts fails if the sampler tick stops pruning.

Results. bun scripts/gate.ts and bun x tsc --noEmit pass. The migration makes the diff too broad for scripts/fast.ts to pick tests, so I ran the covering files by hand: schema/migrate, boards-*, retention, telemetry, efficiency, write-clock, hot-reads and tui-system. One test failed: tui-system "the System command passes the session…". It fails the same way on main in that same combination of files and passes on its own, so this change didn't cause it.

Choices to look at

• Hourly rows are kept with no end date, as the issue says, so the table still grows slowly. Rows for short-lived check targets are now kept as hourly rows instead of being deleted after 30 days. If the table needs a hard limit, that's a follow-up.
• On first boot the migration copies the whole table, about 2.0M rows, in one go.
• There's no backlog to clear: the old code already deleted everything past 24 h and 30 days. The first drains only remove what ages out from then on.

Try: on the preview, call the MCP tool telemetry_history with { "hours": 72 }, and open a board over the machine source. Both should answer as before.

I didn't keep a plan on the issue with keep_plan, because that tool isn't available in this session.

🤖 Generated with Claude Code

Closes #12007

machine_sample now keeps 15-s points for 48 h, 5-minute rows for 30 days, and one hourly row per target after that. A new module, src/system/retention_samples.ts, holds this rule, and the cleanup runs in small batches instead of inside every 15-s write.

What moved where

• src/system/retention_samples.ts (new) defines the time limits once: RAW_KEEP_MS (48 h), FIVE_MIN_KEEP_MS (30 days), HOUR_MS, and a row cap ROW_CAP of 5,000.

- pruneSamples(db, now, rowCap) does one batch in a transaction that won't wait on another writer, and returns {rolled, deleted, more}. - Each batch first deletes 15-s points older than 48 h, but only once their 5-minute window has been rolled up. - It then folds 5-minute rows older than 30 days into hourly rows: the mean of the means and the max of the maxes. It always takes whole hours, so a small cap never splits an hour across two batches. - drainSamples calls it repeatedly until more is false, letting the server handle other work between batches.

• src/boards/ring.ts only writes and rolls up now. Its two DELETEs and its own keep-time constants are gone.
• src/boards/sampler.ts: each 15-s sampler tick starts a drain, unless one is already running. This is the periodic job that already owned the table.
• src/boards/sources.ts: the machine board source also reads hourly rows, only for times before the oldest 5-minute row, so no moment is counted twice. The step and at descriptions mention hourly rows.
• Migration 20261114010000-machine-sample-hourly rebuilds the table so the step CHECK also allows 3600, keeping the same columns and rows. I added it to REBUILT in test/schema-additive.test.ts, the same way earlier CHECK widenings were handled.

Readers. telemetry.ts's historyOver (at most 72 h, using only 15-s and 5-minute rows) and efficiency/readings.ts (the last hour of 15-s points) are unchanged in code and give the same answers. A test seeds 3 days of samples and checks the 1 h, 24 h and 72 h history and cpuSecondsLastHour before and after a full drain.

Tests I changed, and why. Three ring tests in test/boards-sampler.test.ts checked the old rule: record deleting at 24 h and 30 days. The issue replaces that rule, so they now prune after recording and check 48 h, plus the hourly rows past 30 days.

New tests in test/retention-samples.test.ts:

• One batch over 50,400 seeded old rows takes under 50 ms and touches at most the cap.
• A 15-s point whose 5-minute window hasn't been rolled up is kept.
• Draining with a cap of 7 gives exactly the same rows as a cap of 5,000.
• The hourly means and maxes are exact.
• The table still rejects unknown steps.
• The board source doesn't double-count a moment covered by both an hourly and a 5-minute row.

A wiring test in test/boards-sampler.test.ts fails if the sampler tick stops pruning.

Results. bun scripts/gate.ts and bun x tsc --noEmit pass. The migration makes the diff too broad for scripts/fast.ts to pick tests, so I ran the covering files by hand: schema/migrate, boards-*, retention, telemetry, efficiency, write-clock, hot-reads and tui-system. One test failed: tui-system "the System command passes the session…". It fails the same way on main in that same combination of files and passes on its own, so this change didn't cause it.

Choices to look at

• Hourly rows are kept with no end date, as the issue says, so the table still grows slowly. Rows for short-lived check targets are now kept as hourly rows instead of being deleted after 30 days. If the table needs a hard limit, that's a follow-up.
• On first boot the migration copies the whole table, about 2.0M rows, in one go.
• There's no backlog to clear: the old code already deleted everything past 24 h and 30 days. The first drains only remove what ages out from then on.

Try: on the preview, call the MCP tool telemetry_history with { "hours": 72 }, and open a board over the machine source. Both should answer as before.

I didn't keep a plan on the issue with keep_plan, because that tool isn't available in this session.

🤖 Generated with Claude Code

The sampler now trims old machine samples in three tiers: raw samples are kept for 48 h, 5-minute rollups to 30 days, and hourly rows after that. Old rows are removed a bounded batch at a time on each sampler tick. The "machine source" — the code that reads this history — now reads the hourly rows. The machine_sample table's step CHECK widens to 3600 so it accepts hourly rows. The rule is defined once, in src/system/retention_samples.ts. The ring only writes and rolls up; deleting old rows is left to the sampler.

This update merges main, which had changed the same lines in src/boards/sampler.ts (#12026). That change makes each sampler tick also delete one batch of old event rows. I kept both: each tick first deletes one batch of old events, skipping it if the database is busy. It then starts the background sample cleanup, unless the previous one is still running. The event delete goes first so that check can't skip it.

bun scripts/gate.ts, bun x tsc --noEmit, test/boards-sampler.test.ts, test/retention-events.test.ts and test/retention-samples.test.ts pass (30 tests). To try it, look at the machine board on the preview: recent history is raw samples and older history is rolled up.

🤖 Generated with Claude Code

The sampler now trims old machine samples in three tiers: raw samples are kept for 48 h, 5-minute rollups to 30 days, and hourly rows after that. Old rows are removed a bounded batch at a time on each sampler tick. The "machine source" — the code that reads this history — now reads the hourly rows. The machine_sample table's step CHECK widens to 3600 so it accepts hourly rows. The rule is defined once, in src/system/retention_samples.ts. The ring only writes and rolls up; deleting old rows is left to the sampler. The merge with main (#12026) keeps both clean-ups on each tick in src/boards/sampler.ts: first one batch of old event rows, then the sample clean-up.

This update fixes the two check failures this pull caused:

• test/timers-named.test.ts: drainSamples paused between batches with a bare setTimeout(resolve, 0), which that test rejects. It now pauses with setImmediate, as src/maintain/runner.ts and src/http/stranger_code.ts already do.
• test/flake.test.ts: the batch test in test/retention-samples.test.ts checked that one batch took under 50 ms, a timing limit the repository forbids in tests. It now counts the SQL statements one batch runs (once and expectWork), at most 6, and still checks that a batch deletes no more than ROW_CAP rows. The test was wrong, not the code.

Three other check failures don't come from this pull:

• test/pages-budget.test.ts and the spawn-stopped-restart case in test/flake.test.ts are red on main at f53d2cf too.
• test/search-production-size.test.ts ran out of memory in its shard. Nothing in this pull touches search.
• test/traffic-history.test.ts ("the page offers the three windows…") passed in all three runs here. It counts the forge's own requests, not samples, so it looks like interference from other tests in its shard.

Gate, tsc --noEmit, and the retention-samples, timers-named, boards-sampler and traffic-history tests pass. The only failure in test/flake.test.ts is the spawn-stopped-restart case that is also red on main.

To try it, look at the machine board on the preview: recent history is raw samples and older history is rolled up.

🤖 Generated with Claude Code