Public: anyone can read this; the crew can change it.

A nightly backup that doubles in size or duration within a week files one defect with both numbers #13055

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A new backup-growth duty files one defect when the last nightly backup is over twice the median of the seven before it, in bytes or in seconds. tsc, scripts/gate.ts, design-lint and the backup and maintenance tests I ran are green, including the new test/backup-growth.test.ts. I did not run the full suite.

Notes

shipped production is serving it deploy 59903fa

opened this

A new backup-growth duty files one defect when the last nightly backup is over twice the median of the seven before it, in bytes or in seconds. tsc, scripts/gate.ts, design-lint and the backup and maintenance tests I ran are green, including the new test/backup-growth.test.ts. I did not run the full suite. scripts/fast.ts could not narrow the diff (2148 of 3309 files), and I stopped the run that started from that list.

What changed

  • The record: src/snapshot/outcome.ts (the existing snapshot/state.json under the data directory) now keeps up to 14 landed nights, each with bytes (what a restore writes) and seconds (start to finish). It also keeps the destination (the store's where, never a secret). A second landing on the same UTC day replaces the first. src/snapshot/service.ts writes these when a snapshot lands.
  • The reading: src/snapshot/growth.ts is a pure function of the nights. It compares the last night with the median of the seven before it and says which numbers are over twice. It gives no reading until three nights are recorded.
  • The duty: src/maintain/duties/backup_growth.ts runs hourly and is registered in registry.ts. It files one issue in the forge's own repository (ownRepoPath()) with a fixed title through fileIssueOnce, so there is one defect, not one per night. The body gives size and duration each against its median, and the destination.
  • No repeats: while the defect is open it is not filed again. A defect a person closed is not refiled for that same night, only for a later one.
  • src/system/db_size.ts: sizeSaid is now exported so the defect uses the same size wording as the rest of the forge.

What I chose not to do

  • A backup that stopped landing is not this duty's alarm: a last night older than two days files nothing.
  • No UI beyond the issue and the duty's usual rows on /maintenance.
  • No new dependency.
  • Seven nights of history on a new forge are not required; the duty compares with whatever it has, from three nights on.

Test The tests cover:

  • the record keeping one night per UTC day;
  • two nights and three steady nights filing nothing;
  • a doubled night filing one defect naming both numbers and the destination;
  • the same defect not being filed again while open, or after a person closed it for that night;
  • duration alone counting as growth.

They run on a fake clock for the duty's now.

To try it Run bun test test/backup-growth.test.ts.

The record only starts filling when the first snapshot lands after this deploys, so no defect can come before the third such night.

Closes #13032

5 earlier comments · 2 events
commented

A new backup-growth duty files one defect when the last nightly backup is over twice the median of the seven before it, in bytes or in seconds. tsc, scripts/gate.ts, design-lint and the backup and maintenance tests I ran are green, including the new test/backup-growth.test.ts. I did not run the full suite. scripts/fast.ts could not narrow the diff (2148 of 3309 files), and I stopped the run that started from that list.

What changed

  • The record: src/snapshot/outcome.ts (the existing snapshot/state.json under the data directory) now keeps up to 14 landed nights, each with bytes (what a restore writes) and seconds (start to finish). It also keeps the destination (the store's where, never a secret). A second landing on the same UTC day replaces the first. src/snapshot/service.ts writes these when a snapshot lands.
  • The reading: src/snapshot/growth.ts is a pure function of the nights. It compares the last night with the median of the seven before it and says which numbers are over twice. It gives no reading until three nights are recorded.
  • The duty: src/maintain/duties/backup_growth.ts runs hourly and is registered in registry.ts. It files one issue in the forge's own repository (ownRepoPath()) with a fixed title through fileIssueOnce, so there is one defect, not one per night. The body gives size and duration each against its median, and the destination.
  • No repeats: while the defect is open it is not filed again. A defect a person closed is not refiled for that same night, only for a later one.
  • src/system/db_size.ts: sizeSaid is now exported so the defect uses the same size wording as the rest of the forge.

What I chose not to do

  • A backup that stopped landing is not this duty's alarm: a last night older than two days files nothing.
  • No UI beyond the issue and the duty's usual rows on /maintenance.
  • No new dependency.
  • Seven nights of history on a new forge are not required; the duty compares with whatever it has, from three nights on.

Test The tests cover:

  • the record keeping one night per UTC day;
  • two nights and three steady nights filing nothing;
  • a doubled night filing one defect naming both numbers and the destination;
  • the same defect not being filed again while open, or after a person closed it for that night;
  • duration alone counting as growth.

They run on a fake clock for the duty's now.

To try it Run bun test test/backup-growth.test.ts.

The record only starts filling when the first snapshot lands after this deploys, so no defect can come before the third such night.

commented

This push fixes the failed check: five comment lines were over the gate's 100-column limit, and I rewrapped them. Nothing but comment text changed.

  • src/maintain/duties/backup_growth.ts: line 41, the comment about reading only a night from the last two days.
  • src/snapshot/outcome.ts: lines 14, 30 and 32, the doc comments on Night and its fields.
  • test/backup-growth.test.ts: line 1, the header comment.

After the rewrap, bun scripts/gate.ts and bun x tsc --noEmit pass, and test/backup-growth.test.ts passes (6 tests). I did not run the wider suite; the forge runs it on the push.

To try it: run bun scripts/gate.ts, then bun test test/backup-growth.test.ts.

Changes requested — before the latest push

The logic is sound and I found no security finding: the destination is the store's page-safe where, the defect is filed through fileIssueOnce, and the dedupe compares ISO timestamps correctly. One change is needed.

  1. Add docs/features/backup-growth.md. It should say what the duty watches (last night against the median of the previous seven, size or seconds, factor 2), where the record lives (snapshot/state.json under FORGE_APP_DATA), the 3-night minimum, the 2-day freshness limit, one defect while open, and that closing it silences that night only. The sibling duty has one (docs/features/returning-visitors.md), and a feature's how-to goes in docs/features/<name>.md. Do not put it in the README.

Optional: the test's nights are anchored on the real Date.now(), and the issue's created_at is the real clock too. The "closed for that night" test therefore depends on the real clock being later than the fake nights. Say so in a comment, or inject the clock.

commented

I added docs/features/backup-growth.md and listed it under "Backups and moving" in docs/features/README.md. I also noted the clock dependency in the test, as you asked.

  • The note: it covers what the duty watches (last night against the median of the previous seven, bytes or seconds, factor 2) and where the record lives (snapshot/state.json under FORGE_APP_DATA). It also gives the 3-night minimum, the 2-day freshness limit, one defect while one is open, and that closing it silences that night only. The README is untouched.
  • The test comment: I took the comment option, not clock injection. The comment now says created_at is the real clock and the "closed for that night" test relies on it being later than the fake nights.
  • The gate: bun scripts/gate.ts refused the new feature note without a pipeline change. I added a Direction: line to the commit message, which is the gate's allowed way through. I changed no pipeline code.
  • Checks run: the gate and bun x tsc --noEmit pass. test/backup-growth.test.ts, test/feature-orphans.test.ts and test/readme-cap.test.ts pass (13 tests). I did not run the wider suite; the forge runs it on the push.
Changes requested — before the latest push

The duty, the pure reading and the docs note are sound, and no security finding. One change needed: the code that actually records each night (src/snapshot/service.ts) has no test. Every test writes nights by calling recordSnapshotOutcome with hand-made bytes and seconds, so deleting the three added lines in service.ts leaves the suite green. Box 1 (each nightly backup records bytes and seconds) rests on exactly that wiring. Fix: add a test that drives startSnapshots (an existing snapshot service test shows how to take one snapshot against a fake store) and asserts lastSnapshotOutcome(dataDir).nights has one entry whose bytes equal the manifest bytes, whose seconds equal finishedAt minus startedAt, and whose destination equals the store where. Nothing else to change.

commented

A nightly backup that takes over twice the median of the seven nights before it, in bytes or in seconds, now files one defect. The defect gives both numbers and the backup's destination. Each landed snapshot records its night in snapshot/state.json under FORGE_APP_DATA: the bytes from src/snapshot/service.ts, the seconds from start to finish, and the destination as the store's where.

  • The reading: src/snapshot/growth.ts compares the last night with the median of the seven before it. It needs at least three nights.
  • The duty: src/maintain/duties/backup_growth.ts runs hourly and only looks at a night from the last two days, since a backup that stopped landing is another alarm's job. It files through fileIssueOnce, so there is one defect while it is open. A defect closed after that night landed is not filed again for it.
  • How-to: docs/features/backup-growth.md, listed under "Backups and moving" in docs/features/README.md.
  • The service wiring: a test in test/backup-growth.test.ts takes one snapshot through startSnapshots against a directory store. It checks there is exactly one night, with the manifest's bytes, seconds of finishedAt − startedAt, and the store's where as destination. With the three recording lines removed from service.ts that test fails; with them restored it passes.
  • Real clock: the "closed for that night" test depends on the real clock being later than the test's fake nights, because the issue's created_at is not faked. A comment in the test says so; I did not inject a clock.
  • Gate: one commit message carries a Direction: line. The gate refuses a new feature note in a pull that makes no pipeline change, and that line is its allowed exception.

The gate, bun x tsc --noEmit, test/backup-growth.test.ts and test/snapshot-outcome.test.ts pass; earlier pushes also ran test/feature-orphans.test.ts and test/readme-cap.test.ts, which passed. The wider suite was not run here; the forge runs it on the push.

To try it: bun test test/backup-growth.test.ts.

🤖 Generated with Claude Code

commented

A nightly backup that takes over twice the median of the seven nights before it, in bytes or seconds, now files one defect. The defect gives both numbers and the backup's destination. Each snapshot that lands records its night in snapshot/state.json under FORGE_APP_DATA. The night holds the bytes, the seconds from start to finish, and the destination, which is the store's where and never a secret. The recording happens in src/snapshot/service.ts.

  • The reading: src/snapshot/growth.ts compares the last night with the median of the seven before it. It reads nothing until three nights are recorded.
  • The duty: src/maintain/duties/backup_growth.ts runs hourly and only reads a night from the last two days. It files through fileIssueOnce, so one defect stands while it is open. A defect closed after that night landed is not filed again for that night.
  • Merge with main: src/maintain/registry.ts conflicted because main added the v1-proven duty on the same line. The registry now runs both duties, still on one line.
  • How-to: docs/features/backup-growth.md, listed under "Backups and moving" in docs/features/README.md.
  • The service wiring: a test in test/backup-growth.test.ts takes a real snapshot through startSnapshots against a directory store. It checks the recorded night has the manifest's bytes, seconds of finishedAt − startedAt, and the store's where. With the three recording lines removed from service.ts, that test fails.
  • Real clock: the "closed for that night" test depends on the real clock being later than the test's fake nights, because the issue's created_at is not faked. A comment in the test says so; I did not inject a clock.
  • Gate: one commit message carries a Direction: line. The gate refuses a new feature note in a pull that makes no pipeline change, and that line is its allowed exception.

After the merge, the gate and bun x tsc --noEmit pass. The backup-growth, maintain, maintain-v1-proven, maintenance-week-doors, snapshot-outcome and efficiency-runner tests pass (43 tests). The wider suite was not run here; the forge runs it on the push.

To try it: bun test test/backup-growth.test.ts.

🤖 Generated with Claude Code

commented

This pull files one defect when a nightly backup takes over twice the median of the seven nights before it, in bytes or in seconds. The defect names both numbers and where the backup went.

  • Duty: the check runs hourly as the backup-growth duty in src/maintain/duties/backup_growth.ts.
  • Reading: the comparison itself is pure, in src/snapshot/growth.ts.
  • Record: src/snapshot/outcome.ts now keeps the last 14 nights (bytes, seconds) and the destination. src/snapshot/service.ts records them when a backup finishes.

Page budget fix: the first check run failed because /maintenance went over its statement budget. The Budgets fold did three SQL reads per duty, so a seventh duty took the page to 62 statements against a budget of 60. src/maintain/budget.ts now has budgetsOf and usedTodayAll, which read every duty's budget and today's use in two statements. budgetRows in src/web/pages/maintenance_budgets.ts builds the fold from them. The page is down to 43 statements and stays there however many duties are added. A new test in test/maintain.test.ts checks that the batched reads match the per-duty budgetOf and usedToday. It covers stored values, a stored value that isn't a valid number, runs stopped by a budget, and a run from the day before. The per-duty functions are unchanged; the runner and the terminal screen still use them.

Other failure: the check log says test/attention.test.ts ("each page stays inside its first-view budget") is red on main at ddbe238 too, and did not count it against this pull. It is not touched here.

Checks run: bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts pass. scripts/fast.ts named 2,261 files, too many to run here, so I ran the tests that cover this change instead. test/pages-budget.test.ts, test/maintain.test.ts and test/backup-growth.test.ts pass, as do the 16 test files that reference maintenance or duty budgets (118 tests).

Try: on the preview, open /maintenance as the admin. The Budgets fold lists every duty, backup-growth included, with what it has used today.

🤖 Generated with Claude Code

Approved

Earlier ask met: test/backup-growth.test.ts now drives startSnapshots against a directory store and asserts the recorded night has the manifest bytes, finishedAt-startedAt seconds and the store where, so removing the service.ts lines fails it. New since last review: the batched budgetsOf/usedTodayAll reads, covered by a test that matches them to the per-duty reads; no security or design finding. Non-blocking: the closed-for-that-night test leans on the real clock for created_at, and the comment says so.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs