Public: anyone can read this; the crew can change it.
A failed nightly backup says why in the system card: cause word, bytes sent and seconds spent #13074
Check · Review · Land · Live done — Landed and live in production.
A failed nightly backup now says why on /system, in the terminal and in the MCP system tool. The new and existing tests I ran pass, along with tsc, scripts/gate.ts and scripts/design-lint.ts. I did not run the full suite; the forge runs it on push.
shipped production is serving it deploy 59903fa
2 earlier comments · 1 event
A failed nightly backup now says why on /system, in the terminal and in the MCP system tool. The new and existing tests I ran pass, along with tsc, scripts/gate.ts and scripts/design-lint.ts. I did not run the full suite; the forge runs it on push.
What changed
- Cause, bytes and seconds are kept. A new
src/snapshot/cause.tsturns the error into one word:timeout,auth,disk,refusedorunknown.src/snapshot/service.tsrecords that word, the bytes landed before the failure and the seconds spent into the kept outcome. A snapshot that lands clears it. - One line on
/system. The Off-site backup section gets a "Last nightly backup" row that reads, for example,timeout after 42 s, 1.5 MiB sent. The terminal's off-site screen prints the same line. - MCP.
systemwithkind: "offsite"now also returnsbackupFailure: { cause, bytes, seconds, says }, ornullwhile the last snapshot landed. There is no new tool, andtest/mcp-size.test.tsis green with no ceiling raised.
Tests
test/backup-failure-cause.test.tsruns each cause through the classifier, the kept outcome, the lane reading, the/systemcard and the terminal screen. It also covers a landed snapshot clearing the failure.test/snapshot-outcome.test.tsnow asserts that a real failing nightly keeps the cause, bytes and seconds. That test caught "no answer in 180 s" being read asunknown, so the classifier now counts it as a timeout.
What I chose not to do
- Mirror failures. The off-site mirror pass already shows its own error, and I left it as it was.
- Old outcomes. An outcome written before this change has no cause, so no line appears for it until the next failure.
How to try it
Sign in as an admin, open /system, then Off-site backup, after a failed nightly. The Try: commit has the same steps.
A failed nightly backup now reads as one line, for example timeout after 42 s, 1.5 MiB sent. It shows under "Last nightly backup" on /system and on the terminal's Off-site screen. The MCP system tool with kind: offsite returns it as backupFailure. This push fixes the gate failures from the first check; the gate, tsc --noEmit and the two test files covering the change pass here. I did not run the full suite.
- Cause word:
src/snapshot/cause.tsderives it from the error. A refused login counts as auth, and an error that got no answer counts as a timeout. - Where it is kept: the failure's cause, bytes sent and seconds spent are stored with the nightly's outcome. Outcomes kept before this change have no failure, so they show nothing. A snapshot that lands clears it.
- Test types:
test/backup-failure-cause.test.tsno longer casts withas never; the cases are typed asFailureCause. - Terminal assertion: it moved into
test/offsite-lane.test.ts, which already builds a real session, and now checks theFailed timeout after 42 s, 1.5 MiB sentline. - Comments: the three over-long comment lines are rewrapped to 100 columns.
- Gate lines: the commit message carries
No door:andLooks: /system. The "No door" reason is that the terminal's Off-site screen already shows the failure anddocs/MCP.mdalready names thesystemtool.
To try it: open /system as an admin and look under Off-site backup. After a failed nightly, "Last nightly backup" shows the line above.
🤖 Generated with Claude Code
Security is clean: admin-gated, only a cause word, bytes and seconds are exposed, output goes through row(). Two things to fix in one round. (1) Both doors: no docs/features/<name>.md how-to was added or extended for the failure line (the commit message claims No door, but the how-to is required; put it in docs/features, not README). (2) Tests: nothing reaches the MCP tool for backupFailure (box 3), and the service test only checks typeof bytes === number, which passes if sent is always 0. Also tighten the disk regex (see comment).
All three points from the review are fixed and pushed. The gate and tsc --noEmit are clean, and test/backup-failure-cause.test.ts, test/offsite-lane.test.ts and test/snapshot-outcome.test.ts pass; I did not run the full suite.
- How-to:
docs/features/offsite-lane.mdhas a new "When the nightly fails (#13046)" section. It covers the failure line, the cause words, and where it shows on/system, the terminal's Off-site screen and the MCPsystemtool. - MCP test:
test/offsite-lane.test.tsnow records a failed outcome, readssystemwithkind: "offsite", and assertsbackupFailureis{ cause: "timeout", bytes: 1572864, seconds: 42, says: "timeout after 42 s, 1.5 MiB sent" }. It then records a landed snapshot to restore state. The null case is covered by the existing "a landed snapshot clears it" test intest/backup-failure-cause.test.ts(it checks the lane, not the MCP read). - Sent-bytes test: the failing store in
test/snapshot-outcome.test.tslets every chunk land and then gives the manifest no answer. The test counts the bytes that landed and assertsfailure.bytesequals that count and is above 0, so a hard-coded 0 now fails it. I used the manifest put because that test's snapshot has only one chunk, so there is no second chunk to fail. - Disk regex:
src/snapshot/cause.tsnow matches the errno codesENOSPC,EDQUOT,EROFSandEIOas whole, case-sensitive words. It also matchesno space,disk fullanddisk quota; barequotais gone. New cases check that an error witheioinside a word and an "API quota exceeded" error both read as unknown, and thatEIOas a code reads as disk.
One slip: I pushed 72491d983 with the gate still red on a test comment line and a new as never cast. ef4152a0c fixes both, so the latest push is clean. I kept the existing { app } as never call and added no new cast.
🤖 Generated with Claude Code
Earlier asks are met. The how-to is in docs/features/offsite-lane.md. The MCP read is asserted with backupFailure in test/offsite-lane.test.ts. The sent-bytes test now fails on a hard-coded 0. The disk regex is tightened, with cases for a word holding eio and for API quota. Nothing new on the changed lines: only the cause word and two numbers are shown, never the error text; the row is the named component and works without JavaScript; there is no new dependency and README is untouched.
A failed nightly backup now says why on
/system, in the terminal and in the MCPsystemtool. The new and existing tests I ran pass, along withtsc,scripts/gate.tsandscripts/design-lint.ts. I did not run the full suite; the forge runs it on push.What changed
src/snapshot/cause.tsturns the error into one word:timeout,auth,disk,refusedorunknown.src/snapshot/service.tsrecords that word, the bytes landed before the failure and the seconds spent into the kept outcome. A snapshot that lands clears it./system. The Off-site backup section gets a "Last nightly backup" row that reads, for example,timeout after 42 s, 1.5 MiB sent. The terminal's off-site screen prints the same line.systemwithkind: "offsite"now also returnsbackupFailure: { cause, bytes, seconds, says }, ornullwhile the last snapshot landed. There is no new tool, andtest/mcp-size.test.tsis green with no ceiling raised.Tests
test/backup-failure-cause.test.tsruns each cause through the classifier, the kept outcome, the lane reading, the/systemcard and the terminal screen. It also covers a landed snapshot clearing the failure.test/snapshot-outcome.test.tsnow asserts that a real failing nightly keeps the cause, bytes and seconds. That test caught "no answer in 180 s" being read asunknown, so the classifier now counts it as a timeout.What I chose not to do
How to try it Sign in as an admin, open
/system, then Off-site backup, after a failed nightly. TheTry:commit has the same steps.Closes #13046