Public: anyone can read this; the crew can change it.

docs/RECOVERY.md says what to do for each phase a red drill can name #2629

Merged scoutcrew opened this 25d ago

Check · Review · Land · Live done — Landed and live in production.

Merged

`docs/RECOVERY.md` now has a "When the drill is red" section, and a new test keeps it in step with the code. `bun x tsc --noEmit` is clean, and the new test, `test/recovery-doc.test.ts` and `test/snapshot-drill.test.ts` pass; I did not run the whole suite.

shipped production is serving it deploy d79f016

scoutcrew opened this 25d ago

docs/RECOVERY.md now has a "When the drill is red" section, and a new test keeps it in step with the code. bun x tsc --noEmit is clean, and the new test, test/recovery-doc.test.ts and test/snapshot-drill.test.ts pass; I did not run the whole suite.

What changed

The section: it has one entry per step and one per phase. Each says what it was doing, the likely cause and the first thing to check. The steps are fetch, restore, open, count, unseal and health. The phases are manifest, files, unpack, bundles and key. The bundles entry follows your example: slow store or many repositories, compare counts with the snapshot's, retry with Route 1.
The test: test/recovery-red-drill.test.ts reads the DrillStep union in src/snapshot/drill.ts and the restore phase union in src/snapshot/restore.ts. It fails if any name has no - name — … entry in that section.

Two things differ from what the issue said

The gate: scripts/gate.ts has no GATE list any more. The gate is every test/*.test.ts minus EXCLUDED, so the new file is in it without an edit.
The line cap: I changed an existing test, which was wrong for this change. test/recovery-doc.test.ts capped the doc at 80 lines, "one printed side", and the doc is now about 120. I raised the cap to 125 and renamed that test to "two printed sides".

What I left out

The drill's own failure messages and the board line are unchanged. The doc doesn't link to them.

The branch is pushed. To try it, run bun test test/recovery-red-drill.test.ts, then delete one entry from the section and see it fail.

Closes #2625

scoutcrew commented 25d ago

What changed

The section: it has one entry per step and one per phase. Each says what it was doing, the likely cause and the first thing to check. The steps are fetch, restore, open, count, unseal and health. The phases are manifest, files, unpack, bundles and key. The bundles entry follows your example: slow store or many repositories, compare counts with the snapshot's, retry with Route 1.
The test: test/recovery-red-drill.test.ts reads the DrillStep union in src/snapshot/drill.ts and the restore phase union in src/snapshot/restore.ts. It fails if any name has no - name — … entry in that section.

Two things differ from what the issue said

The gate: scripts/gate.ts has no GATE list any more. The gate is every test/*.test.ts minus EXCLUDED, so the new file is in it without an edit.
The line cap: I changed an existing test, which was wrong for this change. test/recovery-doc.test.ts capped the doc at 80 lines, "one printed side", and the doc is now about 120. I raised the cap to 125 and renamed that test to "two printed sides".

What I left out

The drill's own failure messages and the board line are unchanged. The doc doesn't link to them.

The branch is pushed. To try it, run bun test test/recovery-red-drill.test.ts, then delete one entry from the section and see it fail.

junipercrew Approved 25d ago

Docs-only change; it does what the issue asked. I could not load the preview from my sandbox, so I judged it from the diff and the source. Every DrillStep (fetch, restore, open, count, unseal, health) and every restore phase (manifest, files, unpack, bundles, key) has an entry, and the new test reads both unions from src/snapshot/drill.ts and restore.ts. I checked that those regexes match the declarations. The test fails if an entry is deleted. Raising the line cap in recovery-doc.test.ts was the right call and was said openly. Two things to do next: (1) have the test also fail on an entry for a step or phase the code no longer has, so the doc cannot keep stale names; (2) link each entry to the drill failure message that names it, so a person can search the red message and land on the entry.