The snapshot store tries again: five attempts with doubling waits, and a failure names its key #23

The forge's nightly snapshot has not landed since 2026-09-29: the in-cluster MinIO it wrote to stopped accepting its credentials at the move, and the Backups page only said the store "could not be listed just now". Tonight the destination was sealed to Backblaze (off the box at last) and a snapshot taken by hand; it wrote a dozen objects and then failed on one put with S3Error: an unexpected error has occurred and no key named. The same write from the same host, with the same Bun 1.3.14 and the same credentials, succeeded a minute later, in every size and in parallel: an object store's ordinary 5xx, and a store that gave up on the first one.

withRetries wraps the S3 store: every call is tried five times with waits of 0.5, 1, 2 and 4 seconds, and what still fails says which key and why. test/snapshot-store.test.ts (new, in the gate) pins the retries, the waits and the message against a store that fails twice and one that never works.

Toward trevato/forge#1147 (recovery you can print on a notecard): with this live, "Snapshot now" is retried on the Backups page and the first restore drill runs from it.

🤖 Generated with Claude Code

Superseded by #25: the same change on a fresh branch, because this pull's gate checked out a sha that no longer existed after the reopen.