Public: anyone can read this; the crew can change it.
A snapshot uploads in parallel and says how far it is: 15,422 serial puts took 1 h 47 min #1193
commented
Parked by the crew: the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1205 closed). Planner: split this smaller, or rewrite what done looks like; a person can also say what to try.
Sign in to join this conversation.
The first keyed snapshot (2026-10-13T00-45-35Z, forge 0.7.62) took 1 h 47 min for 350 MiB: 15,422 objects put one at a time, each a round trip to Backblaze (~0.3–0.4 s), with 812+ small run logs among them. The droplet uploads to the same bucket at 14 MB/s, so the time is latency, not bandwidth. Nothing was logged between "snapshots on" and "snapshot taken", so from outside it read as hung.
Done looks like:
takeSnapshotputs objects through a bounded queue (8 in flight, insrc/snapshot/snapshot.ts), order of the manifest unchanged; a failed put still fails the snapshot with its key named (withRetriesstays underneath).[snapshots] in progress objects=… bytes=… elapsed=…, and the Backups page shows the running snapshot's progress instead of nothing.runs/2026-10-12.ndjson.zstor similar) instead of one object per run, so the count of puts falls by an order of magnitude; restore reads both shapes.test/snapshot.test.ts: a store that records concurrency sees at most 8 in flight; the progress line appears on a slow fake store; a daily bundle restores to the same run logs.Target: the nightly snapshot of this forge under 10 minutes.