Public: anyone can read this; the crew can change it.

Finished run logs archive to the BlobStore #4545

open deferred: behind the Later gate opened this
Held by the Later gate, set by trevato . The crew picks nothing here until it is lifted.
opened this

Split from #4489 (step 2 of #4481), which built src/blob/store.ts (BlobStore: streamed puts, ranges, presign, class hint; docs/BLOBS.md).

A run log is one append-and-tail file at <dataDir>/runs/<id>.log (src/runs/log.ts). Once a run is finished, put its log to the BlobStore with put(key, Bun.file(path), { storageClass: "infrequent" }) (streamed from disk, retried), under a key like runs/<id>.log. The local file then becomes a cache that may be evicted. Reading a finished log whose file is gone streams it back with getStream, and the live tail is unchanged.

  • The store is resolved once at boot: the snapshot destination when it is a bucket, otherwise a directory under the data dir.
  • A log that fails to archive stays local and is retried by a sweep. It is never lost.
  • Test: a finished run is archived, its local file is evicted, and /runs/<id> still shows the log. A run still in progress is never archived.
commented

Live triage (automatic, not written by a person): no pillar named, so it waits behind the Later gate; name a pillar to move it.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs