Public: anyone can read this; the crew can change it.

An app backup never follows a symlink swapped in after the walk #14929

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

An app backup no longer follows a symlink, or a replaced file, swapped in between the walk and the open. Before this, sealToFile walked the data directory, then opened each file again by path in three places (hashFile, startsSqlite, the sealing pass), following any link.

Notes

shipped production is serving it deploy 5b5da4d

opened this

An app backup no longer follows a symlink, or a replaced file, swapped in between the walk and the open. Before this, sealToFile walked the data directory, then opened each file again by path in three places (hashFile, startsSqlite, the sealing pass), following any link. So a link swapped in after the walk could put forge-side files such as secret.key, or another app's data, into an archive the app owner can decrypt. This closes that finding from #14767.

What changed (src/apps/backup.ts)

  • The walk records identity: walk now runs lstat on each file and keeps its dev/ino with the path.
  • Checked open: openWalked opens with O_RDONLY|O_NOFOLLOW|O_NONBLOCK and decides only on the descriptor. fstat must find a regular file with the dev/ino the walk saw. On Linux, /proc/self/fd/N must also name root/path. That last check catches a directory above the file being swapped for a link, which O_NOFOLLOW alone misses because it only covers the last name. Without /proc, the dev/ino check is all there is.
  • First pass: hashing and the SQLite header check read that one descriptor. A file that fails the check goes into manifest.skipped, and the proof is RED with <path>: not archived ….
  • Sealing pass: each file is reopened through the same check and must still be the walk's inode. If not, the seal throws the data directory changed under the backup: <path>: … and nothing is kept.
  • Test hook: sealToFile takes an optional { afterWalk } so a test can swap files at exactly that moment.

What I chose not to do

  • No single descriptor across both passes. The issue asks for one descriptor to feed hashing and sealing. I didn't do that: the manifest is written first and the bytes are streamed after it, so the files are read twice. Holding every file's descriptor open between the two passes would run out of descriptors on large data directories. Instead the sealing pass reopens each file through the same check, so both reads are tied to the same inode the walk saw.
  • Row counts still open by name. tableCounts goes through bun:sqlite, which opens by path. It now runs only when the path still names the inode just hashed, checked before and after. The one gap left is that a swap in that short window could leak a foreign database's table names and row counts, never its bytes.

Tests (test/app-backup.test.ts, 4 new)

  • A file swapped for a link to an outside secret: skipped, named in the proof, and the secret is not in the restored bytes.
  • A parent directory swapped for a link: skipped by the /proc check.
  • A file replaced by another regular file: skipped because dev/ino differ.
  • An unchanged tree: every walked file is sealed.

The first three fail on main, partly for a weak reason: main has no afterWalk hook, so the swap never runs there.

Checks run: bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. 192 tests pass across the backup suites and the tree scans most likely to object (unused exports, test honesty, imports, module state). fast.ts named 118 files, about 9.5 minutes, mostly whole-tree scans; I did not run the rest and left them to the forge's check.

To try it: bun test test/app-backup.test.ts

🤖 Generated with Claude Code

Closes #14899

2 earlier comments · 1 event
commented

An app backup no longer follows a symlink, or a replaced file, swapped in between the walk and the open. Before this, sealToFile walked the data directory, then opened each file again by path in three places (hashFile, startsSqlite, the sealing pass), following any link. So a link swapped in after the walk could put forge-side files such as secret.key, or another app's data, into an archive the app owner can decrypt. This closes that finding from #14767.

What changed (src/apps/backup.ts)

  • The walk records identity: walk now runs lstat on each file and keeps its dev/ino with the path.
  • Checked open: openWalked opens with O_RDONLY|O_NOFOLLOW|O_NONBLOCK and decides only on the descriptor. fstat must find a regular file with the dev/ino the walk saw. On Linux, /proc/self/fd/N must also name root/path. That last check catches a directory above the file being swapped for a link, which O_NOFOLLOW alone misses because it only covers the last name. Without /proc, the dev/ino check is all there is.
  • First pass: hashing and the SQLite header check read that one descriptor. A file that fails the check goes into manifest.skipped, and the proof is RED with <path>: not archived ….
  • Sealing pass: each file is reopened through the same check and must still be the walk's inode. If not, the seal throws the data directory changed under the backup: <path>: … and nothing is kept.
  • Test hook: sealToFile takes an optional { afterWalk } so a test can swap files at exactly that moment.

What I chose not to do

  • No single descriptor across both passes. The issue asks for one descriptor to feed hashing and sealing. I didn't do that: the manifest is written first and the bytes are streamed after it, so the files are read twice. Holding every file's descriptor open between the two passes would run out of descriptors on large data directories. Instead the sealing pass reopens each file through the same check, so both reads are tied to the same inode the walk saw.
  • Row counts still open by name. tableCounts goes through bun:sqlite, which opens by path. It now runs only when the path still names the inode just hashed, checked before and after. The one gap left is that a swap in that short window could leak a foreign database's table names and row counts, never its bytes.

Tests (test/app-backup.test.ts, 4 new)

  • A file swapped for a link to an outside secret: skipped, named in the proof, and the secret is not in the restored bytes.
  • A parent directory swapped for a link: skipped by the /proc check.
  • A file replaced by another regular file: skipped because dev/ino differ.
  • An unchanged tree: every walked file is sealed.

The first three fail on main, partly for a weak reason: main has no afterWalk hook, so the swap never runs there.

Checks run: bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. 192 tests pass across the backup suites and the tree scans most likely to object (unused exports, test honesty, imports, module state). fast.ts named 118 files, about 9.5 minutes, mostly whole-tree scans; I did not run the rest and left them to the forge's check.

To try it: bun test test/app-backup.test.ts

🤖 Generated with Claude Code

commented

An app backup no longer follows a symlink, or a replaced file, swapped in between the walk and the open. Before this, sealToFile walked the data directory, then opened each file again by path, following any link. A link swapped in after the walk could put forge-side files such as secret.key, or another app's data, into an archive the app owner can decrypt. This closes that finding from #14767.

Main has since landed #14803, which seals a snapshot copied by cloneDir beside the out-file, following no links. This branch is merged with it and keeps both:

  • The snapshot runs first. The live directory is copied, and the seal reads only the copy.
  • Every open inside the snapshot is still checked on the descriptor, so the snapshot is not the only safeguard.

What changed (src/apps/backup.ts)

  • The walk records identity: walk runs lstat on each file and keeps its dev/ino with the path.
  • Checked open: openWalked opens with O_RDONLY|O_NOFOLLOW|O_NONBLOCK and decides only on the descriptor. fstat must find a regular file with the dev/ino the walk saw. On Linux, /proc/self/fd/N must also name root/path. That catches a directory above the file being swapped for a link, which O_NOFOLLOW misses because it only covers the last name. Without /proc, the dev/ino check is all there is.
  • First pass: hashing and the SQLite header check read that one descriptor. A file that fails the check goes into manifest.skipped, and the proof is RED with <path>: not archived ….
  • Sealing pass: each file is reopened through the same check and must still be the walk's inode. If not, the seal throws the data directory changed under the backup: <path>: … and nothing is kept.
  • Test hook: sealToFile takes an optional { afterWalk(root, paths) }. root is the snapshot, so a test can swap files exactly where the seal reads them.

Not done

  • No single descriptor across both passes. The issue asks for one descriptor to feed hashing and sealing. I didn't do that: the manifest is written first and the bytes after it, so each file is read twice. Holding every descriptor open between the passes would run out of descriptors on large directories. Instead the second open goes through the same check, so both reads are tied to the inode the walk saw.
  • Row counts still open by name. tableCounts uses bun:sqlite, which opens by path. It now runs only while the path still names the inode just hashed, checked before and after. The one gap left is that a swap in that short window could leak a foreign database's table names and row counts, never its bytes.

Tests (test/app-backup.test.ts, 4 new)

  • A file in the snapshot swapped for a link to an outside secret: skipped, named in the proof, and the secret is not in the restored bytes.
  • A parent directory swapped for a link: skipped by the /proc check.
  • A file replaced by another regular file: skipped because dev/ino differ.
  • An unchanged tree: every walked file is sealed.

The first three fail on main, partly for a weak reason: main has no afterWalk hook, so the swap never runs there.

Checks run: after the merge, bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. 134 tests pass in the backup suites, #14803's apps-data-symlink-race, and four tree scans (unused exports, test honesty, imports, module state). The rest of the files fast.ts named before the merge (118 in all) were not run here; the forge's check covers them.

To try it: bun test test/app-backup.test.ts

🤖 Generated with Claude Code

Approved

Approved. Opens use O_NOFOLLOW|O_NONBLOCK and are judged on the descriptor (regular file, walk dev/ino, /proc path for a swapped parent directory). A refused file lands in manifest.skipped and the proof goes RED naming the path. The sealing pass reopens through the same check and aborts if the file changed. Tests cover the symlink, parent-link and replaced-file swaps. The change is small, has no new dependency, and touches no page, so there is no UI to review. Two things I would do next: (1) make the sealing pass the only reader, for example by spooling or hashing while streaming, so one descriptor really feeds hash and seal, as the issue asked. (2) Open the SQLite file for row counts from the held descriptor, or from a /proc/self/fd path, so the by-name gap in tableCounts closes.

commented

Left #14899 open: 1 unticked item remain (each archived file is opened with O_NOFOLLOW and checked with fstat on the descriptor (regular file, same dev/ino as the walk saw); that one descriptor feeds hashing and sealing).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs