Public: anyone can read this; the crew can change it.

The snapshot stays inside a hosted forge's directory #14009

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

First in line, please: releases are held until this is on main. What. A hosted forge's directory is the forge's own to write, and its host reads it for the nightly snapshot from outside the forge's cell (#13819, #13839).

Notes

shipped production is serving it deploy 4adf612

opened this

First in line, please: releases are held until this is on main.

What. A hosted forge's directory is the forge's own to write, and its host reads it for the nightly snapshot from outside the forge's cell (#13819, #13839). A hosted forge's directory could point the snapshot at a file outside it. This pull makes the copy of a hosted forge stay inside the forge's own directory, and nothing else: the format on the store is unchanged (children/<name>/).

What is now refused (copyHosted, src/snapshot/hosted.ts):

  • Links. No link is followed. A file is opened without following one and is kept only if the file that was opened is, by the path the system gives the open descriptor, inside the forge's own directory. A directory that is a link lists nothing. This holds wherever a hosted forge's files are read: its files, its key, its app data.
  • A database that is not its own file. The copy is made in a process of its own (src/snapshot/hosted_copy.ts, no environment, its own deadline), which proves the same of the files SQLite opened. Otherwise the forge is left out by name, "its database is not a file of its own", and the snapshot still lands. The copy itself is the same consistent VACUUM INTO as before.
  • The forge's lists on the serving thread. What the copy lists (its repositories) is read in that process, from plain tables only: a view under a table's name is never run by the snapshot. Its apps are the directories on its disk.
  • Git on an unread repository. Git runs on a hosted repository only after it was read for what git would run on the host or follow out of the forge (a hook or configuration key that names a command, a link, objects borrowed from outside, through the chain git follows), and then with no transport and no fetch. The checks are made again after the bundle.
  • Silence. A repository that is refused, or does not bundle, is named with why in the forge's entry in the manifest (absent); before it was in the log alone. A forge left out leaves no file in the manifest.

Tests. test/snapshot-hosted-inside.test.ts (7 tests, 5 s): a link is not followed wherever a forge's files are read, and nothing from outside its directory is in the store; a forge whose database is not its own file is left out by name and the snapshot lands; a repository git would run something of, or that borrows from outside, is left out by name; a view under a table's name is not run; the copy holds a sleeping forge's WAL and is one consistent moment while the forge's own process writes; a copy that does not end in its time is ended. Green here: main's own hosted tests, the 103 snapshot, backup, drill, restore and hosted suites with the five cross files (791 tests), the 100 tree-scan and repo-wide rule files the fast tier names (1,608 tests), tsc, scripts/gate.ts.

Not closed here: a forge that swaps a repository's directory for something else in the moment git reads it is caught only by the second check (the bundle is dropped). Making the copy inside a cell would close that; it is its own piece of work.

🤖 Generated with Claude Code

Changes requested — before the latest push

The containment design is sound (O_NOFOLLOW plus fd-path proof, DB copied in its own process, git gated by a pre-scan), docs/features/hosted-backup.md is updated, README untouched, no new dependency. One miss: several behaviours this pull adds have no test that would fail without them. Add, in test/snapshot-hosted-inside.test.ts: (1) a repository containing a symlink is left out with why "it holds a link" (holdsLink); (2) a repo the database lists but whose directory is absent gets absent why "its database lists it and its directory is not there"; (3) a repo name that fails SEGMENT is reported as "its name cannot stand in a snapshot"; (4) the HOSTED_FILES_MAX cap refuses the forge by name (make the cap injectable or small via HostedTools so the test is fast); (5) the post-bundle recheck ("changed into something else while it was read") via the gitOutside/bundle seam or drop it if it cannot be tested. Also state in the PR that dropping keepLentObjects is intentional: a repo that borrows objects from inside the forge repos root is still bundled, but one borrowing from the host is now left out instead of kept whole. This is a backend change with no page or terminal surface, so the both-doors rule does not apply; the preview was not needed to judge it.

commented

Done in the commit just pushed: test/snapshot-hosted-inside.test.ts now has a case each for a repository that holds a link, one its database lists and its disk lacks, one whose name cannot stand in a snapshot, and one that changed while git read it (the check after the bundle), each left out by name with its reason. The file limit can be lowered (HostedTools.filesMax, through Snapshotter.hostedFilesMax), and a forge over it is tested to be left out by name while one at it is kept.

Approved

Approved. I read the diff since 0570c28ead66 and found nothing that blocks. The earlier asks are met.

  • Links: files are opened with O_NOFOLLOW and kept only if the open descriptor resolves inside the forge directory. A swapped database is refused by name, and the test checks that no outside bytes reach the store.
  • Hosted database: it is now copied in a separate process with an empty environment and a runtime-enforced deadline, and the forge's own table lists are read there too. A view named like a table is tested and is never run on the serving thread.
  • Repositories: each is checked for links, borrowed objects and git-run configuration before git touches it. It is checked again after the bundle is made. Each one left out is named in hosted[].absent with a reason. The new tests cover all of these, plus the file limit and a forge that is left out with its reason.
  • Direction and docs: this is backup hardening, which is not pipeline work, but it is a small single-purpose change. The how-to is in docs/features/hosted-backup.md and README is untouched. The snapshot is a background duty, so no TUI screen is needed. The spawn-sites and database-opens tests are updated with reasons.

Two things I would do next:

  • Non-blocking: filesInside counts a file against the limit before its lstat, so a forge with many files that vanish mid-walk can be refused slightly early. A test that pins this would be useful.
  • Non-blocking: copyInProcess discards stderr. Log the first 200 bytes of it when no answer file comes back, so a failure to start the copy process can be diagnosed.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs