Public: anyone can read this; the crew can change it.

The packs' upstreams are read-only forge mirrors under the packs owner, re-synced on a schedule #10589

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

An admin can now bring each pack's https `source` in as a public, read-only repo `packs/<app>`, and the scheduled follow keeps it current. The new test file `test/packs-mirror.test.ts` passes, along with `test/mcp-size.test.ts`, `tsc`, `scripts/gate.ts` and the design lint. I did not run the whole suite: `scripts/fast.ts` named 1711 of 2455 files, so I ran only the files around the change (packs,…

shipped production is serving it deploy cca14ac

opened this

An admin can now bring each pack's https source in as a public, read-only repo packs/<app>, and the scheduled follow keeps it current. The new test file test/packs-mirror.test.ts passes, along with test/mcp-size.test.ts, tsc, scripts/gate.ts and the design lint. I did not run the whole suite: scripts/fast.ts named 1711 of 2455 files, so I ran only the files around the change (packs, git over ssh, the follow, MCP tool listing).

Changes

  • Door: src/packs/mirror.ts has mirrorPacks. On /packs, an admin sees a "Mirror the upstreams" button, and POST /packs/mirror runs the same action. The MCP door is repo_settings with { mirrorPacks: true }, which answers { made, kept, failed, says }. I put it on that existing tool because there is no tool named repos; the repo argument is no longer required. mcp-size still passes without raising any ceiling.
  • How a mirror is made: the packs account owns every mirror and is created on first use; nobody can sign in as it. The mirror's default branch is whatever the upstream's HEAD points to, and the follow is enabled through putFollow. The first sync runs at once.
  • Following: I added one early branch in checkTarget in src/actions/upstream_follow.ts. On a packs/* repo it moves the default branch to the fetched upstream tip and opens no pull. Forks and followed sources behave as before.
  • Push refusal: a push to packs/* is refused over http (src/http/server.ts) and ssh (src/tui/ssh.ts), including by the packs owner, who is told why. Everyone else gets "not found", as before.
  • Running twice: an existing packs/<app> is kept, so nothing new is opened.
  • Tests: the upstream is a real bare repo served over dumb http. They cover admin-only access, creation, running twice, a new upstream commit arriving on checkDue with no pull opened, push refusal, the button, and the MCP argument. I did not call the MCP action end to end as an admin; the MCP test only checks the schema and that a non-admin is refused.
  • Docs: a note in docs/features/packs.md.

Left out

  • Other writes to a mirror, such as issues or hand-opened pulls, are not blocked. The issue asked only for push refusal.
  • If two packs name the same app, the first source wins.
  • An app whose upstream can't be reached is reported as failed and no repo is made, so running it again retries it.
  • The initial sync is not retried within the run; the schedule picks it up.

Try it

  • Sign in as an admin, open /packs, and press "Mirror the upstreams". It needs outbound access to github.com.
  • Over MCP, call repo_settings with {"mirrorPacks": true}.
  • Then git clone <forge>/packs/hextris.git.

🤖 Generated with Claude Code

Closes #10530

commented

An admin can now bring each pack's https source in as a public, read-only repo packs/<app>, and the scheduled follow keeps it current. The new test file test/packs-mirror.test.ts passes, along with test/mcp-size.test.ts, tsc, scripts/gate.ts and the design lint. I did not run the whole suite: scripts/fast.ts named 1711 of 2455 files, so I ran only the files around the change (packs, git over ssh, the follow, MCP tool listing).

Changes

  • Door: src/packs/mirror.ts has mirrorPacks. On /packs, an admin sees a "Mirror the upstreams" button, and POST /packs/mirror runs the same action. The MCP door is repo_settings with { mirrorPacks: true }, which answers { made, kept, failed, says }. I put it on that existing tool because there is no tool named repos; the repo argument is no longer required. mcp-size still passes without raising any ceiling.
  • How a mirror is made: the packs account owns every mirror and is created on first use; nobody can sign in as it. The mirror's default branch is whatever the upstream's HEAD points to, and the follow is enabled through putFollow. The first sync runs at once.
  • Following: I added one early branch in checkTarget in src/actions/upstream_follow.ts. On a packs/* repo it moves the default branch to the fetched upstream tip and opens no pull. Forks and followed sources behave as before.
  • Push refusal: a push to packs/* is refused over http (src/http/server.ts) and ssh (src/tui/ssh.ts), including by the packs owner, who is told why. Everyone else gets "not found", as before.
  • Running twice: an existing packs/<app> is kept, so nothing new is opened.
  • Tests: the upstream is a real bare repo served over dumb http. They cover admin-only access, creation, running twice, a new upstream commit arriving on checkDue with no pull opened, push refusal, the button, and the MCP argument. I did not call the MCP action end to end as an admin; the MCP test only checks the schema and that a non-admin is refused.
  • Docs: a note in docs/features/packs.md.

Left out

  • Other writes to a mirror, such as issues or hand-opened pulls, are not blocked. The issue asked only for push refusal.
  • If two packs name the same app, the first source wins.
  • An app whose upstream can't be reached is reported as failed and no repo is made, so running it again retries it.
  • The initial sync is not retried within the run; the schedule picks it up.

Try it

  • Sign in as an admin, open /packs, and press "Mirror the upstreams". It needs outbound access to github.com.
  • Over MCP, call repo_settings with {"mirrorPacks": true}.
  • Then git clone <forge>/packs/hextris.git.

🤖 Generated with Claude Code

commented

No code change this turn. The preview is up, and the deploy log you pasted doesn't show a failure from this pull.

  • The "failure": The deploy message says "running … no tests declared, none run". The app came up and the log is only routine forge output. The one ERROR line is the snapshot restore drill, "this forge has nowhere to drill from". It comes from the preview forge's setup and doesn't touch the packs code.
  • Checks on the pushed code:

- bun scripts/gate.ts passes. - bun x tsc --noEmit passes. - bun scripts/design-lint.ts passes. - 118 tests across 13 files pass, with 0 failures. They are test/packs-mirror.test.ts plus the packs, upstream-follow and repo-settings tests.

  • Not run here: scripts/fast.ts named 1711 of 2455 test files, so the diff is too broad to run in full. I ran the covering subset above instead. The forge's check runs the full suite on push.

What the pull does

  • Mirrors: src/packs/mirror.ts brings each pack's upstream in once as a read-only packs/<app> mirror. The scheduled upstream follow then re-syncs it.
  • How to trigger it: the admin button "Mirror the upstreams" on /packs, or the repo_settings tool with {"mirrorPacks": true}.
  • Other changes: the pull also touches src/tui/ssh.ts, src/http/server.ts and docs/features/packs.md. test/packs-mirror.test.ts covers the new behaviour.

Left out

It does not mirror anything on its own at boot. An admin has to start it.

commented

The check failed because src/packs/mirror.ts calls createUser, and the test that lists every createUser caller didn't know about it yet.

The call already declared its origin as seed, so no code change was needed. I added "src/packs/mirror.ts": ["seed"] to the expected list in test/signup-gate-parent.test.ts. That test now passes, and I've pushed the fix.

I ran only that test file. The full check will run again on the push.

Changes requested — before the latest push

Core design is sound (reuses the follow, SSRF-vetted ls-remote, admin-only on both doors, same-origin form read, idempotent). Changes needed, in one round:

  1. FIRST: no terminal door. What an admin can do on /packs must be doable in the terminal: add a screen in src/tui/ with its entry where AGENTS.md says, gated to admins (a guest/non-admin sees nothing), calling mirrorPacks; mention it in docs/features/packs.md.
  2. Tests missing: (a) the ssh push refusal in src/tui/ssh.ts has no test (box 2 says 'by anyone', and you added the branch); (b) the MCP test never runs the action as an admin, so a broken success path ({made,kept,failed,says}) passes; (c) the admin POST /packs/mirror success path (flash, mirrors made) is untested; the anon check only asserts >=400. Also tighten [403, 404] to the exact status for the owner/admin.
  3. Pushes are not the only writes: web file edits, contents/branch API writes and merging a hand-opened pull move a mirror's branch, and the follow force-moves the branch back, silently discarding them. Refuse write paths that move refs on a packs/* repo (or at least pulls/branch creation and contents writes), or say plainly in the docs which are open. 'Not done' in the doc is not enough for a security-relevant read-only guarantee.
  4. mirrorPacks runs serially inside one request: an ls-remote (60s timeout) plus a full fetch per app, for every pack source, in the HTTP request/MCP call. A few slow upstreams hang the admin request and a worker. Bound the total time, or create the repos and follow rows in the request and leave the first sync to the schedule (checkUpstream(...).catch(() => undefined) also swallows failures silently; record them).
  5. /packs form: the paragraph explains itself (principle 1) and links 'packs' to /packs, the page you are already on (wrong target). Cut it to one plain sentence and drop the self-link.
  6. Small: the import of ../packs/owner.ts in src/actions/upstream_follow.ts sits between git imports out of order; reportWords' local n helper is noise. A mirror's defaultBranch is fixed at creation while upstream HEAD may change; note or handle.

Boxes 1-3 are met by this head.

commented

I've addressed all six points and pushed. Of the tests I picked, the only failure was the palette roster test, which I fixed in the last commit. The forge's full check hasn't run on this head yet. bun scripts/gate.ts, tsc, design-lint and the packs, follow, MCP, TUI, palette and signup-gate tests pass locally. scripts/fast.ts said the diff was too broad to name tests, because canWrite touches nearly everything. The full suite has not run here, and the sandbox denied one loopback port during my test run.

  1. Terminal door. src/tui/screens/mirrorpacks.ts adds a "Mirror the packs' upstreams" screen, registered in entries.ts after packs. Only a non-agent admin is offered it, and Enter calls mirrorPacks. I added it to test/palette-roster.test.ts and documented it in docs/features/packs.md.
  2. Tests.

- ssh: a push is refused with the reason, and a clone is served. - MCP: an admin run is checked for {made,kept,failed,says}, and a non-admin is refused. - POST /packs/mirror: the anonymous POST is checked for 403/404 only, which is not an exact status. The admin POST redirects 303 to /packs with the report in the flash. The test pre-creates every shipped mirror, so it reaches no network and reports "0 brought in … already there". - Owner/admin push: the exact status is now 403. - Terminal: one test shows an admin is offered the screen and a non-admin is not. Another runs it end to end. - Writes: the contents API is refused for an admin, and canWrite is true for an ordinary repository's admin.

  1. Other writes. canWrite in src/core/repos.ts is now false for every packs/* repository except its owner, which only the follow acts as. That refuses web edits, the contents and branch API, and opening or merging a pull, admins included. The http and ssh push refusals now run before the canWrite check, so a pusher is told why instead of getting a 404. The docs say what stays open: issues and comments, and a repository admin's settings (visibility, delete, transfer).
  2. Bounded request.

- Request: it now only reads each upstream's default branch (8s each, 20s in all), creates the repo and its follow, and leaves the first sync to the schedule. An upstream not reached in time is listed as failed ("press again"). - Failures: the silent .catch is gone. A failed sync is noted on the follow's row. - Remaining cost: the one cost left in the request is the ls-remote for the default branch, capped at that budget.

  1. /packs form. The paragraph is now one sentence with no link.
  2. Small. I moved the import into order and dropped reportWords' n helper. I did not handle a renamed upstream HEAD. The docs say a mirror tracks the branch that was HEAD when it was made, and you delete and re-mirror to change it.
Approved

Every earlier ask is met: the terminal screen and palette entry, the MCP and ssh tests, the exact 403 on owner and admin pushes, canWrite closed on mirrors, a bounded request with no silent catch, a one-sentence /packs form, and the how-to in docs/features/packs.md rather than README. I could not diff against fc4c44de384d because the clone lacks that commit, so I read the whole change again. I found nothing high. The upstream URL goes through vettedEnv, the POST uses readForm with the admin scope, and every door checks admin and non-agent. Non-blocking: isPacksMirror matches on owner name alone, so keep packs reserved for org creation.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs