deploy_pull is trimmed to recent deploys and rebuilt from git when an older one is asked for #12022

deploy_pull now keeps only rows for recent deploys. When someone asks for an older deploy whose rows were trimmed, its list of carried pulls is rebuilt from git and stored again.

• Rebuild path: carriedFor(db, run, dir) in src/core/carried.ts returns the stored rows. If a finished, successful production deploy has none, it runs recordCarried against git, stores the result and returns it. The run page (src/web/studio/run.ts) and GET /api/v1/repos/:owner/:name/runs/:id (which becomes async) both read through it. Preview and failed deploys are never walked.
• Retention: the new src/system/retention_deploys.ts adds pruneDeploys(db, now, rowCap), which returns {deleted, more}.

- It keeps rows for deploys from the last 14 days (DEPLOY_PULL_KEEP_DAYS, longer than the 7-day ask-to-live window). - For every repository with an app, it also keeps the newest successful production deploy. Live and sleeping apps both run that deploy, since the app table stores no running-deploy id. - Each batch reads the rows to delete first, then deletes them by rowid in one short transaction.

• Schedule: deployRetention runs from the crew tick (every 5 s) in src/crew/dispatch.ts. It runs a batch on each tick while the last batch was full, otherwise once an hour. I used this tick rather than the hourly clone sweep because 5,000 rows an hour would never drain 9.9M rows; at one batch per tick it takes about 3 hours.
• Tests: test/retention-deploys.test.ts covers:

- Delete a deploy's rows, ask through carriedFor, get the same list back from git. - Preview deploys are not walked. - Old rows go, while recent rows and the app's running deploy stay. - A 5,000-row batch finishes in under 50 ms (timed over the whole call, read included). - The tick keeps going while batches are full and waits an hour once one isn't.

Not done here:

• Rows whose run was deleted are not swept, because finding them means scanning the whole table.
• The TUI deploys screen still uses carriedByRuns without a rebuild, so a trimmed old deploy there shows no "carries".
• No admin setting and nothing on /system yet; those are later items of #11973.
• The issue asks for a plan kept with keep_plan, but that tool wasn't available in this session, so no plan was posted.

Checks: bun scripts/gate.ts and bun x tsc --noEmit are clean. scripts/fast.ts named 2,303 of 3,001 test files, too many to run here, so I picked the ones that cover the change: the new suite, the carried, run page, runs API and crew-tick tests — 18 files, all passing.

To try it: on a production deploy older than 14 days, open /<owner>/<repo>/runs/<id>. It still lists the pulls it carries.

🤖 Generated with Claude Code

Closes #12009

deploy_pull now keeps only rows for recent deploys. When someone asks for an older deploy whose rows were trimmed, its list of carried pulls is rebuilt from git and stored again.

• Rebuild path: carriedFor(db, run, dir) in src/core/carried.ts returns the stored rows. If a finished, successful production deploy has none, it runs recordCarried against git, stores the result and returns it. The run page (src/web/studio/run.ts) and GET /api/v1/repos/:owner/:name/runs/:id (which becomes async) both read through it. Preview and failed deploys are never walked.
• Retention: the new src/system/retention_deploys.ts adds pruneDeploys(db, now, rowCap), which returns {deleted, more}.

- It keeps rows for deploys from the last 14 days (DEPLOY_PULL_KEEP_DAYS, longer than the 7-day ask-to-live window). - For every repository with an app, it also keeps the newest successful production deploy. Live and sleeping apps both run that deploy, since the app table stores no running-deploy id. - Each batch reads the rows to delete first, then deletes them by rowid in one short transaction.

• Schedule: deployRetention runs from the crew tick (every 5 s) in src/crew/dispatch.ts. It runs a batch on each tick while the last batch was full, otherwise once an hour. I used this tick rather than the hourly clone sweep because 5,000 rows an hour would never drain 9.9M rows; at one batch per tick it takes about 3 hours.
• Tests: test/retention-deploys.test.ts covers:

- Delete a deploy's rows, ask through carriedFor, get the same list back from git. - Preview deploys are not walked. - Old rows go, while recent rows and the app's running deploy stay. - A 5,000-row batch finishes in under 50 ms (timed over the whole call, read included). - The tick keeps going while batches are full and waits an hour once one isn't.

Not done here:

• Rows whose run was deleted are not swept, because finding them means scanning the whole table.
• The TUI deploys screen still uses carriedByRuns without a rebuild, so a trimmed old deploy there shows no "carries".
• No admin setting and nothing on /system yet; those are later items of #11973.
• The issue asks for a plan kept with keep_plan, but that tool wasn't available in this session, so no plan was posted.

Checks: bun scripts/gate.ts and bun x tsc --noEmit are clean. scripts/fast.ts named 2,303 of 3,001 test files, too many to run here, so I picked the ones that cover the change: the new suite, the carried, run page, runs API and crew-tick tests — 18 files, all passing.

To try it: on a production deploy older than 14 days, open /<owner>/<repo>/runs/<id>. It still lists the pulls it carries.

🤖 Generated with Claude Code

This pull now keeps deploy_pull rows only for recent deploys and for the deploy an app is currently running. When someone opens an older deploy, the run page and the run API rebuild its pull list from git.

Fix for the red check: test/retention-deploys.test.ts had a took < 50 wall-clock check, which test/flake.test.ts rejects (#4429, #7824). It now counts the SQL statements one 5,000-row tick prepares, using once() from test/helpers/statements.ts, and caps them with expectWork(cost, { max: 4 }). If the delete ever ran one prepared statement per row, that count would rise and the test would fail; a clock wouldn't reliably catch that. The batch results it checks are unchanged: 5,000 deleted with more: true, then 2,000 with more: false, then a wait until the hour.

The other failures in that check, left alone:

• test/spawn-stopped-restart.test.ts:131 (the polling loop that flake.test.ts rejects) came in on main with 2bd5aeb9b. The check itself says those flake.test lines are red on main too.
• test/pages-budget.test.ts is red on main at c4449b7.
• test/flow-cache.test.ts hit "SQLiteError: out of memory" in shard 2. Here it passes 9/9.
• The MCP system stalls test in shard 55 doesn't touch anything this pull changes.

Checks run here: bun scripts/gate.ts and bun x tsc --noEmit came back clean. bun test test/retention-deploys.test.ts test/flake.test.ts gives 39 pass and 1 fail, the one fail being the spawn-stopped-restart line from main above.

To try it: on the preview, open a production deploy older than 14 days at /<owner>/<repo>/runs/<id>. It should still list the pulls it carries.

🤖 Generated with Claude Code

deploy_pull now keeps only the rows of deploys from the last 14 days and of the deploy each app is currently running. A crew tick deletes the rest in batches of at most 5,000 rows (src/system/retention_deploys.ts). When someone asks for an older production deploy, its pulls are rebuilt from git and saved again, the same way through all three doors: the run page, GET /api/v1/repos/:owner/:name/runs/:id and the terminal's deploys screen.

Changes from review:

• Repeated git walks on GET (HIGH): carriedFor in src/core/carried.ts now shares one walk between views of the same run that arrive together.

- A deploy that carries no pulls is walked once and the empty answer is remembered for the life of the process, for up to 10,000 runs per database. A finished deploy's commit and the pulls merged before it don't change, so an empty answer stays correct. - If git can't walk the commit, the forge waits 60 s before trying again rather than remembering that run as carrying none. - Nothing is written for an empty answer. A row marking it would need a schema migration, and an in-process cache keeps this pull free of one. After a restart each such deploy is walked once more.

• Terminal door: the deploys screen paints the saved rows first. If any deploy on the list has none, it calls the new carriedForRuns, which rebuilds them one at a time, then redraws. Its load waits for that rebuild.
• How-to: added docs/features/deploy-pull-retention.md.
• Tests: test/retention-deploys.test.ts has one test per door (API, run page, terminal), one proving an empty deploy is not walked again, and one proving two views at once share a walk.

- The first three fail if that door's call goes back to carriedBy. The other two fail without the remembered empty answer and the shared walk. I checked each by making that change and running the file.

• Lock time: I didn't add a clock to the test, because test/flake.test.ts rejects any wall-clock bound (#4429, #7824), and a 50 ms timer was what turned the last check red.

- What the test shows instead: the batch reads its candidate rows outside any write transaction, then deletes them by rowid in one transaction. That transaction holds at most 5,000 deletes, and the test counts that a whole tick prepares no more than 4 SQL statements. - This bounds the work done under the lock without timing it. It doesn't check the issue's literal "under 50 ms".

• Direction: this comes from #11973. Its measurements on a production copy show forge.db at 4.56 GB, over Bun's 2 GiB mmap cap, with deploy_pull the largest table at 9.9M rows / 906 MB. That issue also reports the forge's one thread blocked for most of every minute, which slows everything, pipeline work included.

- The issue is open and was ready to take. If the owner wants it held behind #6534 and the rest of that list, this pull can wait without harm: nothing else depends on it.

Checks run here: bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. scripts/fast.ts named 2,303 files, so I chose the covering tests myself. Across retention-deploys, all the tui-deploys, carried and flow-carried files, flake, readme-cap, features-empty, crew-brief-docs, both-doors and the run-page tests: 113 pass and 1 fails. The failure is the test/spawn-stopped-restart.test.ts:131 rule in flake.test, which is red on main from 2bd5aeb9b.

To try it: on the preview, open a production deploy older than 14 days at /<owner>/<repo>/runs/<id>, or that app's deploys screen in the terminal. Both list the pulls it carries.

🤖 Generated with Claude Code

deploy_pull now keeps only the rows of deploys from the last 14 days and of the deploy each app is currently running. A crew tick deletes the rest in batches of at most 5,000 rows (src/system/retention_deploys.ts). When someone asks for an older production deploy, its pulls are rebuilt from git and saved again, the same way through all three doors: the run page, GET /api/v1/repos/:owner/:name/runs/:id and the terminal's deploys screen.

Gate fix: two comments in src/core/carried.ts are rewrapped to 100 columns. The latest commit carries a Direction: line, which the "pipeline first" rule asks for. bun scripts/gate.ts is now clean.

What to look at:

• No repeated git walks on GET: carriedFor in src/core/carried.ts shares one walk between views of the same run that arrive together.

- A deploy that carries no pulls is walked once and the empty answer is remembered for the life of the process, for up to 10,000 runs per database. A finished deploy's commit and the pulls merged before it don't change, so the empty answer stays correct. - If git can't walk the commit, the forge waits 60 s before trying again rather than remembering that run as carrying none. - Nothing is written for an empty answer. A row marking it would need a schema migration, and the in-process cache keeps this pull free of one. After a restart each such deploy is walked once more.

• Terminal door: the deploys screen paints the saved rows first. If any deploy on the list has none, it calls carriedForRuns, which rebuilds them one at a time, then redraws.
• How-to: docs/features/deploy-pull-retention.md.
• Tests: test/retention-deploys.test.ts has one test per door (API, run page, terminal), one proving an empty deploy is not walked again, and one proving two views at once share a walk.

- The first three fail if that door's call goes back to carriedBy. The other two fail without the remembered empty answer and the shared walk. I checked each by making that change and running the file.

• Lock time: the test doesn't time the batch, because test/flake.test.ts rejects any wall-clock bound (#4429, #7824).

- Instead it counts SQL statements: a whole tick prepares no more than 4. - The batch reads its candidate rows outside any write transaction, then deletes them by rowid in one transaction of at most 5,000 deletes. - This bounds the work done under the lock, but doesn't check the issue's literal "under 50 ms".

• Direction: this comes from #11973. Its measurements on a production copy show forge.db at 4.56 GB, over Bun's 2 GiB mmap cap, with deploy_pull the largest table at 9.9M rows / 906 MB. That issue also reports the forge's one thread blocked for most of every minute, which slows the pipeline work as well. Nothing depends on this pull, so it can be held behind #6534 and the rest of that list if the owner prefers.

Checks run here:

• Last change: after the rewrap, bun scripts/gate.ts is clean, bun x tsc --noEmit passes, and test/retention-deploys.test.ts passes 9/9.
• Previous push:

- bun scripts/design-lint.ts was clean. - scripts/fast.ts named 2,303 files, too many to run here, so I chose the covering tests myself: retention-deploys, all the tui-deploys, carried and flow-carried files, flake, readme-cap, features-empty, crew-brief-docs, both-doors and the run-page tests. - Results: 113 pass and 1 fails. The failure is the test/spawn-stopped-restart.test.ts:131 rule in flake.test, which is red on main from 2bd5aeb9b.

To try it: on the preview, open a production deploy older than 14 days at /<owner>/<repo>/runs/<id>, or that app's deploys screen in the terminal. Both list the pulls it carries.

🤖 Generated with Claude Code

Left #12009 open: 1 unticked item remain (A 5,000-row batch holds the write lock under 50 ms in the test.).