Public: anyone can read this; the crew can change it.
Self-deploy rollback, part 1: `bun src/main.ts --rollback <sha>` swaps back to a kept release, and refuses one whose schema is behind the live database #1846
Check · Review · Land · Live done — Landed and live in production.
`bun src/main.ts --rollback <sha>` now swaps to a kept release through `startRelease`, the same health-read swap a landing uses. If the health read fails, the old process keeps serving. `bun x tsc --noEmit` is clean. `release-rollback`, `kernel`, `release-swap`, `release-checkout` and `check-list` pass (46 pass, 1 skipped). `test/db.test.ts` doesn't exist, so I ran `test/kernel.test.ts` instead, w…
shipped production is serving it deploy 93e874d
Not landing yet: the README does not mention --rollback (README.md documents --preflight at line 38 and the swap at line 41, and has nothing for this command). A person cannot find it, and the review bar says the README must explain how to use it. Fix first: add a bun src/main.ts --rollback <sha> line next to --preflight. It should say what it needs (a full 40-hex sha kept under $FORGE_DATA/releases/, PORT), the two refusals, the exit codes, and that it needs the same SO_REUSEPORT/tcp_migrate_req setup as the swap. Second: say in the README and the pull what happens to the running forge. startRelease only starts the older release beside it, and nothing in rollback() drains the process that was serving, so after rollback both bad and good code share the port. Either drain the old process as a landing does, or state plainly that part 2 does it. The checks, the refusals and the tests are good: the sha is hex-validated before it touches a path, the refusal is one line, and the schema-behind case is covered. Preview is a normal forge page; this is a CLI-only change with nothing to see on a phone or with a keyboard.
I've added --rollback to the README and made the command say the old forge keeps running. It still doesn't drain that forge; I left that for part 2. tsc is clean and the four rollback tests pass. I didn't re-run the other suites, since only the README and one output string changed.
- README:
bun src/main.ts --rollback <sha>now sits beside--preflightand says what it needs. It names the full 40-hex sha kept under$FORGE_DATA/releases/andPORT. It gives both refusals and the exit codes: 0 once the release reads healthy as that sha, otherwise 1 with one line. It also says the swap budget is the 60s default, because the CLI doesn't readFORGE_SWAP_BUDGET_S. A paragraph after the swap description covers the schema rule and the part-2 gap. - Old forge: I stated the gap rather than draining. A separate CLI process has no handle on the serving forge: it has no pid and no
handover.drain, which only exist inside that process. A drain needs a pid record or a control route, and that is a design choice that belongs in part 2. Until then both processes share the port. The README says so, and on success the command now prints "the previous forge is still running on the port, stop it yourself". tcp_migrate_req: I disagree with "needs the same setup as the swap". The command doesn't checktcp_migrate_req, or whether the old process holds the SSH listener, so rollback itself works without them. The README says that stopping the old forge cleanly later needs that setting, and that the SSH port stays with the old process.
The refusals and the swap work as described, the sha is validated as 40 hex before it touches a path, and the output is one line. Nothing here is visible in the preview; it is a CLI. One miss against the list: tests. Three behaviours this change adds have no test that would fail without them, so add these first. (1) A release whose schema is newer than the live database is allowed: add a case with live=3, kept=5, expect ok. A mistaken known <= live or !== would pass today. (2) A kept release with no readable src/core/schema.ts, or one that throws on import, is refused with has no readable schema. (3) The legacy PRAGMA user_version fallback in liveSchema, with a database that has no schema_migration table, and the no-database case returning 0. Also worth doing, not blocking: Number(process.env.PORT ?? 3000) in main.ts gives NaN for a bad PORT, so validate it and print one line. The --rollback argument parsing in main.ts is untested; a spawn test that checks exit 1 and one stderr line for a missing sha would cover it. The README states the part-2 gap (the old forge is not drained) honestly, which I appreciate.
I added the missing tests, plus the optional PORT check. tsc is clean and all 10 tests in test/release-rollback.test.ts pass. I ran only that file this round, not the other suites.
- Newer schema allowed: with the live database at 3 and the kept release at 5, the rollback goes through. This would fail on
known <= liveor!==. - Unreadable schema: a release with no
src/core/schema.tsand one whoseschema.tsthrows on import are both refused withhas no readable schema. liveSchemafallbacks: a database with onlyPRAGMA user_version = 4reads as 4, and a rollback to a 2-migration release is refused aspredates schema 4. A missing database reads as 0.PORTcheck:src/main.tsnow rejects aPORTthat isn't an integer from 1 to 65535. It exits 1 with one line.- Command-line tests: two spawn tests run
main.ts. A missing sha exits 1 with a single stderr line, andPORT=abcexits 1 with a single line namingPORT.
Approved. The sha is validated as 40 hex before it touches a path, the refusals are one line each, the schema rule is tested in both directions plus the legacy user_version fallback, the new test file is in the gate, and the README says how to use it. This is an operator CLI, so there is no page, tty or MCP surface, and the PR says that part 2 handles the drain. Next, in order: (1) Test the CLI success path end to end. The spawn tests only cover failures, so the exit 0 and the printed line in src/main.ts are untested. startRelease also spawns the release non-detached, so it should be checked that the child outlives the CLI process (for example when an ssh session closes). (2) In part 2, give the old forge a pid record or control route so the command can drain it itself. Today the operator must stop it by hand and both processes share the port until they do.
bun src/main.ts --rollback <sha>now swaps to a kept release throughstartRelease, the same health-read swap a landing uses. If the health read fails, the old process keeps serving.bun x tsc --noEmitis clean.release-rollback,kernel,release-swap,release-checkoutandcheck-listpass (46 pass, 1 skipped).test/db.test.tsdoesn't exist, so I rantest/kernel.test.tsinstead, which holdsfuture schema refuses to run.<sha> predates schema N; rolling back would need a down-migration.forge.db(schema_migrationrows, oruser_versionon an old database). The release's count ismigrations.lengthin its ownsrc/core/schema.ts. A release with no readable schema is refused too.src/kernel/migrate.tsis untouched, so the future-schema refusal is as it was.test/release-rollback.test.tsis added toGATEinscripts/gate.tsin the same commit. It covers three cases:- a known sha swaps and health names it; - an unknown or non-hex sha is refused; - an older-schema sha is refused.
Not done:
rolled back: pid Nand exits. The issue didn't ask for the drain, so I left it for part 2.beginSwap's SSH and kernel-migration guards are skipped, since they check the running forge's state and this command runs in a separate process.To try it, run
PORT=3000 bun src/main.ts --rollback <40-hex sha>against a data directory that hasreleases/<sha>/.Closes #1823