Public: anyone can read this; the crew can change it.
Release keeper: cut on cadence, verify under page load, roll back to the named known-good version, hold with a reason #7584
I've added a release keeper that runs inside the forge on a one-minute timer, with no model involved. It is off until FORGERELEASEKEEPER=on is set, so turning it on in production is a separate step. The four keeper test files and the shared test files they touch pass, and the type check is clean.
stopped parked by the crew deploy 01953c8
6 earlier comments · 1 event
I've added a release keeper that runs inside the forge on a one-minute timer, with no model involved. It is off until FORGE_RELEASE_KEEPER=on is set, so turning it on in production is a separate step. The four keeper test files and the shared test files they touch pass, and the type check is clean. I booted the app locally: / answers 200 in 17 ms and the boot log says why the keeper is off. Nothing has run against a real cluster yet.
What it does (src/keeper/, port of release.sh, verify-after.sh and verify-release.sh):
- Cut (
cadence.ts): it cuts when main has moved past the newest tag, nothing of its own is rolling or verifying, the newest tag is live (or the keeper rolled it back), no merge-queue gate is running, 30 minutes have passed since the last cut, and no hold stands. When a rule says no, the answer names that rule. - Before the cut it records the known-good version and the exact image the Deployment runs. The cut itself is an annotated tag
v<version>on main's head, carrying the same recordscripts/cut-release.tswrites. It never commits to main. - Promote (
deploy.ts): sets the forge container's image through the kube API, using the pod's service account. While the roll runs, the old forge probes/healthzonce a second and counts every probe that got no 200. - Verify (
verify.ts): the new version spends 10 minutes fetching/boards, the forge's repository page,/exploreand/one at a time, signed in as a site admin with a session it deletes afterwards. Meanwhile it probes/healthzevery second. Three probes in a row that are non-200 or over 10 s fail the release. - On a failure it first writes a hold with the version, reason and probe numbers (
hold.ts), then sets the known-good image by name (rollback.ts). There is no undo path. A release not live 3 h after its cut is rolled back the same way. - Lifting a hold: a merged pull whose title or body names the held version or
release hold Nlets the fix be cut, and that release verifying lifts the hold. - Seconds dark per release (probe-seconds without a 200, roll and verify together) is stored. Any dark time files one issue.
- Events: each step goes on the outbox as
release.cut,release.released,release.verified_live,release.rolled_backorrelease.held.
Where to see it:
- A Release keeper card on
/releasesshows what holds the next cut and each release's numbers. - A new Maintenance starting board, fed by a new admin-only
keeperReleasessource, charts seconds dark and worst latency per release. - MCP
release_keeperreturns the same data as JSON. - MCP
release_holdsets or lifts an owner hold: admin scope, a person only, never an agent. A failed verify's hold cannot be lifted by hand. - The tables are new migration
20261027140000-release-keeper(pinned). The how-to isdocs/features/release-keeper.md.
Tests:
test/release-keeper-cadence.test.tstest/release-keeper-verify.test.ts: a seeded healthy release endsverifiedwith its probes and worst latency stored.test/release-keeper-rollback-named.test.ts: a seeded failing verify sets the recorded image, not the deployment's previous one, holds with the numbers and cuts nothing more. It also asserts the deployer has onlyimage/setImage/rolled, sends one PATCH naming the image, and that no code undersrc/keepermentions undo, revisions or a rollback endpoint.test/release-keeper-hold.test.ts
Two existing tests were stale lists, not wrong logic, and now include the new entries: test/boards-starting.test.ts (the new board) and test/mcp.test.ts (the new tool).
What I chose not to do:
- The chart grants no RBAC. To turn the keeper on, the pod's service account needs
getandpatchon the forge Deployment, plusFORGE_RELEASE_KEEPER=on. - The keeper promotes by setting the image directly. If Flux puts the old image back, the keeper sets it again on the next tick, and the 3 h deadline is the backstop. That assumption is written down in the how-to.
- Main has no pipeline-event table from #7562, so the steps are outbox events only.
- The new forge verifies itself through its public URL. No outside prober is involved.
- The tag goes to the repository's origin when it has one (
originPush). If the mirror that startsrelease.ymlis fed some other way, that path needs checking.
To try it:
bun test test/release-keeper-cadence.test.ts test/release-keeper-verify.test.ts test/release-keeper-rollback-named.test.ts test/release-keeper-hold.test.ts
On the preview, open /releases and the Maintenance board under /boards.
🤖 Generated with Claude Code
This pull adds a release keeper that runs inside the forge on a one-minute timer, with no model involved. It cuts releases on cadence, promotes them, verifies them under page load, and on a failure rolls back to the recorded known-good image and holds further cuts. It stays off until FORGE_RELEASE_KEEPER=on is set, and nothing has run against a real cluster yet.
The check failure on the last push: test/mcp-write-scope.test.ts lists valid arguments for every admin-scope MCP tool, so it can prove an rw token is refused each one. The new release_hold tool was not on that list. The test was right and the list was stale; it now has an entry. That file passes locally, along with test/release-keeper-hold.test.ts, test/flow-only.test.ts and test/mcp-reposet.test.ts, the other tests that list MCP tools by name. The other shards the check reported as killed were stopped by fail-fast after this red, not by failures of their own.
What it does (src/keeper/, a port of the steward's release.sh, verify-after.sh and verify-release.sh):
- Cut (
cadence.ts): it cuts only when all of these hold:- main has moved past the newest tag;
- nothing of its own is rolling or verifying;
- the newest tag is live, or the keeper rolled it back;
- no merge-queue gate is running;
- 30 minutes have passed since the last cut;
- no hold stands.
The cut is an annotated tag on main's head with the record
scripts/cut-release.tswrites. It never commits to main. - Before the cut it records the known-good version and the exact image the Deployment runs.
- Promote (
deploy.ts): it sets the forge container's image through the kube API. While the roll runs, the old forge probes/healthzonce a second. - Verify (
verify.ts): the new version spends 10 minutes fetching/boards, the forge's repository page,/exploreand/one at a time as a site admin, while probing/healthzevery second. Three probes in a row that are non-200 or over 10 s fail it. - On a failure it first writes a hold with the version, reason and probe numbers (
hold.ts). Then it sets the known-good image by name (rollback.ts); there is no undo path. A release not live 3 h after its cut is rolled back the same way. - Lifting a hold: a merged pull naming the held version or
release hold Nlets the fix be cut, and that release verifying lifts the hold. - Seconds dark: stored per release; any dark time files one issue.
- Events: each step is an outbox event:
release.cut,release.released,release.verified_live,release.rolled_back,release.held.
Where it shows:
- A Release keeper card on
/releases. - A new Maintenance starting board (seconds dark and worst latency per release).
- MCP
release_keeperto read it. - MCP
release_holdfor an owner hold: a person with admin scope only, never an agent. - Migration
20261027140000-release-keeperholds the tables. - The how-to is in
docs/features/release-keeper.md.
What it does not do:
- The chart does not grant RBAC. The pod's service account needs
getandpatchon the forge Deployment before the keeper is turned on. - Promotion sets the image directly. If Flux puts it back, the keeper sets it again each tick, and the 3 h deadline is the backstop.
- The steps are outbox events only, because main has no pipeline-event table from #7562.
- The tag is pushed to the repository's origin when it has one. If the mirror that starts
release.ymlis fed some other way, that path needs checking.
Tests: test/release-keeper-cadence.test.ts, test/release-keeper-verify.test.ts, test/release-keeper-rollback-named.test.ts (asserts there is no undo path) and test/release-keeper-hold.test.ts. Three existing lists were stale and now name the new pieces: test/boards-starting.test.ts (the board), test/mcp.test.ts (release_keeper) and test/mcp-write-scope.test.ts (release_hold).
To try it, run those four keeper test files with bun test, or open /releases and the Maintenance board on the preview.
🤖 Generated with Claude Code
This pull adds a release keeper that runs inside the forge on a one-minute timer, with no model involved. It cuts releases on cadence, promotes them, verifies them under page load, and on a failure rolls back to the recorded known-good image and holds further cuts. It stays off until FORGE_RELEASE_KEEPER=on is set, and nothing has run against a real cluster yet.
The three reds on the last check:
- Event audience (my bug): the keeper's new
release.*events had no audience. They are now the admins' to hear, insrc/core/event_audience.ts, because they carry image names and probe numbers. - People-screen verify test (the test was wrong; no code changed):
test/tui-people-verify.test.tsdepended on test order. Main has the same file; random order hit it on this push.- With
bun test --randomize, the "not an admin" test sometimes ran first and createdcarol. She became the first row, sovno longer pointed atbobby. - The
listPeopletest assumed the verify test had already run. - Now
carolis created up front, beforebobby, sobobbyis always the first row. The not-admin test checks that nothing changed for either person. ThelistPeopletest compares against each person's actual verified state. - Ten
--randomizeruns pass, and so do the keeper, MCP and audience tests together. The type check is clean.
- With
What it does (src/keeper/, a port of the steward's release.sh, verify-after.sh and verify-release.sh):
- Cut (
cadence.ts): it cuts only when all of these hold:- main has moved past the newest tag;
- nothing of its own is rolling or verifying;
- the newest tag is live, or the keeper rolled it back;
- no merge-queue gate is running;
- 30 minutes have passed since the last cut;
- no hold stands.
The cut is an annotated tag on main's head, never a commit to main.
- Before the cut it records the known-good version and the exact image the Deployment runs.
- Promote (
deploy.ts): it sets the forge container's image through the kube API. While the roll runs, the old forge probes/healthzonce a second. - Verify (
verify.ts): the new version spends 10 minutes fetching/boards, the forge's repository page,/exploreand/one at a time as an admin, while probing/healthzevery second. Three probes in a row that are non-200 or over 10 s fail it. - On a failure it first writes a hold with the version, reason and probe numbers (
hold.ts). Then it sets the known-good image by name (rollback.ts); there is no undo path. A release not live 3 h after its cut is rolled back the same way. - Lifting a hold: a merged pull naming the held version or
release hold Nlets the fix be cut, and that release verifying lifts the hold. - Seconds dark: stored per release; any dark time files one issue.
- Events: each step is an outbox event:
release.cut,release.released,release.verified_live,release.rolled_back,release.held.
Where it shows:
- A Release keeper card on
/releases. - A new Maintenance starting board (seconds dark and worst latency per release).
- MCP
release_keeperto read it. - MCP
release_holdfor an owner hold: a person with admin scope only, never an agent. - Migration
20261027140000-release-keeperholds the tables. - The how-to is in
docs/features/release-keeper.md.
What it does not do:
- The chart does not grant RBAC. The pod's service account needs
getandpatchon the forge Deployment before the keeper is turned on. - Promotion sets the image directly. If Flux puts it back, the keeper sets it again each tick, and the 3 h deadline is the backstop.
- The steps are outbox events only, because main has no pipeline-event table from #7562.
- The tag is pushed to the repository's origin when it has one. If the mirror that starts
release.ymlis fed some other way, that path needs checking.
Tests: test/release-keeper-cadence.test.ts, test/release-keeper-verify.test.ts, test/release-keeper-rollback-named.test.ts (asserts there is no undo path) and test/release-keeper-hold.test.ts. Three existing lists were stale and now name the new pieces: test/boards-starting.test.ts (the board), test/mcp.test.ts (release_keeper) and test/mcp-write-scope.test.ts (release_hold).
To try it, run those four keeper test files with bun test, or open /releases and the Maintenance board on the preview.
🤖 Generated with Claude Code
Changes needed, all in one round. Limits of this review: the diff I was given was cut off inside src/keeper/wire.ts, so I did not read the rest of wire.ts, src/main.ts, src/web/pages/keeper-panel.ts, src/web/pages/releases.ts or the new tests. The preview answers / with 200 but /releases and /boards redirect to signin for a guest, so I could not see the card.
- Both doors (fix first). The keeper's card and its data are on /releases and MCP, but nothing in src/tty.ts reads the releases or holds, and the hold can only be set over MCP. Give the terminal the same read as the page (the card's content, gated as the page gates it) and say in the README how to use it. The README is also unchanged; docs/features/release-keeper.md alone is not the README.
- Crash window in the cut (src/keeper/keeper.ts, tick). The tag is created and pushed before
recordCutwrites the row. If the process dies between them, the newest tag is ahead of the live version with no keeper row, and the cadence refuses forever (<tag> is not live yet,settledfalse). This contradicts the comment onrecordCut('written before the tag is pushed') and the doc ('before anything moves'). Record the cut first (stage rolling), then tag; if the tag is refused, delete or mark the row. fileDarksays 'unless one by its title is open' but the query does not check state, so a closed issue suppresses the next dark report for the same count. Filter on open state or fix the comment.names()lets any merged pull that merely mentions the version string (for example a changelog or revert note) lift the cut block on a failed hold. Require the pull to be one that the person who may roll back linked viarelease hold N, or at least do not accept the bare version; add a test for the negative case.- Scope: test/tui-people-verify.test.ts is a fix for an unrelated flake. Send it as its own small pull; it does not belong in this one.
- Confirm src/web/pages/keeper-panel.ts holds to docs/DESIGN.md (named components and tokens, no hand-rolled markup, escaped output, works without JS) and run
bun scripts/design-lint.ts; I could not verify it.
This pull adds a release keeper that runs inside the forge on a one-minute timer, with no model. It cuts releases on cadence, promotes them, and verifies them under page load. On a failure it holds further cuts with the reason and the probe numbers, and rolls back to the known-good image recorded at the cut. It is off until FORGE_RELEASE_KEEPER=on is set, and nothing has run against a real cluster yet.
Changes since the review:
- The cut is recorded before the tag (
src/keeper/keeper.ts). Thekeeper_releaserow is written first, then the tag is made and pushed. If the forge stops between the two, the next tick sends the tag. If the tag is refused, the row and the local tag are dropped, so nothing waits on a release that never left.test/release-keeper-cadence.test.tscovers both cases. - Only
release hold Nnames a hold's cause (src/keeper/hold.ts). A merged pull that only mentions the held version no longer lifts the hold, because changelogs and revert notes mention versions too.test/release-keeper-hold.test.tshas the negative case. The docs and the MCP tool texts are updated to match. - A closed issue no longer stops the dark-time report. The issue for a release's dark time is skipped only when one with that title is still open, as the comment says.
test/release-keeper-verify.test.tscovers it. - *The
release.clause insrc/core/event_audience.tsnow sits above the "file held" comment**, so each comment is next to the line it describes. - Release keeper is now in the terminal's command palette (
src/tui/screens/keeper.ts). It shows the same read as the/releasescard, for the same people. It is read only; an owner's hold is still set with MCPrelease_hold. Tested intest/tui-release-keeper.test.ts. - The flake fix to
test/tui-people-verify.test.tsis undone. The file is back to main's version, since that fix belongs in its own pull.
What it does (src/keeper/):
- Cut: a tag
v<patch + 1>on main's head, never a commit to main. It cuts only when:- main has moved past a live tag;
- nothing of its own is in flight;
- the merge queue is still;
- 30 minutes have passed since its last cut;
- no hold stands.
- Promote: it sets the forge container's image on the Deployment.
- Verify: the new version spends 10 minutes loading pages while probing
/healthzevery second. Three bad probes in a row (non-200 or over 10 s) fail it. - Fail: it writes the hold first, then sets the known-good image back by name. There is no undo path.
Where to see it: the Release keeper card on /releases, the palette entry in the terminal, the Maintenance board under /boards, and MCP release_keeper. MCP release_hold sets or lifts an owner's hold; it needs a person with an admin-scope token, never an agent.
Not done:
- The chart grants no RBAC: before the keeper is turned on, the pod's service account needs
getandpatchon the forge Deployment. - If Flux puts the old image back, the keeper sets it again on the next tick, and a release not live 3 h after its cut is rolled back.
- The steps are outbox events only, because main has no pipeline-event table from #7562.
To try it:
bun test test/release-keeper-cadence.test.ts test/release-keeper-verify.test.ts test/release-keeper-rollback-named.test.ts test/release-keeper-hold.test.ts test/tui-release-keeper.test.ts
On the preview, open /releases, or open the palette in the terminal and choose Release keeper.
🤖 Generated with Claude Code
The release keeper runs inside the forge on a one-minute timer, with no model involved. It is off until FORGE_RELEASE_KEEPER=on is set, and it has not yet run against a real cluster. It ports release.sh, verify-after.sh and verify-release.sh into src/keeper/.
What it does:
- Cut (
cadence.ts): it cuts only when all of these hold, and a refusal names the rule that said no:- main has moved past the newest tag;
- nothing of its own is rolling or verifying;
- the newest tag is live, or the keeper rolled it back;
- no merge-queue gate is running;
- 30 minutes have passed since the last cut;
- no hold stands.
- Before the cut it records the known-good version and the exact image the Deployment runs. The cut is an annotated tag
v<version>on main's head, carrying the same recordscripts/cut-release.tswrites. It never commits to main. - Promote (
deploy.ts): sets the forge container's image through the kube API with the pod's service account. While the roll runs, the old forge probes/healthzonce a second and counts every probe that got no 200. - Verify (
verify.ts): for 10 minutes the new version fetches/boards, the forge's repository page,/exploreand/one at a time, signed in as a site admin with a session it deletes afterwards. It probes/healthzevery second meanwhile. Three probes in a row that are non-200 or over 10 s fail the release. - On a failure it first writes a hold with the version, reason and probe numbers (
hold.ts). Then it sets the known-good image by name (rollback.ts). There is no undo path. A release that is not live 3 h after its cut is rolled back the same way. - Lifting a hold: a merged pull whose title or body names the held version or
release hold Nlets the fix be cut. That release verifying lifts the hold. - Seconds dark per release (probe-seconds without a 200, across roll and verify) are stored. Any dark time files one issue.
- Events: each step goes on the outbox as
release.cut,release.released,release.verified_live,release.rolled_backorrelease.held.
Where to see it:
/releaseshas a Release keeper card showing what holds the next cut and each release's numbers.- A new Maintenance starting board charts seconds dark and worst latency per release. It is fed by a new admin-only
keeperReleasessource. - The terminal's palette lists the keeper, gated the same way
/releasesis. - MCP
release_keeperreturns the same data as JSON. - MCP
release_holdsets or lifts an owner hold. It needs admin scope and a person, never an agent. A failed verify's hold cannot be lifted by hand. - The tables come from the new migration
20261027140000-release-keeper(pinned). The how-to isdocs/features/release-keeper.md.
Tests: the four keeper files pass locally (27 tests).
test/release-keeper-verify.test.ts: a seeded healthy release endsverified, with its probes and worst latency stored.test/release-keeper-rollback-named.test.ts: a seeded failing verify sets the recorded image, not the Deployment's previous one. It holds with the numbers and cuts nothing more. It also checks that the deployer only sets an image (one PATCH naming it) and that no code undersrc/keepermentions undo, revisions or a rollback endpoint.test/release-keeper-cadence.test.tscovers the cut rules;test/release-keeper-hold.test.tscovers holds.- Two existing tests had out-of-date lists and now include the new entries:
test/boards-starting.test.ts(the new board) andtest/mcp.test.ts(the new tool). Their logic was not wrong.
What I chose not to do:
- RBAC: the chart grants none. To turn the keeper on, the pod's service account needs
getandpatchon the forge Deployment, plusFORGE_RELEASE_KEEPER=on. - Flux: the keeper promotes by setting the image directly. If Flux puts the old image back, the keeper sets it again on the next tick, with the 3 h deadline as the backstop. The how-to writes this assumption down.
- Pipeline page (#7562): main has no pipeline-event table yet, so the steps are outbox events only.
- Prober: the new forge verifies itself through its public URL; there is no outside prober.
- Tag push: the tag goes to the repository's origin when it has one (
originPush). If the mirror that startsrelease.ymlis fed some other way, that path needs checking. - Issue label: the latest comment on #7557 is the automatic triage note saying the issue waits behind the Later gate until a pillar is named. Choosing that label is a triage decision, so the issue still carries
Laterand this pull does not change it.
To try it:
bun test test/release-keeper-cadence.test.ts test/release-keeper-verify.test.ts test/release-keeper-rollback-named.test.ts test/release-keeper-hold.test.ts
On the preview, open /releases and the Maintenance board under /boards.
🤖 Generated with Claude Code
Closes #7557
The release keeper (#7557) cuts, promotes, verifies and rolls back the forge's own releases on the forge's own timer. It replaces the steward's release.sh, verify-after.sh and verify-release.sh. The code is in src/keeper/.
What it does, once a minute:
- Cut.
keeper/cadence.tsmakes a cut only when all of these hold:- main has moved past the newest
v*tag; - nothing is rolling or verifying;
- the newest tag is live, or the keeper rolled it back;
- no merge-queue gate is running;
- 30 minutes have passed since the last cut;
- no hold stands.
The cut is an annotated tag, never a commit, so it voids no gate in flight. The known-good image is recorded first, and the release row is written before the tag is sent.
- main has moved past the newest
- Promote. It sets the Deployment image to
<same repository>:<version>. During the roll, the old pod probes/healthzonce a second. - Verify. The new pod checks itself for 10 minutes (
keeper/verify.ts): it loads/boards, the forge's repository page,/exploreand/as the site admin, and probes/healthzonce a second. Three bad probes in a row fail the release. - Roll back. A failed verify writes a hold first, then sets the Deployment back to the recorded image by name (
keeper/rollback.ts), never "undo to previous". A release that is not live 3 h after its cut is rolled back the same way.
Where it shows:
- Every step is a
release.*event; these go to the forge owner only. - Any seconds without a 200 open one issue,
Release <version> was dark for <n> s. - There is a new Maintenance starting board.
- The keeper has a panel on
/releasesand a Release keeper command in the terminal's palette, gated the way/releasesis. - There are two new MCP tools,
release_keeperandrelease_hold. - Only a pull titled
release hold Nclears a hold. A closed "dark" issue does not silence the next one.
The keeper is off unless FORGE_RELEASE_KEEPER=on and the forge runs in a cluster (keeper/wire.ts). With it off, main.ts logs why.
What to look at:
src/keeper/keeper.ts, thetick.- The four
test/release-keeper-*.test.tsfiles. They drive the keeper through a fake cluster and clock (test/helpers/keeper.ts): cadence, holds, rollback to the named image, and verify under load. docs/features/release-keeper.md.
Test fix in this push. The last check failed in test/tui-people-verify.test.ts, which this pull did not otherwise touch. The test was wrong, not the code: it assumed bobby is the first row, but listPeople orders by created_at DESC, username. When alice and bobby are created in the same millisecond, alice sorts first and v lands on her. I reproduced it once in six parallel runs. The test now moves the cursor onto bobby and asserts it is there before pressing v; eight more parallel runs then passed.
Left alone: the same check also failed GET /settings/cloud/audit names its phases and answers within its budget. That is a timing budget on a loaded shard; it already takes the best of seven requests against a baseline, and this pull does not touch that page.
Not done: the keeper does not build images. If the image for a version is not built yet, the new pod waits on its pull while the old pod keeps serving.
To try it: bun test test/release-keeper-*.test.ts test/tui-release-keeper.test.ts. On the preview, open /releases as an admin, or press ctrl+k in the terminal and type Release keeper.
🤖 Generated with Claude Code
Parked by the crew: the check stayed red through 2 automatic tries
I've added a release keeper that runs inside the forge on a one-minute timer, with no model involved. It is off until
FORGE_RELEASE_KEEPER=onis set, so turning it on in production is a separate step. The four keeper test files and the shared test files they touch pass, and the type check is clean. I booted the app locally:/answers 200 in 17 ms and the boot log says why the keeper is off. Nothing has run against a real cluster yet.What it does (
src/keeper/, port ofrelease.sh,verify-after.shandverify-release.sh):cadence.ts): it cuts when main has moved past the newest tag, nothing of its own is rolling or verifying, the newest tag is live (or the keeper rolled it back), no merge-queue gate is running, 30 minutes have passed since the last cut, and no hold stands. When a rule says no, the answer names that rule.v<version>on main's head, carrying the same recordscripts/cut-release.tswrites. It never commits to main.deploy.ts): sets the forge container's image through the kube API, using the pod's service account. While the roll runs, the old forge probes/healthzonce a second and counts every probe that got no 200.verify.ts): the new version spends 10 minutes fetching/boards, the forge's repository page,/exploreand/one at a time, signed in as a site admin with a session it deletes afterwards. Meanwhile it probes/healthzevery second. Three probes in a row that are non-200 or over 10 s fail the release.hold.ts), then sets the known-good image by name (rollback.ts). There is no undo path. A release not live 3 h after its cut is rolled back the same way.release hold Nlets the fix be cut, and that release verifying lifts the hold.release.cut,release.released,release.verified_live,release.rolled_backorrelease.held.Where to see it:
/releasesshows what holds the next cut and each release's numbers.keeperReleasessource, charts seconds dark and worst latency per release.release_keeperreturns the same data as JSON.release_holdsets or lifts an owner hold: admin scope, a person only, never an agent. A failed verify's hold cannot be lifted by hand.20261027140000-release-keeper(pinned). The how-to isdocs/features/release-keeper.md.Tests:
test/release-keeper-cadence.test.tstest/release-keeper-verify.test.ts: a seeded healthy release endsverifiedwith its probes and worst latency stored.test/release-keeper-rollback-named.test.ts: a seeded failing verify sets the recorded image, not the deployment's previous one, holds with the numbers and cuts nothing more. It also asserts the deployer has onlyimage/setImage/rolled, sends one PATCH naming the image, and that no code undersrc/keepermentions undo, revisions or a rollback endpoint.test/release-keeper-hold.test.tsTwo existing tests were stale lists, not wrong logic, and now include the new entries:
test/boards-starting.test.ts(the new board) andtest/mcp.test.ts(the new tool).What I chose not to do:
getandpatchon the forge Deployment, plusFORGE_RELEASE_KEEPER=on.originPush). If the mirror that startsrelease.ymlis fed some other way, that path needs checking.To try it:
On the preview, open
/releasesand the Maintenance board under/boards.🤖 Generated with Claude Code
Closes #7557