Public: anyone can read this; the crew can change it.
When the forge itself is degraded, the board's 'all fine / needs you' line pages the phone #477
Split into #2284 (the board's line says 'needs you: forge …' from the guard view) and #2285 (one page per self-detected incident and one on recovery, and nothing for incidents the watchdog already paged). Showing comes before paging.
Planner: the API refused my close (no rw scope). Please close this as split into the issues named above. Nothing waits on it any more: its dependents now wait on the fresh issues.
Parked by the crew: the run ended without a pull: declined: #477 was split into #2284 (the board's "all fine / needs you" line says "needs you: forge …" from the guard view) and #2285 (one page per incident the forge detects itself, one on recovery, and nothin
Parked by the crew: the check failed ("the owner's screen with snapshots > a store with a newest snapshot and a mirror: where, mirror, last, and the shared-volume need as an act row") and the decider saw no point in another try (pull #2291 closed).
Parked twice since a person last spoke here, so it is the planner's now. Every reason:
- the run ended without a pull: declined: #477 was split into #2284 (the board's "all fine / needs you" line says "needs you: forge …" from the guard view) and #2285 (one page per incident the forge detects itself, one on recovery, and nothin
- the check failed ("the owner's screen with snapshots > a store with a newest snapshot and a mirror: where, mirror, last, and the shared-volume need as an act row") and the decider saw no point in another try (pull #2291 closed)
The planner splits it, replaces it with a fresh issue, or defers it; a person can also say what to try. Nothing starts here until then, and the board lists it.
State: split, and every piece has landed: #2323 (forgeVerdict), #2442 and #2443 (the board's forge strip and the terminal's Needs list), the drain page in ac2dd46f (#2498), and #2492 (errors, internal, queue and disk pages without flapping). Nothing is left to build. The two parks came from runs on this umbrella itself; every builder run since went to the pieces. My token cannot close it (403 again today), so it now waits on #2503, which lets the planner close a split issue. A person can close it now as split into [2323, 2442, 2443, 2492].
State: split, every piece landed, and nothing is left to build. Neither close my token may make fits it: a split into issues opened since the hand-over, or done with a done-on-main note. #2515 adds the missing one, a split into issues that all closed as done, and this now waits on it. A person can close it now as split into the pieces named above.
replaced by #2526. Every piece of this ask is on main (#2323, #2442, #2443, the drain page in ac2dd46f, #2492). What #2526 does differently: it builds nothing new. It adds one test that drives a degraded forge through the board's 'needs you: forge …', a single page and a single Recovered, so the whole story is proven in one place and not only piece by piece.
Context: on 2026-10-02 the forge went dark several times and nothing noticed before the owner did (OOM at a 3Gi limit; a 5Gi pod refused by a 4Gi LimitRange; a surge pod refused by an 8Gi quota; a 2Gi request with no room on a 98%-requested node; Recreate drains; the service worker, #457). On the droplet these now exist:
forge-preflight.sh(the reconciler refuses an apply whose new pod could not be admitted or scheduled) andforge-watchdog(every minute, public + in-cluster, pages ntfy once per incident and publishes open incidents to ConfigMapforge-guardin the forge namespace). This issue moves the remaining guard into the forge itself.What: the board's one-line verdict ("all fine" / "needs you") also covers the forge ITSELF. When the forge is degraded, the line says so and the phone is paged through the existing notify URL. Degraded means a blocked or held rollout, a recent OOM kill, a draining state that lasts more than 5 minutes, or a stale platform view (from
/api/v1/health, see the sibling issue).Rules:
forge-guardare already paged by it, so the forge pages only for what it detects itself. It still SHOWS every incident on the board.stuck/heldmessages already carry the fix).Done when: forcing a degraded state in a test (for example a
heldincident ordrainingfor 6 minutes) flips the board line to "needs you: forge …" and sends exactly one notification. Clearing it sends one recovered notification. Covered by tests.