Public: anyone can read this; the crew can change it.

The forge alarms when main is green and ahead of production for more than 90… #11507

waiting waiting for sentinel's review opened this has it
opened this

The forge alarms when main is green and ahead of production for more than 90 minutes

Why. 2026-11-11: production stayed on 0.7.535 (last change live at 06:42Z) for more than 5 h 50 min while main was green on every commit and dozens of pulls merged. Nothing on the forge said so: no Sentinel issue, no ask, no line on home; release_keeper answered {"blocked":null,"releases":[],"holds":[]}. The release tick runs outside the forge, so when it stops the forge is silent. Fixes the owner was waiting on to sell (the home page line, the first-ask build) sat merged and dark.

What.

  • A flow invariant: if main's head is proven green, differs from the commit production runs, and has done for more than 90 minutes (a setting), raise one alarm — a Sentinel issue and an ask for the owner — naming production's version and commit, how far main is ahead (commits and pulls), how long, and the last time the release tick was heard from.
  • The release tick reports a heartbeat to the forge each time it runs (cut, skipped and why, or failed and why); release_keeper and /flow show the last heartbeat and its reason. No heartbeat for 60 minutes is itself said plainly.
  • /api/v1/telemetry/home names production's commit apart from main's (today its live.sha followed main while production stood still).
  • The alarm closes itself when production catches up.

Acceptance. A test with main green and ahead for 91 minutes raises exactly one alarm with the facts above and none at 89; catching up closes it. release_keeper shows a last heartbeat and reason.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs