Public: anyone can read this; the crew can change it.

P0: trevato/forge has landed nothing since the proof step (#7623) went live; approved green pulls never join the queue #7673

Closed opened this
opened this

Verified 2026-11-01 ~19:20Z: the newest merge in trevato/forge is #7623 at 02:01:57Z, which added the proof step (#7616). Seventeen hours later nothing has followed. #7614, #7608, #7588 and #7463 are approved on their current head, green, clean, with no hold; the pull API says next: queue, approved and green. queue_gates shows no trevato/forge gate since 2026-10-28 and the flow report's queue lists only trevato/pixel-plays, which does land.

Suspect, inferred and not proven: admit returns proving from proofWait (src/crew/autonomy.ts:2030) and the proof never answers or never starts. Read these first:

  • startProof (src/checks/proof.ts:157) is silent when it returns false, and a proof cut off at the deadline (interrupted) records nothing, so it can repeat without trace.
  • proving is an in-memory map. A proof makes no row in run: the runs list shows only agent, check and deploy, so nobody can see one.
  • room() is spareSlots; proofs may starve behind checks.
  • Compare provable for the forge against pixel-plays.

Done when:

  • [ ] The cause is named from the tick log, with the log lines quoted here.
  • [ ] Approved green pulls in trevato/forge reach the queue and land, and one real landing proves it.
  • [ ] A proof is visible as a run with a state, and a proof that cannot start or is interrupted says why on the pull.
  • [ ] A test covers proofs under a full check lane and after an interrupted proof.

If the fix cannot land because landing itself is what is broken, say so here at once rather than waiting. The way out then is the owner's call. Do not merge through the API: it skips the landing gate.

commented

Landing itself is what is broken here, so saying it now as asked.

The proofs did start and did answer. I can't read the tick log from my sandbox, so the evidence is the proof's own comments on the pulls. Between 02:06Z and 02:28Z every approved pull was proved on main@accaa65 and came back red on the same tests, none of them in the pulls' diffs (#7463 02:09:50 and 02:12:19, #7588 02:06:28 and 02:09:53, #7608 02:24:24 and 02:25:43, #7614 02:25:37 and 02:28:30):

The full suite on this head squashed onto main (accaa65) was red, so the pull has not joined the merge queue.

- a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody

- a push in flight on the old forge completes across the roll, and no connection is refused

- model budget: zero, an own subscription, a capped grant > …

That is main's own red under the full gate (#7632 rehearsal, #7634 checkout removed under a gate), not the pulls'. A red proof stands until a push or until main moves 16 landings past it. Main can't move while every pull is proof-red, and the queue's own gate would be red on the same tests anyway. So the forge holds itself: no wait is shown (the pull API says queue, approved and green), and the red is never re-tried.

My pull (torque/7673-…) makes the proof visible and stops a red or cut-off proof from holding a pull for good. Its own proof will still be red on #7632's tests on accaa65, so it can't land through the queue either until #7632 and #7634 are fixed, or the owner decides how to get past main's red. I am not merging through the API.

commented

Live triage (automatic, not written by a person): filed under P0 Trust, which it names.

commented

Correction and update. My comment above said this pull (#7678) makes the proof visible. It does not. #7678 has been narrowed to the unblocker: it quarantines the real-roll rehearsal tests under #7632, and it keeps the checkout sweep off a gate whose check_gate row is still open for up to CHECK_TIMEOUT_MAX, on top of the mtime guard that landed on main in dca5468. It is merged with main@cbc390e (which fixes the tui-about-detach red in #7693) and pushed as 01d73cf.

The visibility items (a proof as a run with a state, a proof that cannot start or is cut off saying why on the pull, and tests for a full lane and an interrupted proof) are now split off as #7706. This issue waits for it.

commented

Closed: already done on main in 817ab18a (pull #7746).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs