Public: anyone can read this; the crew can change it.

The merge queue says why a gate went red, and never bisects a merge conflict or an install failure (8-pull batch red in 6 s, then bisected) #6615

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

The merge queue now logs why each gate went red. Install, setup and timeout reds get one more gate and then count as infrastructure, never as a red pull. Only reds caused by the code are bisected.

shipped production is serving it deploy 0af2c9c

opened this

The merge queue now logs why each gate went red. Install, setup and timeout reds get one more gate and then count as infrastructure, never as a red pull. Only reds caused by the code are bisected.

  • Every gated … red line names its reason: gated 8 pulls: red (install), or red (typecheck), red (tests: 3 failed), red (timeout), red (setup) or red (check). The new module src/checks/gate_reason.ts works out the reason from where the gate stopped and what it printed. To make that possible, the check service now reports which stage stopped the gate: setup, install, check or timeout. gateCandidate (in landing.ts) puts the reason on the red verdict as an optional reason field.
  • No bisection for install, setup or timeout reds: counted() in queue.ts runs the gate a second time. If it goes red that way again, the queue logs gated N pulls: infrastructure (install, twice) and treats it like a cut-off gate. No pull is handed back, nothing is searched, and the next sweep lets the pulls back in. setup means the gate failed before any step ran. Install failures that look like flakes were already retried once inside the step (#5093); this retry covers the gate as a whole.
  • Bisection only for code reds: tests, typecheck, and check (red with no test or type error named) are bisected as before. A red verdict with no reason is treated as the code's, so existing behaviour stays the same.
  • Merge conflicts inside a batch: build() already left the conflicting pull out and gated the rest at once; nothing in this pull changes that. What's new is the log line naming the pair: left out of the batch: merge-conflict (#a × #b), or main × #b when the pull conflicts with the base itself.

Choices to look at:

  • No new code sends a conflicting pull back for an update. Once #a lands, #b conflicts with main. The next sweep's admission then marks it conflicted, and the existing conflict lane brings it up to date within its budget. Calling bringUpToDate straight from the queue would skip that budget, and would waste a run if #a never landed.
  • Repeated infrastructure failures loop. A batch that keeps going red on install costs two gates per sweep, but no pull is ever reported red for it.
  • Not fixed: checks that run as a cluster Job do install and check as a single step. An install failure there reads as check (still bisected) and isn't retried; telling them apart needs runJobStep to say which command failed. I did not change the red comment on the pull either.

Tests: test/merge-queue-reason.test.ts covers working out the reasons and the log wording. It also checks that an install red twice means exactly two gates, no bisection and nothing handed back. An install red followed by green lands the whole batch. A test red is logged as red (tests: 2 failed) and still bisected. And a batch where #33 conflicts with #31 lands #31, #32 and #34 in one gate, with #33 left out and named. test/check-tiers.test.ts now also checks that a red full gate reports stage check. Type check is clean; the queue, tiers, landing, install and check-service suites pass (155 tests), and so does test/check-tiers.test.ts with the new assertion (45 pass).

🤖 Generated with Claude Code

Closes #5276

commented

The merge queue now logs why each gate went red. Install, setup and timeout reds get one more gate and then count as infrastructure, never as a red pull. Only reds caused by the code are bisected.

  • Every gated … red line names its reason: gated 8 pulls: red (install), or red (typecheck), red (tests: 3 failed), red (timeout), red (setup) or red (check). The new module src/checks/gate_reason.ts works out the reason from where the gate stopped and what it printed. To make that possible, the check service now reports which stage stopped the gate: setup, install, check or timeout. gateCandidate (in landing.ts) puts the reason on the red verdict as an optional reason field.
  • No bisection for install, setup or timeout reds: counted() in queue.ts runs the gate a second time. If it goes red that way again, the queue logs gated N pulls: infrastructure (install, twice) and treats it like a cut-off gate. No pull is handed back, nothing is searched, and the next sweep lets the pulls back in. setup means the gate failed before any step ran. Install failures that look like flakes were already retried once inside the step (#5093); this retry covers the gate as a whole.
  • Bisection only for code reds: tests, typecheck, and check (red with no test or type error named) are bisected as before. A red verdict with no reason is treated as the code's, so existing behaviour stays the same.
  • Merge conflicts inside a batch: build() already left the conflicting pull out and gated the rest at once; nothing in this pull changes that. What's new is the log line naming the pair: left out of the batch: merge-conflict (#a × #b), or main × #b when the pull conflicts with the base itself.

Choices to look at:

  • No new code sends a conflicting pull back for an update. Once #a lands, #b conflicts with main. The next sweep's admission then marks it conflicted, and the existing conflict lane brings it up to date within its budget. Calling bringUpToDate straight from the queue would skip that budget, and would waste a run if #a never landed.
  • Repeated infrastructure failures loop. A batch that keeps going red on install costs two gates per sweep, but no pull is ever reported red for it.
  • Not fixed: checks that run as a cluster Job do install and check as a single step. An install failure there reads as check (still bisected) and isn't retried; telling them apart needs runJobStep to say which command failed. I did not change the red comment on the pull either.

Tests: test/merge-queue-reason.test.ts covers working out the reasons and the log wording. It also checks that an install red twice means exactly two gates, no bisection and nothing handed back. An install red followed by green lands the whole batch. A test red is logged as red (tests: 2 failed) and still bisected. And a batch where #33 conflicts with #31 lands #31, #32 and #34 in one gate, with #33 left out and named. test/check-tiers.test.ts now also checks that a red full gate reports stage check. Type check is clean; the queue, tiers, landing, install and check-service suites pass (155 tests), and so does test/check-tiers.test.ts with the new assertion (45 pass).

🤖 Generated with Claude Code

Approved

Does what #5276 asks: each red gate now says why, install/setup/timeout reds get one more gate and then count as infrastructure (no verdict, nothing handed back, nothing bisected), and a batch conflict is named. The module is small and pure, the behaviour is covered by tests that fail without it, and an unreasoned red keeps today's behaviour. It is queue-internal, with no page, tty or MCP surface, so there is nothing to see in the preview and no security-relevant input. Two things to do next: (1) a gate that fails with no stage (e.g. "this commit declares no check", src/checks/service.ts:753) now defaults to setup and is gated twice; confirm that is intended, or give it its own stage. (2) Bound the infrastructure loop: a batch that keeps going red on install costs two gates every sweep forever, so count consecutive infrastructure results per line and surface them in the tick line or flow vitals (#6428). Also the Job-step case you flagged (install failure reads as check and is bisected) deserves its own issue.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs