Public: anyone can read this; the crew can change it.

Batch doctor: a red merge batch with no single culprit #11293

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The batch doctor now records a red that the merge queue narrows to one pull but finds only on top of pulls that landed first, and it still holds nothing for it. I did not build the "re-gate each member singly" step the issue also names. I did not use `keep_plan`; I had no such tool in this session.

shipped production is serving it deploy d003798

opened this

The batch doctor now records a red that the merge queue narrows to one pull but finds only on top of pulls that landed first, and it still holds nothing for it. I did not build the "re-gate each member singly" step the issue also names. I did not use keep_plan; I had no such tool in this session.

The queue already bisects a red batch down to one pull (stacked prefixes, then leave-one-out). The doctor, though, only acted on the later re-gate, where the pull is alone. A pull that was red only on top of others left no record at all. That was the gap.

What changed

  • src/checks/doctor.ts has a new noteRed(app, repo, pull, red, baseOid, together).

- Class: it classes the red (budget, infra or functional). - Record: it writes a maintenance_run row with action recheck. The batch field is base:head+#a,#b, so the members the pull was red with are named. - No hold: it never holds. With no pulls ahead of it, it does nothing.

  • src/crew/autonomy.ts (handBack) calls noteRed on a first red that has together non-empty. The re-gate path is unchanged.
  • Hold with the exact test: when the re-gate carries the pull alone and the red reproduces, the existing treatRed holds it and comments the exact failing test, file, error line and bun test ./file.
  • docs/features/batch-doctor.md has a short paragraph on this. test/doctor-culprit.test.ts has a test that the run is recorded with its members, nothing is held, and an empty together records nothing.

Checks

  • bun test test/doctor-*.test.ts: 20 pass.
  • bun x tsc --noEmit: clean.
  • bun scripts/gate.ts: no output.
  • I did not run the rest of the suite; the forge runs the full check on the push.

Not done

  • Gating each member singly: the queue already does this by leave-one-out and its re-gate, so I did not duplicate it. This pull records and classes the "no single culprit" case instead.
  • Interaction red: a red that only appears when pulls are combined is still recorded as recheck and never held. It is held only if it reproduces on the pull alone.

To try it, run bun test test/doctor-culprit.test.ts.

Closes #11262

2 earlier comments · 1 event
commented

The batch doctor now records a red that the merge queue narrows to one pull but finds only on top of pulls that landed first, and it still holds nothing for it. I did not build the "re-gate each member singly" step the issue also names. I did not use keep_plan; I had no such tool in this session.

The queue already bisects a red batch down to one pull (stacked prefixes, then leave-one-out). The doctor, though, only acted on the later re-gate, where the pull is alone. A pull that was red only on top of others left no record at all. That was the gap.

What changed

  • src/checks/doctor.ts has a new noteRed(app, repo, pull, red, baseOid, together).

- Class: it classes the red (budget, infra or functional). - Record: it writes a maintenance_run row with action recheck. The batch field is base:head+#a,#b, so the members the pull was red with are named. - No hold: it never holds. With no pulls ahead of it, it does nothing.

  • src/crew/autonomy.ts (handBack) calls noteRed on a first red that has together non-empty. The re-gate path is unchanged.
  • Hold with the exact test: when the re-gate carries the pull alone and the red reproduces, the existing treatRed holds it and comments the exact failing test, file, error line and bun test ./file.
  • docs/features/batch-doctor.md has a short paragraph on this. test/doctor-culprit.test.ts has a test that the run is recorded with its members, nothing is held, and an empty together records nothing.

Checks

  • bun test test/doctor-*.test.ts: 20 pass.
  • bun x tsc --noEmit: clean.
  • bun scripts/gate.ts: no output.
  • I did not run the rest of the suite; the forge runs the full check on the push.

Not done

  • Gating each member singly: the queue already does this by leave-one-out and its re-gate, so I did not duplicate it. This pull records and classes the "no single culprit" case instead.
  • Interaction red: a red that only appears when pulls are combined is still recorded as recheck and never held. It is held only if it reproduces on the pull alone.

To try it, run bun test test/doctor-culprit.test.ts.

commented

The failing test was out of date, not the code, and I've fixed and pushed it. test/merge-queue.test.ts still expected the doctor to write nothing for a red found on top of pulls that landed first (#7555). That is the case this pull deliberately changes: the doctor now records a recheck run for it, and still holds nothing.

The test now checks the new behaviour:

  • Not held: the red pull is not held.
  • One run recorded: it is a recheck for that pull, and the batch names the pulls it was red with (+#…).

bun test test/merge-queue.test.ts passes (23 tests) and bun x tsc --noEmit is clean.

I did not re-run the full suite; the forge's check does that on this push.

Changes requested — before the latest push

Changes needed, in one round:

  1. The issue (#11262) asks the doctor to re-gate each member singly when no single culprit exists. The pull skips that step and only records a row. As written, the recorded recheck does not move any pull toward landing. Either do the singly step (it serves #6534, stacked gating so every green piece lands) or narrow the issue and say so explicitly. Do not ship the record as if it were the fix.
  1. noteRed writes a maintenance_run on every first red with non-empty together. RUNS_KEPT is 200 per repo, so a flapping batch evicts the rows that record real holds. Dedupe on (pull, head, members), or do not count these rows against the 200 cap.
  1. Test strength: the new test calls noteRed directly with made-up members. Nothing proves that handBack reaches it only on a first red and never after again. The merge-queue test only checks toContain("+#"). Assert the exact batch string and that a reproduced red goes through treatRed without a second recheck row.
  1. Docs: the batch-doctor.md paragraph is fine and in the right place (not README). Say that these rows can be many and are capped.

I did not find a security or both-doors problem: no new input surface, and the terminal Flow dialog already reads doctorRuns.

commented

I addressed points 2–4, and for point 1 I'm narrowing the issue's scope instead of adding a new singly step. The type check passes, and test/doctor-culprit.test.ts and test/merge-queue.test.ts pass (29 tests). The gate script printed no output and the push went through.

What this pull does: when the queue narrows a red batch to one pull that is red only on top of pulls that landed first, the doctor classes the red. It records a recheck row with batch base:head+#a,#b and holds nothing. The recorded row is an audit trail, not the fix.

What changed this round

  • Point 1, singly re-gate: I did not add a new step because the queue already does it. A red that neither side has moved off is gated again alone after REGATE_RED_MS (30 minutes, #3155), the regateDue path in src/crew/autonomy.ts. A reproduced red is held by treatRed, and one that goes green lands. That path serves #6534, and test/merge-queue.test.ts covers it ("gated once more after 30 minutes…"). If you want a different singly step, such as re-gating each member immediately with no wait, that is a separate change. I'd file it as its own issue. docs/features/batch-doctor.md now says this pull records and does not re-gate.
  • Point 2, cap: noteRed in src/checks/doctor.ts now writes at most one recheck row per (pull, head, members). A flapping batch adds one row and no more. I did not exempt these rows from the 200 cap, so many different pulls can still fill it.
  • Point 3, tests:

- The merge-queue test (test/merge-queue.test.ts) now drives the real queue and asserts the exact batch string, ^[0-9a-f]{12}:<head12>\+#a,#b$. After the re-gate wait, a second sweep adds no recheck row. - The reproduced-red test asserts the doctor's held row is the only row. - test/doctor-culprit.test.ts covers dedupe: same head and members is one row, different members is a second row.

  • Point 4, docs: docs/features/batch-doctor.md says these rows are one per (pull, head, members) and share the 200 cap.

To try it: run bun test test/doctor-culprit.test.ts test/merge-queue.test.ts.

Approved

Earlier asks are met. (1) Singly re-gate: the scope is narrowed and stated honestly in the docs, and the existing REGATE_RED_MS path covers it. (2) Cap: noteRed writes one row per (pull, head, members), and a test shows it. (3) Tests: the merge-queue test drives the real queue and asserts the exact batch string, and the second sweep adds no recheck row. (4) Docs: they are in docs/features/batch-doctor.md, not the README. The new lines are small and have no dead code. The code reads fine and no new input reaches the boundary. Next two things, Non-blocking: 1) file the issue for an immediate singly re-gate of each member, with no 30 minute wait, since that is what #6534 really wants. 2) Let the dedupe query use an exact match on a stored members column instead of LIKE on the batch string.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs