A check stopped for CPU reads as the forge's failure, is re-checked once on its own, and never sends its builder a check_failed #11658

When the cell's watch stops a check for CPU, the check is now treated as an infrastructure failure. Its first stop on a head is re-queued once by itself and sends its builder no check_failed. A second stop in a row is reported like any other red. /flow's sentence now names CPU stops when they are the last hour's main failure. Merging is not the last step: the CPU-stop count still has to be taken after release (below).

What changed

• Spotting the stop: stoppedForCpu in src/apps/contain.ts sits next to cpuBreach and matches the error runStep returns (check stopped: it used 16.3 cores for 31s, over its limit of 16…). The match is anchored to the start of that error. A check's own output only ever appears after check exited with code N —, so a pull cannot print the stop line to get a free re-run.
• Re-queue once: src/checks/cpu_stop.ts (requeueCpuStop) uses the same once-only re-run rule as install failures (infraBefore), with its own CPU_STOP_SUMMARY / cpuStopBefore in src/core/runs.ts. The first attempt is marked cancelled, not failed, and only the retry is announced. The crew sends check_failed only when a check ends failed, so it is not sent for that attempt. src/checks/service.ts calls this just before a red is recorded.
• /flow: src/flow/cpu_stops.ts counts the last hour's failed checks. Both kinds of CPU stop count: re-queued ones and reported ones. When CPU stops are at least half of those failures, the sentence leads with "checks stopped for CPU: N of M failed checks in the last hour" (src/flow/clog_view.ts). The read uses the existing run_kind_finished index, and the test checks the query plan of the SQL it actually runs.
• Test: test/check-cpu-stop.test.ts:

- the first stop is cancelled with the summary and re-queued at attempt 1 on the same head; - the second stop is not re-queued and is reported failed; - an ordinary red is not re-queued; - the match can't be faked by a check's output; - the /flow wording and the hour window are right.

It tests requeueCpuStop directly, not a whole check run, because a real CPU stop needs a cell and this sandbox can't create one.

What I left out: merge-queue gates are unchanged. They already have their own "red that way twice is infrastructure" rule in src/checks/queue.ts. The issue asks for a keep_plan tool, but this run didn't have it, so the issue has no plan.

Production evidence (from the runs API, before this change is released). Checks that ended "stopped: used … over its limit", by hour (UTC), out of all failed checks in that hour:

The last one was at 2026-11-12T16:51:44Z, so 17:00–18:30Z is clean (0 of 25). The count for the hour after release still has to be taken and pasted here.

For the steward to re-queue: 33 pulls closed unmerged since 2026-11-11 21:00Z whose last failed check before closing was this stop. I didn't check why each one was closed, so some may have been closed for other reasons: #11465 #11467 #11471 #11489 #11523 #11537 #11540 #11542 #11546 #11547 #11550 #11552 #11554 #11557 #11567 #11568 #11569 #11570 #11571 #11572 #11574 #11575 #11576 #11588 #11589 #11590 #11591 #11592 #11605 #11608 #11609 #11610 #11622

Checks run here: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts picked 2,403 test files for this diff, so I chose 10 that cover it: the new test, check-install, checks-job-vanish, contain, cpu-breach, red-approved and the flow-clog tests. 79 pass and 0 fail. The 30 skipped are cell and unix-socket suites this sandbox can't run; the forge's check runs them.

🤖 Generated with Claude Code

Closes #11617

When the cell's watch stops a check for CPU, the check is now treated as an infrastructure failure. Its first stop on a head is re-queued once by itself and sends its builder no check_failed. A second stop in a row is reported like any other red. /flow's sentence now names CPU stops when they are the last hour's main failure. Merging is not the last step: the CPU-stop count still has to be taken after release (below).

What changed

• Spotting the stop: stoppedForCpu in src/apps/contain.ts sits next to cpuBreach and matches the error runStep returns (check stopped: it used 16.3 cores for 31s, over its limit of 16…). The match is anchored to the start of that error. A check's own output only ever appears after check exited with code N —, so a pull cannot print the stop line to get a free re-run.
• Re-queue once: src/checks/cpu_stop.ts (requeueCpuStop) uses the same once-only re-run rule as install failures (infraBefore), with its own CPU_STOP_SUMMARY / cpuStopBefore in src/core/runs.ts. The first attempt is marked cancelled, not failed, and only the retry is announced. The crew sends check_failed only when a check ends failed, so it is not sent for that attempt. src/checks/service.ts calls this just before a red is recorded.
• /flow: src/flow/cpu_stops.ts counts the last hour's failed checks. Both kinds of CPU stop count: re-queued ones and reported ones. When CPU stops are at least half of those failures, the sentence leads with "checks stopped for CPU: N of M failed checks in the last hour" (src/flow/clog_view.ts). The read uses the existing run_kind_finished index, and the test checks the query plan of the SQL it actually runs.
• Test: test/check-cpu-stop.test.ts:

- the first stop is cancelled with the summary and re-queued at attempt 1 on the same head; - the second stop is not re-queued and is reported failed; - an ordinary red is not re-queued; - the match can't be faked by a check's output; - the /flow wording and the hour window are right.

It tests requeueCpuStop directly, not a whole check run, because a real CPU stop needs a cell and this sandbox can't create one.

What I left out: merge-queue gates are unchanged. They already have their own "red that way twice is infrastructure" rule in src/checks/queue.ts. The issue asks for a keep_plan tool, but this run didn't have it, so the issue has no plan.

Production evidence (from the runs API, before this change is released). Checks that ended "stopped: used … over its limit", by hour (UTC), out of all failed checks in that hour:

The last one was at 2026-11-12T16:51:44Z, so 17:00–18:30Z is clean (0 of 25). The count for the hour after release still has to be taken and pasted here.

For the steward to re-queue: 33 pulls closed unmerged since 2026-11-11 21:00Z whose last failed check before closing was this stop. I didn't check why each one was closed, so some may have been closed for other reasons: #11465 #11467 #11471 #11489 #11523 #11537 #11540 #11542 #11546 #11547 #11550 #11552 #11554 #11557 #11567 #11568 #11569 #11570 #11571 #11572 #11574 #11575 #11576 #11588 #11589 #11590 #11591 #11592 #11605 #11608 #11609 #11610 #11622

Checks run here: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts picked 2,403 test files for this diff, so I chose 10 that cover it: the new test, check-install, checks-job-vanish, contain, cpu-breach, red-approved and the flow-clog tests. 79 pass and 0 fail. The 30 skipped are cell and unix-socket suites this sandbox can't run; the forge's check runs them.

🤖 Generated with Claude Code

When a check is stopped by the cell's CPU watch, the forge treats it as the machine's failure, not the pull's. The first stop of a head is cancelled with CPU_STOP_SUMMARY and queued once more for the same head, and the builder is not sent check_failed. A second stop in a row is reported red like any other failed check. On /flow, when CPU stops are at least half of the last hour's failed checks, the sentence starts with "checks stopped for CPU: N of M failed checks in the last hour".

This round adds tests for the two connections between the parts that review found untested:

• The checks service. ChecksOptions.checkStep is a new option that lets a test replace the check step. A test can then make the watch's stop line, which no check's own output can produce, without a cell. The option is passed through startStandInCrew, and ChecksOptions defaults it to runStep. test/crew-check-cpu-stop.test.ts runs a stand-in agent's pull through the real checks service and crew:

- The first stop is cancelled with CPU_STOP_SUMMARY, and a retry is queued at attempt 1 on the same commit. - The second stop waits until the test has checked that no check_failed run exists and no red run.finished was announced. - The second stop is red, and exactly one check_failed follows. - That commit is never tried a third time.

• /flow's sentence. test/check-cpu-stop.test.ts now checks that readClogView returns a sentence starting "Clogged: checks stopped for CPU: 2 of 3 failed checks in the last hour" when CPU stops are the only problem. That also covers the cpu !== undefined condition that decides whether the sentence is shown.

Each new test fails when the wiring it covers is removed. I tried it by turning off the requeueCpuStop block in checks/service.ts and by removing cpu !== undefined in clog_view.ts.

Production count (box 4). In production, between 17:49Z and 18:49Z on 2026-11-12, 0 checks ended "stopped: used … over its limit", out of 21 failed checks (219 finished). The last such stop was at 16:51:44Z. This was read from the runs API before this pull is merged, so it shows production without this change. The count for the hour after this pull is released can only be taken once it is merged.

For the steward to re-queue: 33 pulls were closed unmerged since 2026-11-11 21:00Z, and their last failed check before closing was this stop. I did not check why each was closed, so some may have been closed for other reasons: #11465 #11467 #11471 #11489 #11523 #11537 #11540 #11542 #11546 #11547 #11550 #11552 #11554 #11557 #11567 #11568 #11569 #11570 #11571 #11572 #11574 #11575 #11576 #11588 #11589 #11590 #11591 #11592 #11605 #11608 #11609 #11610 #11622

Left out: merge-queue checks are unchanged; they already have their own "red twice the same way is infrastructure" rule in src/checks/queue.ts.

Checks run here: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts picked 2,404 test files for this diff, because the stand-in helper and service.ts are imported almost everywhere. I ran the 14 files that cover the change instead: both CPU-stop tests, crew-check-failed, check-install, checks-job-vanish, checks-disk-red and the clog tests. 82 pass, 0 fail. The suites that need a cell or a unix socket can't run in this sandbox; the forge's check runs them.

🤖 Generated with Claude Code

Closes #11617

When a check is stopped for using too much CPU, it is now treated as an infrastructure stop, not as the pull's failure. It is queued again once, sends no check_failed, and /flow names these stops when they are the hour's main cause of failed checks (#11617).

This update merges main, which moved under the pull. One conflict, in src/checks/service.ts: the result type of the check step.

• main (#11669) added cleared?: string to the passing result. It holds the words for a red the pull did not cause because main was already red.
• This branch added cpuStop?: string to the failing result.

The resolution takes main's version of the type and adds cpuStop back to the failing result, so both fields are kept. Nothing else needed changing.

Checks run on the merge, all passing:

• bun x tsc --noEmit is clean.
• bun scripts/gate.ts passes.
• This pull's tests pass: 47 across test/check-cpu-stop.test.ts, test/crew-check-cpu-stop.test.ts and test/flow-clog*.test.ts.
• The 24 test files #11669 touched (test/red-on-main.test.ts, test/tree-scans.test.ts and others) pass, with 2 tests skipped.

To try it: a check whose run is stopped for CPU should appear once more in the queue, marked as infrastructure. No check_failed note should go out for it. When those stops are most of the hour's failed checks, /flow should name them as the reason.

🤖 Generated with Claude Code

Left #11617 open: 1 unticked item remain (After release, the pull pastes the count of production checks in one hour that ended "stopped: used … over its limit": it must be 0. Pulls closed unmerged for this stop since 2026-11-11 21:00Z (#11546, #11547 and any others found) are listed in the pull for the steward to re-queue.).