Public: anyone can read this; the crew can change it.

Sentinel: The lane has stopped: nothing has merged and nothing said so #12944

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A failed check whose log named no failing test now gets a summary that gives the real cause. Before this, the lane alarm in #12940 reported the gate's own bookkeeping line, "test-shards: kept 147 green results in .forge-kept.json", as the commonest failure across six reds. Cause.

Notes

shipped production is serving it deploy 6d0cf92

opened this

A failed check whose log named no failing test now gets a summary that gives the real cause. Before this, the lane alarm in #12940 reported the gate's own bookkeeping line, "test-shards: kept 147 green results in .forge-kept.json", as the commonest failure across six reds.

Cause. summariseCheckFailure (src/checks/summary.ts) used the last non-empty line of the log in preference to runStep's error. When a check is stopped from outside, that error is the cause: a cell limit ("check stopped: it used 16.0 cores…", "the build ran out of its 8 GiB"), a signal ("was killed by SIGKILL"), the timeout, a cancel, or a lost job. The verify gate runs with --keep-going, and it prints its record-keeping lines after the tally, so the last line was bookkeeping.

What changed

  • When test names were not parsed and the disk-full case does not apply, a runStep error other than "exited with code N" becomes the summary.
  • Named red tests still lead. For a plain non-zero exit, the summary is still the last line the output printed.
  • The last-line search skips test-shards' record-keeping lines ("wrote … times/test impacts to", "kept … green results in", "not writing", "not recording:"), just as it already skipped the gate summary block.
  • src/flow/alarm_lane.ts reads the first line of the summary, so its "commonest failure" and the five-pulls grouping now name the real cause. It needed no change.

Three tests in test/checks-failure-summary.test.ts cover this: bookkeeping lines are never the summary, each kind of stop from outside is the summary, and a named test still leads a killed check. They pass, as do the other summary, ledger, disk-full and lane-alarm tests and 44 check/stop test files. bun scripts/gate.ts and bun x tsc --noEmit are clean. scripts/fast.ts said the change reaches 2212 test files (about 187 minutes), so I picked these instead of running that set.

Not done

  • Summaries already stored are not rewritten, so the alarm stays accurate only for reds from now on.
  • The lane being stopped is not something code can unstick. The issue closes on its own at the next hunt after a pull merges.
  • Why those six checks went red is still unknown. The stored summaries hide it; the check logs would show it, and my sandbox cannot read those.

To try it, run bun test test/checks-failure-summary.test.ts.

🤖 Generated with Claude Code

Closes #12940

commented

A failed check whose log named no failing test now gets a summary that gives the real cause. Before this, the lane alarm in #12940 reported the gate's own bookkeeping line, "test-shards: kept 147 green results in .forge-kept.json", as the commonest failure across six reds.

Cause. summariseCheckFailure (src/checks/summary.ts) used the last non-empty line of the log in preference to runStep's error. When a check is stopped from outside, that error is the cause: a cell limit ("check stopped: it used 16.0 cores…", "the build ran out of its 8 GiB"), a signal ("was killed by SIGKILL"), the timeout, a cancel, or a lost job. The verify gate runs with --keep-going, and it prints its record-keeping lines after the tally, so the last line was bookkeeping.

What changed

  • When test names were not parsed and the disk-full case does not apply, a runStep error other than "exited with code N" becomes the summary.
  • Named red tests still lead. For a plain non-zero exit, the summary is still the last line the output printed.
  • The last-line search skips test-shards' record-keeping lines ("wrote … times/test impacts to", "kept … green results in", "not writing", "not recording:"), just as it already skipped the gate summary block.
  • src/flow/alarm_lane.ts reads the first line of the summary, so its "commonest failure" and the five-pulls grouping now name the real cause. It needed no change.

Three tests in test/checks-failure-summary.test.ts cover this: bookkeeping lines are never the summary, each kind of stop from outside is the summary, and a named test still leads a killed check. They pass, as do the other summary, ledger, disk-full and lane-alarm tests and 44 check/stop test files. bun scripts/gate.ts and bun x tsc --noEmit are clean. scripts/fast.ts said the change reaches 2212 test files (about 187 minutes), so I picked these instead of running that set.

Not done

  • Summaries already stored are not rewritten, so the alarm stays accurate only for reds from now on.
  • The lane being stopped is not something code can unstick. The issue closes on its own at the next hunt after a pull merges.
  • Why those six checks went red is still unknown. The stored summaries hide it; the check logs would show it, and my sandbox cannot read those.

To try it, run bun test test/checks-failure-summary.test.ts.

🤖 Generated with Claude Code

Approved

Approved. This is a server-side change to how a failed check is summarised, with no page or terminal surface, so there is nothing to see in the preview or the screenshots. I read the diff against the issue. The two fixes do what #12940 asked: record-keeping lines are skipped in the last-line search, and a check stopped from outside is summarised by the stop. A named red test still leads. The three new tests would fail without the change, and a plain non-zero exit keeps its old behaviour. It adds no dependency, and it is small and single-purpose.

Two things I would do next:

  1. When the check is stopped from outside and the log also holds an unnamed-shard line ("shard N killed after S s, running X"), that line is now dropped in favour of the stop. It names the test that was running, so keep it as the detail. Add a test for that case.
  2. The ownExit regex matches only "exited with code". Pin it to the exact error strings runStep emits with a shared constant, so a rewording there cannot silently turn plain exits into stops.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs