Public: anyone can read this; the crew can change it.
P0: the merge queue records and names the tests that failed each gate (not just the count), with a gates API, so a red batch can be acted on #6740
Every finished merge-queue gate is now stored with the names of the tests that failed, the queue log names them, and they can be read from GET /api/v1/repos/:o/:n/queue/gates?limit=20 and a new queuegates MCP tool. The likely cause of the "2 failed" / "4 failed" counts.
stopped parked by the crew deploy 96bfe06
2 earlier comments · 1 event
Every finished merge-queue gate is now stored with the names of the tests that failed, the queue log names them, and they can be read from GET /api/v1/repos/:o/:n/queue/gates?limit=20 and a new queue_gates MCP tool.
The likely cause of the "2 failed" / "4 failed" counts. scripts/test-shards.ts ends with a tally that prints every failure a second time, as (fail) name [1ms] [shard 3]. failuresOf didn't strip the [shard N] suffix, so it read the repeat as a different test and counted each failure twice. A recent "2 failed" was probably one test. Both parsers now use the same pattern and count each failing test once.
What changed
- Reading the output:
src/checks/gate_tests.tsreads a gate's output for the passed count (summed across shards), the distinct failed count, and up to 20 failing tests. Each test has its file and up to 5 error lines, taken from bun'serror:line onward.landing.tsadds this to the red verdict, andqueue_store.tskeeps it when a verdict is resumed. - Storage: a new migration adds the
gate_resulttable, written bysrc/checks/gate_result.tsthroughQueueStore.judged. Every finished gate gets a row (green, red orcut_off): batches, search prefixes and suspects gated alone. Each row has its pulls, reason, counts, failing tests, start and finish times and duration. Rows older than 7 days are deleted whenever a new one is written. - Log line: it now reads
gated 8 pulls: red (2 failed: "a > b", "c > d"). It names up to 5 tests and ends withand N morepast that. A red that named no test reads as before. - API: the endpoint returns
{ items, next_cursor }, newest first.limitis 1–100 andcursoris a gate id. A bad cursor gets a 422, and a repository the reader can't see gets a 404. Its OpenAPI file and the MCP tool are generated in, anddocs/features/queue-gates.mdsays how to use it. - Flaky flag: each failing test the API returns has
flaky: truewhen the same file and test failed in two red gates that share no pull. It is computed from at most the last 500 red gates in the week the table keeps.
What I chose not to do
- The flaky flag doesn't change what the queue does: a flaky red is still searched and still bisected.
- The log line doesn't mark tests as flaky.
- No web page reads the table yet.
- #6643 turned out to be a pull about a migrate test's timeout, and there is no flake list on main. So the flake detection is built on
gate_resultitself rather than ported from an existing list.
Tests
test/merge-queue-gate-result.test.tsis new. A fake two-shard output with two failing tests gives both names once, with their files, errors and counts, andfailuresOfno longer doubles them. It also covers the 20-test cap, the log wording and the flaky rule.- End to end, it runs a real
sweepGreenagainst a stand-in check lane. It checks that the log line names both tests, and that the API and the MCP tool return both withfailedmatching the number named. It also checks paging, the 404 for a private repository, the 422 for a bad cursor, the flaky flag and the 7-day pruning. - I edited one existing test:
test/mcp.test.tslists every MCP tool by name, soqueue_gateshad to be added to it. - The type check is clean. The merge-queue, landing, gate, schema, OpenAPI and MCP suites pass, as do the five cross files.
To try it: bun test test/merge-queue-gate-result.test.ts. On the preview, after a gate has run, curl -H "authorization: Bearer $TOKEN" "$URL/api/v1/repos/trevato/forge/queue/gates?limit=5".
🤖 Generated with Claude Code
Parked by the crew: the check failed ("no GET route answers 200 without a session phase unless the allowlist names it") and the decider saw no point in another try
One line, verified by the steward. The new route needs the session phase like its siblings (e.g. /flow, api.ts:3577). In router.add("GET", "/api/v1/repos/:owner/:name/queue/gates", …), add as the first line:
notePhase(ctx.request, "session", takeSessionTime(ctx.request));
With it, test/route-timing-coverage.test.ts passes 2/2 on this branch (checked locally). If /gates/:id/log is new in this pull and can answer 200 in that test's fixture, give it the same line. This pull is the top of the pipeline: every gate since 01:30Z is red with unnamed tests.
Superseded by #6741 (same change, rebased, with the session phase; green and approved). Closing so the two don't clash.
Every finished merge-queue gate is now stored with the names of the tests that failed, the queue log names them, and they can be read from
GET /api/v1/repos/:o/:n/queue/gates?limit=20and a newqueue_gatesMCP tool.The likely cause of the "2 failed" / "4 failed" counts.
scripts/test-shards.tsends with a tally that prints every failure a second time, as(fail) name [1ms] [shard 3].failuresOfdidn't strip the[shard N]suffix, so it read the repeat as a different test and counted each failure twice. A recent "2 failed" was probably one test. Both parsers now use the same pattern and count each failing test once.What changed
src/checks/gate_tests.tsreads a gate's output for the passed count (summed across shards), the distinct failed count, and up to 20 failing tests. Each test has its file and up to 5 error lines, taken from bun'serror:line onward.landing.tsadds this to the red verdict, andqueue_store.tskeeps it when a verdict is resumed.gate_resulttable, written bysrc/checks/gate_result.tsthroughQueueStore.judged. Every finished gate gets a row (green, red orcut_off): batches, search prefixes and suspects gated alone. Each row has its pulls, reason, counts, failing tests, start and finish times and duration. Rows older than 7 days are deleted whenever a new one is written.gated 8 pulls: red (2 failed: "a > b", "c > d"). It names up to 5 tests and ends withand N morepast that. A red that named no test reads as before.{ items, next_cursor }, newest first.limitis 1–100 andcursoris a gate id. A bad cursor gets a 422, and a repository the reader can't see gets a 404. Its OpenAPI file and the MCP tool are generated in, anddocs/features/queue-gates.mdsays how to use it.flaky: truewhen the same file and test failed in two red gates that share no pull. It is computed from at most the last 500 red gates in the week the table keeps.What I chose not to do
gate_resultitself rather than ported from an existing list.Tests
test/merge-queue-gate-result.test.tsis new. A fake two-shard output with two failing tests gives both names once, with their files, errors and counts, andfailuresOfno longer doubles them. It also covers the 20-test cap, the log wording and the flaky rule.sweepGreenagainst a stand-in check lane. It checks that the log line names both tests, and that the API and the MCP tool return both withfailedmatching the number named. It also checks paging, the 404 for a private repository, the 422 for a bad cursor, the flaky flag and the 7-day pruning.test/mcp.test.tslists every MCP tool by name, soqueue_gateshad to be added to it.To try it:
bun test test/merge-queue-gate-result.test.ts. On the preview, after a gate has run,curl -H "authorization: Bearer $TOKEN" "$URL/api/v1/repos/trevato/forge/queue/gates?limit=5".🤖 Generated with Claude Code
Closes #6739