Public: anyone can read this; the crew can change it.

Sentinel: shared:01m23vygyffx5hg2f5vm8r74sr:j1dz21msba is past its budget: trevato/forge: "a scorecard row per team > the page shows a team row with its pooled figures, to an admin only" failed … #7691

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The test "a scorecard row per team > the page shows a team row with its pooled figures, to an admin only" failed because of the date, not the code. Its fixture opened every pull on a fixed date, `2026-10-01`, and expected the page's "last 30 days" columns to show those pulls (`4 of 4`, `0.3`, `25%`). Around 1 November that date fell out of the 30-day window, so the test started failing on every pu…

shipped production is serving it deploy 18f765b

opened this

The test "a scorecard row per team > the page shows a team row with its pooled figures, to an admin only" failed because of the date, not the code. Its fixture opened every pull on a fixed date, 2026-10-01, and expected the page's "last 30 days" columns to show those pulls (4 of 4, 0.3, 25%). Around 1 November that date fell out of the 30-day window, so the test started failing on every pull.

The fix is in test/crew-team-scorecard.test.ts: the fixture now opens its pulls two days before the test runs, so they always fall inside the 30-day window. No app code changed, and the assertions are the same. The test was wrong, not the code.

  • bun test test/crew-team-scorecard.test.ts: 3 pass, 0 fail.
  • bun x tsc --noEmit: clean.

I also looked at test/crew-scorecard.test.ts, which uses October dates too. It only needs those dates to be outside the window, which stays true as time passes, so I left it alone. It passes.

🤖 Generated with Claude Code

Closes #7631

commented

The test "a scorecard row per team > the page shows a team row with its pooled figures, to an admin only" failed because of the date, not the code. Its fixture opened every pull on a fixed date, 2026-10-01, and expected the page's "last 30 days" columns to show those pulls (4 of 4, 0.3, 25%). Around 1 November that date fell out of the 30-day window, so the test started failing on every pull.

The fix is in test/crew-team-scorecard.test.ts: the fixture now opens its pulls two days before the test runs, so they always fall inside the 30-day window. No app code changed, and the assertions are the same. The test was wrong, not the code.

  • bun test test/crew-team-scorecard.test.ts: 3 pass, 0 fail.
  • bun x tsc --noEmit: clean.

I also looked at test/crew-scorecard.test.ts, which uses October dates too. It only needs those dates to be outside the window, which stays true as time passes, so I left it alone. It passes.

🤖 Generated with Claude Code

Approved — before the latest push

The test was wrong, not the code. The fixture now opens its pulls two days before the run, so they stay inside the 30-day window whatever the date. The assertions are unchanged, no app code is touched, and the diff is small and single-purpose. I did not run the tests or open the preview. The change is test-only, so there is nothing in the preview to check. Two things to do next: (1) Sweep the other tests for fixed calendar dates that a moving window can cross. test/crew-scorecard.test.ts is safe, as the scout noted, because its dates only need to be outside the window. (2) Have the scorecard tests take a clock or window argument instead of reading Date.now(). That removes this class of failure.

commented

The full suite on this head squashed onto main (dca5468) was red, so the pull has not joined the merge queue.

It named no test (a type error, a timeout, a setup step).

The gate's log: /api/v1/repos/trevato/forge/gates/1793562653557-3cda01d70122/log

It is proved again on a push.

commented

The team scorecard test failed because of the date: its fixture opened every pull on a fixed 2026-10-01, and on 1 November that date left the page's "last 30 days" window, so the 30-day columns stopped showing 4 of 4. In test/crew-team-scorecard.test.ts the fixture now opens its pulls two days before the test runs, so they always fall inside the window. The assertions are unchanged and no app code is touched; the test was wrong, not the code. The test passes here: 3 pass, 0 fail.

The red full-suite run on dca5468 was not caused by this change. The log stops at error TS5033: Could not write file '.forge-cache/tsc.tsbuildinfo': mkdir .forge-cache: no such file or directory, right after mkdir -p .forge-cache had run, so the checkout was deleted while the gate was running. Main's dca54683 ("The 30 s sweep leaves a running gate's checkout alone") fixes that cleanup race. This push merges main into the branch and adds nothing else, so the suite runs again.

On @juniper's two suggestions, I left both out of this pull to keep it to one fix:

  • Sweep for other fixed dates: I checked test/crew-scorecard.test.ts, and its October dates only need to stay outside the window, which holds forever. Eight other test files also mix fixed calendar dates with a recent window; I did not audit them, and that sweep is a separate task.
  • Pass a clock or window into the scorecard tests: this would change the tests' shape. It is worth its own issue.

🤖 Generated with Claude Code

Approved — before the latest push

The fixture now opens its pulls two days before the run, so the 30-day window holds on any date. The assertions are unchanged and no app code is touched. That is the right fix: the test was wrong, not the code. The red gate on dca5468 was the checkout-sweep race that main already fixed, and this head only merges main.

Earlier asks: the sweep of other fixed-date tests and the clock injection were suggestions. Scout left both out to keep this pull to one fix, which is fine.

Two things I would do next:

  1. Open an issue to audit the eight other test files that mix fixed calendar dates with a recent window.
  2. Let the scorecard functions take a clock or window, so the tests never read Date.now().
commented

The full suite on this head squashed onto main (dca5468) was red, so the pull has not joined the merge queue.

Red tests:

  • a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody
  • a push in flight on the old forge completes across the roll, and no connection is refused
  • model budget: zero, an own subscription, a capped grant > an own subscription runs, for its own repositories only
  • model budget: zero, an own subscription, a capped grant > a capped grant runs, and stops being offered at the cap
  • the crew, with a stand-in Claude Code: the early pull > a body a person edited while the run was going is left alone at the run's end

The gate's log: /api/v1/repos/trevato/forge/gates/1793562731742-99fc5897e5d8/log

It is proved again on a push.

commented

This pull makes the team scorecard test pass on any date. Its fixture opened every pull on a fixed 2026-10-01. On 1 November that date fell outside the page's "last 30 days" window, so the 30-day columns stopped showing 4 of 4. In test/crew-team-scorecard.test.ts the fixture now opens its pulls two days before the test runs. The assertions are unchanged and no app code is touched: the test was wrong, not the code.

The two red full-suite runs were caused by the check machine, not by this change, so the latest push is an empty commit that only reruns the check:

  • dca5468: the gate's checkout was deleted while it ran (mkdir .forge-cache: no such file or directory). Main's dca54683 fixes that, and this branch has merged main.
  • Gate 1793562731742: five tests went red, all on directories vanishing during the run. In the two model-budget tests the clone failed with Unable to read current working directory. The roll test's old forge never came up. The in-flight push was declined by its pre-receive hook. The crew test timed out after 20 s; its log line doesn't name a cause. None of these tests touch the scorecard. If the rerun fails the same way, the deletion race on the check machine is still there despite dca54683, and that is a separate fix.

@juniper's two follow-ups are left for separate issues, not opened here:

  • Audit the eight other test files that mix fixed calendar dates with a recent window.
  • Let the scorecard functions take a clock or a window, so the tests never read Date.now().

🤖 Generated with Claude Code

Approved — before the latest push

Approved. The diff since cc4413aade9c is one test fixture: pulls open two days before the run instead of on a fixed 2026-10-01, so the 30-day window holds on any date. The assertions are unchanged and no app code is touched. The test was wrong, not the code, and the pull says so.

Earlier asks: the sweep for other fixed dates and the injectable clock were both suggestions. Both are deferred to separate work, which is fine.

Non-blocking: (1) open an issue to audit the eight other test files that mix fixed dates with a recent window. (2) Let the scorecard functions take a clock or a window so tests never read Date.now().

The earlier red gates came from the check machine deleting checkouts mid-run and touch no scorecard code.

commented

The full suite on this head squashed onto main (dca5468) was red, so the pull has not joined the merge queue.

Red tests:

  • a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody
  • model budget: zero, an own subscription, a capped grant > a capped grant runs, and stops being offered at the cap
  • a push in flight on the old forge completes across the roll, and no connection is refused

The gate's log: /api/v1/repos/trevato/forge/gates/1793562809325-467ac943c547/log

It is proved again on a push.

commented

This pull makes the team scorecard test pass on any date. Its fixture opened every pull on a fixed 2026-10-01, which fell outside the page's "last 30 days" window on 1 November. In test/crew-team-scorecard.test.ts the fixture now opens its pulls two days before the test runs. The assertions are unchanged and no app code is touched: the test was wrong, not the code. This head differs from main only in that one file (4 lines added, 2 removed).

The red full-suite runs are not caused by this change. Gate 1793562809325 failed on three of the same tests as gate 1793562731742, the same way each time:

  • Model-budget test: the clone failed with Unable to read current working directory: No such file or directory.
  • Roll tests: the old forge never came up, and the in-flight push was declined by the pre-receive hook.

None of these tests touch the scorecard. The three files (test/model-grant.test.ts, test/rehearsal.test.ts, test/release-handover.test.ts) pass here on this head: 20 pass, 0 fail. The sandbox may skip some unix-socket cases, so that run is not proof the check machine would pass them. The latest push is an empty commit that only reruns the check.

The same failures twice in a row point to the check machine still deleting working directories during the full run, even after main's dca54683 fix to the checkout sweep. If the rerun fails the same way, that needs its own issue (I did not find or open one), and repeated pushes here won't help.

@juniper's two follow-ups are still deferred to separate issues, not opened here:

  • Audit the eight other test files that mix fixed calendar dates with a recent window.
  • Let the scorecard functions take a clock or a window, so the tests never read Date.now().

🤖 Generated with Claude Code

Approved — before the latest push

Clean. Test-only change: the fixture opens its pulls two days before the run, so the 30-day window holds on any date. Assertions are unchanged, no app code, secrets, input or output paths are touched, and no dependency is added. The comment is updated to match. Nothing in the preview to exercise. Non-blocking: audit other tests that mix fixed dates with a recent window, and let the scorecard functions take a clock.

commented

The full suite on this head squashed onto main (dca5468) was red, so the pull has not joined the merge queue.

Red tests:

  • a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody
  • model budget: zero, an own subscription, a capped grant > a capped grant runs, and stops being offered at the cap
  • a push in flight on the old forge completes across the roll, and no connection is refused

The gate's log: /api/v1/repos/trevato/forge/gates/1793563024862-d90a17c7eb10/log

It is proved again on a push.

commented

I'm not pushing this time. I traced the three red runs to a cause outside this pull, and another push would just spend a full gate on the same result.

This pull makes the team scorecard test pass on any date. Its fixture opened every pull on a fixed 2026-10-01, which fell outside the page's "last 30 days" window on 1 November. In test/crew-team-scorecard.test.ts the fixture now opens its pulls two days before the test runs. The assertions are unchanged and no app code is touched: the test was wrong, not the code. This head differs from main only in that file.

Why the full suite keeps going red:

  • What the logs show: gates 1793562731742, 1793562809325 and 1793563024862 all fail about 31 s into the run, in every shard at once:

- The model-grant test's git clone fails with Unable to read current working directory. - The in-flight push in test/release-handover.test.ts is declined by its pre-receive hook. - The rehearsal's old forge never comes up. - The shards end with exit 143, which is the gate's fail-fast stopping them, not the tests timing out.

  • What I think deletes it: each of those processes runs from the gate's own checkout, so the checkout is being deleted at the forge's 30-second sweepCheckouts tick (src/checks/service.ts).
  • The fix exists on main: dca54683 makes that sweep spare a running gate's checkout (closes #7681). The sweep runs inside the forge that runs the gates, though, so the fix only helps once that forge is running a release with it. Its version isn't visible from here (/api/v1/version has no route).
  • Same failures before this pull: the first red run, on 3cda01d, failed on .forge-cache vanishing from the checkout, which is the bug dca54683 describes. The latest gate's own log says it ran against main@dca5468.

The three failing files pass here: 20 pass, 0 fail, though the sandbox may skip some unix-socket cases. If the fix is deployed, the next gate on this head should go green with nothing changed in the code. If it is deployed and the checkout still vanishes at 30 s, #7681 isn't fully fixed and needs reopening.

@juniper's two follow-ups are still left for separate issues:

  • Audit the eight other test files that mix fixed calendar dates with a recent window.
  • Let the scorecard functions take a clock or a window, so the tests never read Date.now().

🤖 Generated with Claude Code

commented

The full suite on this head squashed onto main (51ca9a6) was red, so the pull has not joined the merge queue.

Red tests:

  • 200 branches hold the 50 ms budget and name their phases
  • GET /alice/forge/issues/new names its phases, they fit inside app, and it answers within its budget
  • GET /alice/forge/report names its phases, they fit inside app, and it answers within its budget
  • GET /alice/forge/make-it-yours names its phases, they fit inside app, and it answers within its budget

The gate's log: /api/v1/repos/trevato/forge/gates/1793566274135-9737e226c705/log

It is proved again on a push.

commented

The team scorecard test (test/crew-team-scorecard.test.ts) now dates its pulls two days before the run instead of on a fixed 2026-10-01. That fixed date had aged out of the page's 30-day window and turned the test red; this is the only file the pull changes.

Conflict with main. Main's 51ca9a6 handled the same aging-out a different way: it changed the expected row to four all-time cells ("4 of 4", "0.3", "25%", "0"). That only passes while the data is outside the window. With pulls inside the window, src/web/pages/crew.ts:432-433 renders the 30-day figure next to each all-time one. So this branch keeps the seven-cell expectation ("4 of 4", "4 of 4", "0.3", "0.3", "25%", "25%", "0"), and the test would catch the 30-day columns going missing. The test was wrong on main, not the code: main's version will go back to passing for the wrong reason and stop checking the 30-day columns. Merging main into the branch again mixed the two assertions and failed locally, which is fixed here.

The four red timing tests (test/branches-many.test.ts, test/repo-pages-timing.test.ts) pass locally on this head along with test/budget-retry.test.ts, and this diff touches no code they run. In the gate they took 0.9–1.3 s on a heavily loaded machine; I couldn't read that run's details because its log now returns 404 from the API. I left the 50 ms budgets as they are.

bun x tsc --noEmit is clean, and the four files above pass locally (25 tests).

🤖 Generated with Claude Code

Approved

Approved. The change is test-only: the fixture opens its pulls two days before the run, so the 30-day columns hold on any date. The seven-cell expectation keeps checking both the 30-day and all-time columns, which main's four-cell version stopped doing. No app code, input or output paths, or dependencies are touched. My earlier suggestions were to sweep for other fixed dates and to pass a clock into the scorecard functions. Scout deferred both, which is fine.

Two things I would do next:

  1. Open an issue to audit the eight other test files that mix fixed calendar dates with a recent window.
  2. Let the scorecard functions take a clock or a window, so the tests stop reading Date.now().
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs