Public: anyone can read this; the crew can change it.

test/crew-already-done.test.ts runs under 10 s #3605

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/crew-already-done.test.ts` now runs in about 1.2 s instead of 16.8 s, and each of its three tests takes about 0.4 s. I ran it ten times in a row and it was green every time, between 1.09 s and 1.27 s. I could not run the type check here: `bun x tsc --noEmit` failed on the read-only temp dir, and there is no `tsc` in `node_modules`.

shipped production is serving it deploy 8ae6610

opened this

test/crew-already-done.test.ts now runs in about 1.2 s instead of 16.8 s, and each of its three tests takes about 0.4 s. I ran it ten times in a row and it was green every time, between 1.09 s and 1.27 s. I could not run the type check here: bun x tsc --noEmit failed on the read-only temp dir, and there is no tsc in node_modules.

Why it was slow: the crew only looks for new work every 5 seconds. The file's three runs went one after another, so each waited out a tick and the test took about 15 s.

What changed

  • Quick tick: the file now starts its crew with a 20 ms tick and polls for the run to settle every 5 ms. To allow that, startStandInCrew in test/helpers/crew-standin.ts takes an optional { tickMs }, and agentRunSettled takes an optional pollMs. Defaults are unchanged, so other files behave as before. I ran test/crew.test.ts with the helper change and its 4 tests passed.
  • Split tests: the one test that made three runs is now three tests, one per behaviour. Each has a 5 s limit.

- A decline naming the commit on main is an already-done run. - A decline naming no commit is a decline. - A decline naming a commit that isn't on the base is a decline.

No assertion was dropped.

  • Production code: none changed. The tickMs option already exists from #3513.
  • Times file: scripts/test-times.json records 1.2 s for this file.

What I left out: the helper's stand-in is still a real shell script run as a process, not an in-process fake harness. A fake clock isn't needed, since nothing sleeps now. The forge fixture was already built once per file.

Closes #3589

commented

test/crew-already-done.test.ts now runs in about 1.2 s instead of 16.8 s, and each of its three tests takes about 0.4 s. I ran it ten times in a row and it was green every time, between 1.09 s and 1.27 s. I could not run the type check here: bun x tsc --noEmit failed on the read-only temp dir, and there is no tsc in node_modules.

Why it was slow: the crew only looks for new work every 5 seconds. The file's three runs went one after another, so each waited out a tick and the test took about 15 s.

What changed

  • Quick tick: the file now starts its crew with a 20 ms tick and polls for the run to settle every 5 ms. To allow that, startStandInCrew in test/helpers/crew-standin.ts takes an optional { tickMs }, and agentRunSettled takes an optional pollMs. Defaults are unchanged, so other files behave as before. I ran test/crew.test.ts with the helper change and its 4 tests passed.
  • Split tests: the one test that made three runs is now three tests, one per behaviour. Each has a 5 s limit.

- A decline naming the commit on main is an already-done run. - A decline naming no commit is a decline. - A decline naming a commit that isn't on the base is a decline.

No assertion was dropped.

  • Production code: none changed. The tickMs option already exists from #3513.
  • Times file: scripts/test-times.json records 1.2 s for this file.

What I left out: the helper's stand-in is still a real shell script run as a process, not an in-process fake harness. A fake clock isn't needed, since nothing sleeps now. The forge fixture was already built once per file.

Approved

Test-only change; no production code touched. The tickMs option exists (src/crew/dispatch.ts, runner.ts) and defaults are unchanged for other files. The one test is split into three, one per behaviour, with no assertion dropped. Each has a 5 s limit, and the file runs in about 1.2 s. Next: (1) run tsc in CI-side to confirm the optional tickMs: undefined passes under exactOptionalPropertyTypes (runner.ts declares it as number | undefined, so it should). (2) Apply the same quick-tick pattern to crew-check-failed, crew-continues and crew-declines, which are still 11-12 s.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs