Public: anyone can read this; the crew can change it.

test/usage.test.ts runs in under 10 s, every test under 1 s, proving what it proves today #10230

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/usage.test.ts` now runs in about 2.5 s, down from 12 s. All 7 tests still pass and none is over 1 s.

shipped production is serving it deploy f21b126

opened this

test/usage.test.ts now runs in about 2.5 s, down from 12 s. All 7 tests still pass and none is over 1 s.

Before / after (bun test test/usage.test.ts, crew machine):

  • Before: 7 pass, 0 fail, 54 expect() calls — Ran 7 tests across 1 file. [11.94s]
  • After, three runs: [2.42s], [2.51s], [2.55s], still 7 pass, 0 fail.
  • Slowest tests after (junit times): the crew-run test at 0.75 s, then the migration repair test at 0.31 s. Every other test is under 3 ms.

What changed (only test/usage.test.ts):

  • The slow test was "each run is attributed to the account it ran on, the app, the agent and the issue". It spent about 5 s waiting for the crew's queue to check again. In that test the first try, on the personal token, is refused and paused, and the retry on the work key waits for the queue's next check, which runs every 5 s by default. startCrew now gets tickMs: 20, the same fix #10223 made in test/crew-wrapup.test.ts.
  • The polling helper until now waits 10 ms between reads instead of 100 ms, so no wait over 50 ms remains.
  • The test now starts with its invariant in one line: a crew run's ledger line names its subscription, app, agent and issue, and only the owner can read it over the API and MCP.

What I chose not to do: no test was deleted and nothing was skipped. The other six tests already took milliseconds, and none proves the same thing as another. The two describe blocks build their app in different ways (one with the server and crew, one without), so I left their setup as it was.

Proof the rewritten test still guards its code: in src/crew/runner.ts:815 I changed subscriptionId: meter.subscriptionId to subscriptionId: null (the line where a finished run records its usage). The test then failed with - "subscriptionId": "01m70gg7…" / + "subscriptionId": null (6 pass, 1 fail). With the line restored it passes.

To try it: bun test test/usage.test.ts. bun scripts/gate.ts and bun x tsc --noEmit are both clean.

The issue asks for a short plan to be kept with the keep_plan tool. That tool wasn't available in this run, so I kept no plan on the issue.

🤖 Generated with Claude Code

Closes #10218

commented

test/usage.test.ts now runs in about 2.5 s, down from 12 s. All 7 tests still pass and none is over 1 s.

Before / after (bun test test/usage.test.ts, crew machine):

  • Before: 7 pass, 0 fail, 54 expect() calls — Ran 7 tests across 1 file. [11.94s]
  • After, three runs: [2.42s], [2.51s], [2.55s], still 7 pass, 0 fail.
  • Slowest tests after (junit times): the crew-run test at 0.75 s, then the migration repair test at 0.31 s. Every other test is under 3 ms.

What changed (only test/usage.test.ts):

  • The slow test was "each run is attributed to the account it ran on, the app, the agent and the issue". It spent about 5 s waiting for the crew's queue to check again. In that test the first try, on the personal token, is refused and paused, and the retry on the work key waits for the queue's next check, which runs every 5 s by default. startCrew now gets tickMs: 20, the same fix #10223 made in test/crew-wrapup.test.ts.
  • The polling helper until now waits 10 ms between reads instead of 100 ms, so no wait over 50 ms remains.
  • The test now starts with its invariant in one line: a crew run's ledger line names its subscription, app, agent and issue, and only the owner can read it over the API and MCP.

What I chose not to do: no test was deleted and nothing was skipped. The other six tests already took milliseconds, and none proves the same thing as another. The two describe blocks build their app in different ways (one with the server and crew, one without), so I left their setup as it was.

Proof the rewritten test still guards its code: in src/crew/runner.ts:815 I changed subscriptionId: meter.subscriptionId to subscriptionId: null (the line where a finished run records its usage). The test then failed with - "subscriptionId": "01m70gg7…" / + "subscriptionId": null (6 pass, 1 fail). With the line restored it passes.

To try it: bun test test/usage.test.ts. bun scripts/gate.ts and bun x tsc --noEmit are both clean.

The issue asks for a short plan to be kept with the keep_plan tool. That tool wasn't available in this run, so I kept no plan on the issue.

🤖 Generated with Claude Code

Approved

Clean, minimal fix: tickMs 20 removes the ~5 s wait for the queue tick and the poll drops to 10 ms. The invariant line is at the top of the slow test, and nothing is skipped or deleted. The before/after (11.94 s to about 2.5 s, slowest test 0.75 s) is quoted in the pull, and the mutation check shows the test still guards runner.ts. I did not run the suite myself. No security, dependency or README concerns.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs