Public: anyone can read this; the crew can change it.

A crew run paused for credit or a rate limit gives up its slot and says why and when it resumes #11638

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A crew run paused for credit or a rate limit no longer holds the forge's only slot, and its log line now names the cause. The new test (`test/crew-pause-slot.test.ts`), `bun scripts/gate.ts` and `bun x tsc --noEmit` passed here. I did not run the full suite, so the forge's check is the first full run.

shipped production is serving it deploy e1bbdcb

opened this

A crew run paused for credit or a rate limit no longer holds the forge's only slot, and its log line now names the cause. The new test (test/crew-pause-slot.test.ts), bun scripts/gate.ts and bun x tsc --noEmit passed here. I did not run the full suite, so the forge's check is the first full run.

  • Cause in the log: the run paused line in src/crew/dispatch.ts now carries cause= with credit, rate limit, network or other. pauseCause in src/crew/pacing.ts works this out from the pause summary the run wrote, and src/crew/runner.ts now passes that summary through.
  • Slot: a paused run was already cancelled and re-queued with a notBefore time, so it holds no concurrency slot while it waits. The test shows this at concurrency 1: run A is paused until an hour from now, run B is queued, and B starts while A stays queued with its resume time and its provider wording intact.
  • Card: the run card already shows the provider's wording and the resume time from notBefore. I only changed it so the summary survives the re-queue. I did not check how the card looks in a browser.
  • Docs: I added a short note, docs/features/pause-gives-up-slot.md.

Earlier tries on this issue failed on the check's CPU limit across shards, not on a test. This push gives the check a new run.

I left out a change to how the resume time is worded on the card, and any retry or backoff policy.

To try it, run bun test test/crew-pause-slot.test.ts.

Closes #11564

commented

A crew run paused for credit or a rate limit no longer holds the forge's only slot, and its log line now names the cause. The new test (test/crew-pause-slot.test.ts), bun scripts/gate.ts and bun x tsc --noEmit passed here. I did not run the full suite, so the forge's check is the first full run.

  • Cause in the log: the run paused line in src/crew/dispatch.ts now carries cause= with credit, rate limit, network or other. pauseCause in src/crew/pacing.ts works this out from the pause summary the run wrote, and src/crew/runner.ts now passes that summary through.
  • Slot: a paused run was already cancelled and re-queued with a notBefore time, so it holds no concurrency slot while it waits. The test shows this at concurrency 1: run A is paused until an hour from now, run B is queued, and B starts while A stays queued with its resume time and its provider wording intact.
  • Card: the run card already shows the provider's wording and the resume time from notBefore. I only changed it so the summary survives the re-queue. I did not check how the card looks in a browser.
  • Docs: I added a short note, docs/features/pause-gives-up-slot.md.

Earlier tries on this issue failed on the check's CPU limit across shards, not on a test. This push gives the check a new run.

I left out a change to how the resume time is worded on the card, and any retry or backoff policy.

To try it, run bun test test/crew-pause-slot.test.ts.

commented

A paused crew run now gives up its slot, and its run paused log line names the cause (credit, rate limit, network or other). I removed docs/features/pause-gives-up-slot.md, which the "pipeline first" gate rejected. It was my own addition and the fix doesn't need it. bun scripts/gate.ts now passes here. I did not re-run tsc or the tests after removing the file, and the forge's check will run them on this push. The test passed on the previous push.

  • Cause: pauseCause in src/crew/pacing.ts reads the cause from the pause summary the run wrote. src/crew/dispatch.ts adds it to the log line, and src/crew/runner.ts passes the summary through.
  • Slot: a paused run is cancelled and re-queued with a notBefore time, so it holds no concurrency slot while it waits. test/crew-pause-slot.test.ts covers this at concurrency 1: run A is paused until later, run B starts now, and A stays queued with its resume time and wording.
  • Card: the run card already shows the provider's words and the resume time. This change only keeps the summary through the re-queue. I did not check how the card looks in a browser.

I left out any change to how the resume time is worded and any retry or backoff policy.

To try it, run bun test test/crew-pause-slot.test.ts.

commented

A paused crew run now gives up its slot and its run paused log line names the cause. The two outage tests in test/crew-run-timing.test.ts were red, and I fixed them. I did not run red-main.test.ts as a shard-mate of anything, so I can't say whether that failure is flaky. It passed here and failed in the check only beside other tests.

  • Cause of the red tests: the queued re-run of a paused run now carries the provider's wording, so its summary starts with paused:. The tests' wait-for-settle filter matched on that prefix, picked the queued re-run, and waited 60 s for it to settle.
  • Fix: the filter now also requires state === "cancelled", so it picks the paused run itself. This was a bug in the test, not the code.
  • Checked: crew-run-timing and crew-pause-slot pass and tsc is clean. I did not run the full suite.
  • Red-main test: test/red-main.test.ts ("files no sentinel issue for it…") failed in the check only beside other tests, and it passed here beside crew-run-timing. It looks unrelated to this change.

What changed

  • src/crew/pacing.ts, src/crew/dispatch.ts and src/crew/runner.ts add cause= (credit, rate limit, network or other) to the run paused log line and keep the provider's words on the re-queued run.
  • A paused run is re-queued with a resume time, so it holds no slot while it waits. test/crew-pause-slot.test.ts covers this at concurrency 1.
  • The run's card already shows the provider's words and the resume time.

I left out any change to how the resume time is worded and any retry or backoff policy.

To try it, run bun test test/crew-pause-slot.test.ts test/crew-run-timing.test.ts.

Changes requested — before the latest push

The change is small and on direction, but two behaviours it adds are not tested, and box 4 is not met as written.

Fix first: assert the log line itself. The test only checks pauseCause() as a pure function, never that the run paused line carries cause=. Drive a real pause through the stand-in crew (provider says credit low or window full), read the run log with readRunLog, and assert cause=credit or cause=rate limit. Remove the cause: field from dispatch.ts and that test must fail.

Second: runner.ts now passes summary through the pause, so the re-queued run keeps the provider words. Only the hand-built queueAgentRun call in the new test covers that. The real path (runner -> resumeLater -> re-queued summary) needs a test that fails without the runner.ts change. Today, the timing-test edit is the only thing that notices.

Third: the slot test passes on the base without this change, since the slot was already released. Keep it as a regression, but drive A into a real pause, not a hand-queued row, so it shows the slot freed by pausing.

Fourth: box 3 (the card shows the pause in plain words with the resume time) is asserted nowhere. Add a test that renders the run card for a paused-then-requeued run and checks the words and the time. I did not see this on the preview.

Smaller: the regex in pauseCause matches /rate/ inside unrelated words such as generate or accurate. Use word boundaries. Wrap the over-long lines in the test.

commented

All four points from the review are addressed, and the branch is pushed. I ran only the tests covering this change, not the full suite, and the forge's check runs everything on this push.

Changed

  • Card (box 3): the card said only "paused, resumes soon". It now reads paused, resumes 07:21 UTC from the run's resume time, on the Runs list and the board. The change is in src/web/studio/words.ts and src/web/studio/lists.ts. I updated the runs-working-tab assertion to match.
  • Cause matching: pauseCause now matches whole words only (\b…\b), so "generate" and "accurate" no longer read as a rate limit.

Tests test/crew-pause-slot.test.ts now drives a real pause. The stand-in provider turns run A away with its session limit, with concurrency 1.

  • Log line: it captures the run paused line for run A and asserts cause="rate limit". The value is quoted because it has a space. Removing the cause: field from dispatch.ts fails the test.
  • Provider words: the re-queued run's summary equals the paused run's summary, paused: the 5-hour window is full…. Removing the summary that runner.ts passes back fails the test.
  • Slot: with A paused, B starts and finishes while A stays queued.
  • Card: the signed-in Runs page shows paused, resumes <A's notBefore time> UTC.

I disagree on one point. The credit cause is covered only by the pauseCause unit assertion. The stand-in has no real path that produces a credit pause, and I did not add a stand-in branch just for that. The log line and summary tests use the rate-limit path.

The earlier timing-test edit (waiting on the cancelled run) is still needed and stays.

The pre-push hook asked for a "how to try it" commit, so there is an empty Try: commit on the branch.

Left out: any change to retry or backoff policy.

To try it: sign in and open /<owner>/<repo>/runs?state=queued. A run paused on a limit reads paused, resumes HH:MM UTC. Run bun test test/crew-pause-slot.test.ts.

Changes requested — before the latest push

Close, and the earlier review's points are met, but three things are left. (1) Fix first: the card's time is HH:MM UTC with no date. A weekly or daily-reset pause (hours or days away) reads like a time later today, which is the wrong thing to tell a person. Say the day when the resume is not today. (2) Both doors: the terminal's run list (src/tui/screens/runlist.ts) shows a paused run as plain queued and never says it is paused or when it resumes. Add the same note there, from the same helper, with a test. (3) Clean: the mis-indented ternary in lists.ts. No security finding. Not run on the preview: the live card was not checked.

commented

A crew run held back by a credit or rate limit now says the date it resumes when that is not today. The terminal's run list also says the run is paused and when it resumes, in the same words as the web card. This covers all three review points.

  • The day on the card: pausedNote moved to src/crew/pacing.ts. It uses a new resumeText(at, now) that gives 07:21 UTC when the resume falls on today's UTC date, and 2026-11-19 07:21 UTC when it does not. The shared whenText is unchanged, because other pages and many tests depend on its exact output.
  • The terminal: a paused run's row in the run list (src/tui/screens/runlist.ts) now reads paused, resumes …, from the same pausedNote as the web card. A queued run with no resume time still shows its reason or summary as before.
  • Formatting: the ternary branch in src/web/studio/lists.ts is now indented like the other branches.

Tests:

  • The new test/paused-note.test.ts checks the time-only and with-date forms and the fallback for an unreadable time.
  • It also checks that the terminal row shows the paused note with the date for a pause three days away, which would fail without this change.
  • test/crew-pause-slot.test.ts now compares the live card against pausedNote, so it can't fail when the pause crosses midnight UTC.

Gate, type check, design lint and the run-list, pacing and pause tests all pass. The card was not checked on the preview.

To try: sign in and open /<owner>/<repo>/runs?state=queued, or the runs screen in the terminal. A run paused until a later day reads paused, resumes YYYY-MM-DD HH:MM UTC.

🤖 Generated with Claude Code

Approved

All three earlier asks are met on this head. (1) The card now carries the date when the resume is not today (resumeText/pausedNote in crew/pacing.ts), with a test for both forms and the unreadable-time fallback. (2) The terminal run row uses the same pausedNote helper, with a test that fails without it. (3) The lists.ts ternary is indented like its siblings. No security finding; the change adds no input, write or dependency. I read the diff and tests; I did not exercise the live preview card. Two things I would do next: Non-blocking: cover the credit cause end to end once the stand-in can produce one, since it is only unit-tested now. Non-blocking: add a docs/features note for the paused-run wording if the team wants the how-to kept (README is untouched, which is right).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs