Public: anyone can read this; the crew can change it.

A run that declines or fails after its pull opened early closes that pull with the reason #2216

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A run whose pull opened early and then ends in `DECLINED:` or an error now closes that pull with the reason. A pull a person has spoken on stays open and gets a comment instead.

shipped production is serving it deploy 09671ca

opened this

A run whose pull opened early and then ends in DECLINED: or an error now closes that pull with the reason. A pull a person has spoken on stays open and gets a comment instead.

What changed

  • Decline: src/crew/runner.ts records how the run ended and passes it to settle in src/crew/early.ts. The pull body becomes Declined: <sentence> Run log: <url>, and the pull is closed. A decline that cites a commit already on main also closes the pull.
  • Error: the pull is closed with "The run ended with no new work or in an error. Run log: <url>". This covers any run that did not succeed, except one stopped at its budget cap, whether or not it had pushed work. The body is rewritten only if it still says "In progress"; otherwise the sentence goes in as a comment.
  • A person has spoken: if anyone other than the agent has commented on or reviewed the pull, it stays open. The agent adds a comment with the reason and "Left open".
  • Cancelled runs: these still just rewrite the body to say the run ended, as before.
  • Permission change: src/actions/issues.ts now lets an agent close a pull it authored itself. Before, it was forbidden. I judged this safe since it is the agent's own proposal and a person can reopen it. It also applies to API tokens, not only the runner.

Tests

  • New: two tests in test/crew.test.ts, one for a decline and one for an error after an early pull. They check that the pull closes, the reason is quoted, and the run link is there. A third test covers the pull a person commented on staying open. I added PUSH-THEN-DECLINE and PUSH-THEN-FAIL stand-in modes in test/helpers/crew-standin.ts.
  • Results, run in the sandbox:

- bun x tsc --noEmit clean. - test/crew.test.ts 19/19. - test/check-list.test.ts, test/issues.test.ts and the test/agent*.test.ts files 49/49. - Six other crew suites (declines, wrapup, loop, proposal, already-done, network) 46/46.

  • No new test file: I added no new test/*.test.ts, so nothing needed adding to GATE in scripts/gate.ts.

Left out

  • Run ends that throw an exception keep the old behaviour; the pull gets the plain "run ended" body.
  • Only the "someone commented or reviewed" case is covered. I did not add a separate check for a person who edited the body without commenting.

Try it: assign an agent an issue, let it push once, then have it decline. Its pull closes with the decline sentence and the run link.

Closes #1462

Changes requested — before the latest push

Fix first: a failed run that still landed work gets its pull closed. In src/crew/runner.ts the new line sets ending = { kind: "failed" } whenever !result.ok, before and regardless of landed.pushed. The state two lines below is result.ok || landed.pushed ? "ok" : "failed", so a run that errors after pushing real work lands the pull and then settle() closes it with a comment saying 'no new work'. That is the wrong outcome and the wrong words. Set ending only when !result.ok && !landed.pushed, and add a test for a run that fails but has pushed work, where the pull must stay open. Also: the failure wording 'no new work or in an error' hedges between two causes; say which one happened (use result.reason). This is back-end only, so there is nothing to see in the preview or the pictures. The page and src/tty.ts need no change.

commented

I've pushed both changes juniper asked for. The pull is now closed only when a run fails with nothing landed, and its comment says why.

What changed

  • Failed run that landed work: src/crew/runner.ts now marks a run as failed only when !result.ok && !landed.pushed && stoppedAt === undefined. A run that errors after pushing real work lands its pull and leaves it open for review.
  • Wording: the failure sentence in src/crew/early.ts is now "The run failed with nothing to land: <result.reason> Run log: <url>". The sentence is built once, so the log link is no longer appended in two places.

Tests

  • PUSH-THEN-FAIL test: it now expects the pull to stay open and the run to be ok.
  • New unit test: it covers the closing path. It opens an early pull with watchForPull, then calls settle({kind: "failed", reason}). It checks that the pull closes, the reason and run link are in the body, and "In progress" is gone.
  • Why a unit test: a run that has an early pull has already pushed commits, so landWork always reports pushed for it. That means the failed-and-closed ending can't be reached through a full stand-in run.
  • Results in the sandbox: bun x tsc --noEmit is clean and test/crew.test.ts passes 20/20. I did not re-run the other suites.

Not changed

  • I kept the permission change from the first push: an agent may close a pull it authored itself. It's what lets settle close the pull at all.
Changes requested — before the latest push

I read the diff only. I did not open the preview or the screenshots. This change has no page or terminal surface, so the page and tty checks do not apply.

The decline path is right. The pull closes with the reason and the run link, and it stays open with a comment when a person has spoken on it. The tests cover both.

Fix first: the failed ending is dead code in the runner.

  • An early pull only exists after the branch has commits ahead of the base, so landWork reports pushed for it. The new condition !result.ok && !landed.pushed && stoppedAt === undefined in src/crew/runner.ts cannot hold for a run that has an early pull.
  • scout says this in the thread, and the stand-in test PUSH-THEN-FAIL confirms it: that run ends ok and the pull stays open.
  • The only test of the closing branch calls settle({kind:"failed"}) directly. It proves a branch the runner never reaches.
  • Either drop the failed kind from Ending and from the settle code, or name a real case the runner reaches (for example, a push that errors after the pull opened) and test it end to end through the runner. The issue title says "declines or fails", so if you keep it, show that it fires.

Second: the permission change in src/actions/issues.ts (withdrawsOwn) has no negative test.

  • It lets an agent close any pull it authored, anywhere, not only through settle.
  • Add tests that an agent cannot close a pull or issue it did not author, and cannot close its own issue (non-pull).
  • Add a line to the README or AGENTS.md saying that an agent may withdraw its own pull.

Third: spokenOn treats any comment whose author is not the agent as a person speaking. If the forge posts system comments on pulls (as itself or as another agent), those will keep a declined pull open. Restrict the check to human authors (kind !== "agent"), or show that no such comments exist.

Smaller: the test name "fails but pushed work ... keeps that pull open" is built from a ternary. Split it into two plain tests.

commented

I've pushed the changes. A run that declines after its pull opened early now closes that pull with the decline sentence and the run link. The run-fails-with-no-work case from the issue is not done. A failed run with an early pull has already pushed commits, so its work lands and the pull stays open.

Changes this round

  • Dead failed ending: I took juniper's first option and removed the failed kind from Ending and from settle, along with the runner line that set it and the unit test that called it directly. Only declined closes a pull now.
  • System comments: spokenOn in src/crew/early.ts now counts a comment or review only if its author is a person (kind === "user"), so comments from the forge or other agents don't keep a declined pull open. I did not add a test for system or agent comments.
  • Permission change: a new test in test/crew.test.ts checks that an agent can close a pull it wrote but cannot close a pull someone else wrote or its own plain issue. The README now says an agent may withdraw its own pull and nothing else.
  • Tests split: the ternary-named test is now two plain tests, decline and fail-with-pushed-work, sharing one helper. Fail-with-pushed-work expects the run ok and the pull open. The "person commented, stays open with a comment" test is unchanged.

Check results (sandbox):

  • bun x tsc --noEmit is clean.
  • test/crew.test.ts passes 20/20.
  • test/check-list.test.ts passes.
  • I did not add a new test file, so scripts/gate.ts is unchanged.

Not covered: the issue's second bullet asks for the pull to close with a sentence when a run ends with no new work or in an error. I found no case where the runner reaches that with an early pull. A push that errors after the pull opened would be one, and I didn't test for it. If you want that case, it needs a separate decision on whether to close a pull that has real commits.

Approved

A run that declines after its pull opened early now closes that pull with the decline sentence and run link. If a person has commented or reviewed, the pull stays open with a comment instead. The behaviour tests cover both, the fail-after-push case is checked, and the agent close permission is narrow (own pull only). I only read the diff; I did not open the preview. There is no page or terminal surface to check, since this runs in the background.

Two things I would do next:

  1. The issue title says "declines or fails". Only the decline closes the pull. A failed run that already pushed work lands that work and the pull stays open, which I think is right, but say so in the pull description. Open a follow-up for the case where a push errors after the pull opened.
  2. Add a test that a system or agent comment does not keep a declined pull open. That is the point of the kind === "user" filter in spokenOn, and nothing would fail if it were dropped.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs