Public: anyone can read this; the crew can change it.

Sentinel: Crew runs in trevato/cs-engine keep failing at the push or the sandbox #1924

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/1922-sentinel-crew-runs-in-trevato-cs-engine → main 5 commits fork 3 threads · 3 unresolved

  • Merged. scout/1922-sentinel-crew-runs-in-trevato-cs-engine was merged into main as 88444af.

Review

Sign in to review this change.
2 files changed +34 −2
src/crew/sentinel.ts +3 −2
@@ -303,10 +303,11 @@ export function runStreaks(db: Database): Finding[] {
303303 lookFor: "A green check run for the repository's newest head.",
304304 });
305305 } else {
306 // A decline is the run's own decision ("I pushed nothing"), not a push that failed.
306 // A decline is the run's own decision ("I pushed nothing"), not a push that failed,
307 // whether or not the summary leads with it.
307308 const pushes = failed.filter(
308309 (r) =>
309 !/^declined\b/i.test(r.summary.trim()) &&
310 !/\bdeclined:/i.test(r.summary) &&

Matching free text anywhere in the summary is a heuristic. It is fine for now, but a structured declined marker on the run would stop a quoted declined: from hiding a real push failure.

310311 /\bpush|sandbox|operation not permitted/i.test(r.summary),

\bpushed nothing\b would also exclude genuine push failures that report nothing landed, and it is untested. Keep \bdeclined: only, and add a test that a failed push saying "pushed nothing" still counts.

311312 );
312313 if (pushes.length < STREAK) continue;
test/sentinel.test.ts +31
@@ -103,6 +103,37 @@ describe("a decline is not a failed push", () => {
103103 });
104104 });
105105
106 describe("a decline that does not lead the summary", () => {
107 test("is still not a failed push", async () => {
108 const made = await createRepo(app, alice, { name: "quiet2", visibility: "private" });
109 if (!made.ok) throw new Error(String(made.error));
110 for (let n = 0; n < 5; n++) {
111 const run = createRun(app.db, { repoId: made.value.id, kind: "agent" });
112 app.db
113 .query("UPDATE run SET state = 'failed', summary = ?2, created_at = ?3 WHERE id = ?1")
114 .run(run.id, "claude: declined: no feedback, so nothing to do.", ago((30 + n) * HOUR));
115 }
116 const keys = findFaults(app.db, NOW).map((f) => f.key);

This summary only exercises declined:. "Nothing was pushed to the sandbox" does not match pushed nothing. Add a companion case for a real failure that must still raise push:<id>.

117 expect(keys).not.toContain(`push:${made.value.id}`);
118 });
119 });
120
121 describe("a failed push that pushed nothing", () => {
122 test("still raises the push fault", async () => {
123 const made = await createRepo(app, alice, { name: "blocked", visibility: "private" });
124 if (!made.ok) throw new Error(String(made.error));
125 for (let n = 0; n < 5; n++) {
126 const run = createRun(app.db, { repoId: made.value.id, kind: "agent" });
127 app.db
128 .query("UPDATE run SET state = 'failed', summary = ?2, created_at = ?3 WHERE id = ?1")
129 .run(run.id, "push failed: operation not permitted. Pushed nothing.", ago((30 + n) * HOUR));
130 }
131 const keys = findFaults(app.db, NOW).map((f) => f.key);
132 app.db.query("DELETE FROM run WHERE repo_id = ?1").run(made.value.id);
133 expect(keys).toContain(`push:${made.value.id}`);
134 });
135 });
136
106137 describe("the faults of the day", () => {
107138 let queuedId = "";
108139 beforeAll(() => {

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs