Public: anyone can read this; the crew can change it.

check.txt holds with ?wait= until a pending check's verdict changes #12516

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

torque/forge:torque/12101-check-txt-holds-with-wait-until-a-pendin → main 8 commits fork 2 threads · 2 unresolved

  • Merged. torque/12101-check-txt-holds-with-wait-until-a-pendin was merged into main as 0220c82.

Review

Sign in to review this change.
4 files changed +246 −8
docs/features/contribute-token.md +1 −1
@@ -22,4 +22,4 @@ or the MCP `start` tool with `action: "contribute"` (same arguments). The answer
2222
2323 **The push opens the pull (#12013).** A push of a new `contrib/<agent>/<segment>/<topic>` branch opens a pull on the default branch (the tip commit's first line is the title, its whole message the body); a later push to the branch updates that pull. While nobody has approved the pull the push also prints "checks run now, behind the owner's; the preview is deployed once a person who can write here approves the pull on its page". The push output prints, as `remote:` lines, the pull's URL and `checks (plain text)`, the URL of `GET /api/v1/repos/<owner>/<name>/pulls/<n>/checks?format=text` (`state:`, `head:`, `required:`). Refused with one line: a push to the default branch or another token's branch (the line names the branches the token may push), a token past its expiry (401), a push over 5 MiB (`CONTRIBUTE_PUSH_MAX_BYTES`; git says "pack exceeds maximum allowed size (5.00 MiB)"), and a new branch once the token has 5 pulls open has opened 20 in a day, the agent name has opened 25, or the repository has taken 30 in a day (the line carries the number). A push that creates more new branches than the token has room to open pulls is refused whole with both numbers. Opens are serialized per token (`inTurn`), shared by the API and the push, with the caps read inside the turn; a branch that loses a race to a cap is still pushed and the output says no pull was opened. Code: `src/actions/contribute_push.ts`; test: `test/contribute-push.test.ts`.
2424
25 **Reading its own check.** A token reads the plain-text check of the pulls it opened, and only those (another pull's, or an unknown one, is 403): `curl -H "authorization: Bearer $TOKEN" https://<forge>/<owner>/<repo>/pulls/<n>/check.txt` answers `pending`, `green` or `red`, `head: <oid>` and, on red, the last 80 lines of the log (`src/http/check_text.ts`).
25 **Reading its own check.** A token reads the plain-text check of the pulls it opened, and only those (another pull's, or an unknown one, is 403): `curl -H "authorization: Bearer $TOKEN" https://<forge>/<owner>/<repo>/pulls/<n>/check.txt` answers `pending`, `green` or `red`, `head: <oid>` and, on red, the last 80 lines of the log (`src/http/check_text.ts`). Add `?wait=S` (a whole number, 1 to 120) to hold the request until the verdict changes or S seconds pass, then read the same text (a hold ends at 115 s at the longest, ahead of the server's 120 s idle limit; it wakes on run events, so a push whose check is not yet queued is seen at the next event or the limit); any other `wait` is a 400. At most 2 waits are held per token at once (a third is 429), and each still counts against the 120 requests a minute.
src/http/check_text.ts +100 −7
@@ -3,6 +3,14 @@
33 * text, for a script or an agent. Line 1 is pending, green or red; line 2 is
44 * `head: <oid>`; on red, a blank line and the last 80 lines of the log, but
55 * only when the log door (`runLog`) would show that reader the log.
6 *
7 * `?wait=S` (S in 1..120) holds the request until the latest check's verdict
8 * differs from the one it began with, or S seconds pass, then answers the same
9 * text. The hold listens to the run events on the bus (no polling) and lets go
10 * of its listener and timer on every exit, the client going away included. At
11 * most HELD_MAX requests are held per contribute token (the next is 429).
12 * Only run events release a hold: a head that moves with no run event (a push
13 * whose check is not yet queued) is seen at the next event or at the limit.
614 */
715
816 import { getRepoRun, runLog } from "../actions/runs.ts";
@@ -10,11 +18,92 @@ import { readRunLogTail } from "../runs/log.ts";
1018 import { checkSummary } from "../core/runs.ts";
1119 import { loadPull } from "./api.ts";
1220 import type { Ctx } from "./context.ts";
13 import { securityFloor } from "./respond.ts";
21 import { apiError, securityFloor, tryAgain } from "./respond.ts";
22 import type { App } from "../app.ts";
1423
1524 export const TAIL_MAX = 80;
1625
17 export function checkText(ctx: Ctx): Response {
26 export const WAIT_MIN = 1;
27 export const WAIT_MAX = 120;
28 /** Requests held at once per contribute token. */
29 export const HELD_MAX = 2;
30 /**
31 * The longest a hold lasts. The server drops a connection idle for 120 s
32 * (idleTimeout in src/http/server.ts), so a `wait=120` is answered at this
33 * mark, just ahead of it, rather than racing it.
34 */
35 export const HOLD_MAX = 115;
36
37 const held = new WeakMap<App, Map<string, number>>();
38
39 /** `?wait=` as seconds; undefined when absent, null when not an integer in range. */
40 function waitOf(url: URL): number | undefined | null {
41 const raw = url.searchParams.get("wait");
42 if (raw === null) return undefined;
43 if (!/^[0-9]{1,4}$/.test(raw)) return null;
44 const n = Number(raw);
45 return n >= WAIT_MIN && n <= WAIT_MAX ? n : null;
46 }
47
48 interface Reading {
49 readonly word: string;
50 readonly response: Response;
51 }
52
53 export async function checkText(ctx: Ctx): Promise<Response> {
54 const seconds = waitOf(ctx.url);
55 if (seconds === null)
56 return apiError("invalid_wait", 400, `wait is a whole number of seconds from ${WAIT_MIN} to ${WAIT_MAX}`);
57 const first = read(ctx);
58 if (first instanceof Response || seconds === undefined) return first instanceof Response ? first : first.response;
59
60 // Only a contribute token's holds are counted: it is the credential an agent leaves running.
61 const key = ctx.auth.contribute?.tokenId;
outdated

Anonymous readers all share the key 'anonymous' with cap 20: one caller can exhaust it for everyone. Not requested by the issue; drop or key per socket address.

62 const cap = HELD_MAX;
63 const counts = held.get(ctx.app) ?? new Map<string, number>();
64 held.set(ctx.app, counts);
65 const now = key === undefined ? 0 : (counts.get(key) ?? 0);
66 if (key !== undefined && now >= cap)
67 return tryAgain("too_many_held", 429, `at most ${cap} check.txt waits are held at once; retry when one ends`, 1);
68 if (key !== undefined) counts.set(key, now + 1);
69
70 const issueId = first.pullId;
71 const changed = Promise.withResolvers<void>();
72 let unsubscribe: (() => void) | undefined;
73 let timer: ReturnType<typeof setTimeout> | undefined;
74 const onAbort = () => changed.resolve();
75 const release = () => {
76 unsubscribe?.();
77 unsubscribe = undefined;
78 if (timer !== undefined) clearTimeout(timer);
79 timer = undefined;
80 ctx.request.signal.removeEventListener("abort", onAbort);
81 if (key === undefined) return;
82 const left = (counts.get(key) ?? 1) - 1;
83 if (left <= 0) counts.delete(key);
84 else counts.set(key, left);
85 };
86 try {
87 unsubscribe = ctx.app.bus.subscribe((event) => {
88 if (!event.kind.startsWith("run.") || event.payload.issueId !== issueId) return;
89 const again = read(ctx);
90 if (again instanceof Response || again.word !== first.word) changed.resolve();
91 });
92 timer = setTimeout(() => changed.resolve(), Math.min(seconds, HOLD_MAX) * 1000);
93 if (ctx.request.signal.aborted) changed.resolve();
94 else ctx.request.signal.addEventListener("abort", onAbort, { once: true });
95 // The state may have moved between the first reading and the listener.
96 const check = read(ctx);
97 if (check instanceof Response || check.word !== first.word) changed.resolve();
98 await changed.promise;
99 } finally {
100 release();
101 }
102 const last = read(ctx);
103 return last instanceof Response ? last : last.response;
104 }
105
106 function read(ctx: Ctx): (Reading & { pullId: string }) | Response {
18107 const found = loadPull(ctx);
19108 if ("error" in found) return found.error;
20109 const { repo, pull } = found;
@@ -34,9 +123,13 @@ export function checkText(ctx: Ctx): Response {
34123 })();
35124 if (lines.length > 0) body += `\n${lines.slice(-TAIL_MAX).join("\n")}\n`;
36125 }
37 return securityFloor(
38 new Response(body, {
39 headers: { "content-type": "text/plain; charset=utf-8", "cache-control": "no-store" },
40 }),
41 );
126 return {
127 word,
128 pullId: pull.id,
129 response: securityFloor(
130 new Response(body, {
131 headers: { "content-type": "text/plain; charset=utf-8", "cache-control": "no-store" },
132 }),
133 ),
134 };
42135 }
test/contribute-check-text.test.ts +144
@@ -144,3 +144,147 @@ test("a contribute token reads the check of its own pull, tail included, and no
144144 const theirs = addPull("o1"); // the owner's pull
145145 expect((await as(theirs.number, a.token)).status).toBe(403);
146146 }, SLOW);
147
148 const finish = (issueId: string, runId: string, state: string) => {
149 app.db.query("UPDATE run SET state = ? WHERE id = ?").run(state, runId);
150 app.bus.append({ kind: "run.finished", repoId, payload: { issueId, runId, state } });
151 };
152
153 // Counts the bus subscriptions the held requests make, so a test waits for
154 // them to exist instead of guessing how long a loaded machine takes.
155 const countSubs = () => {
156 const subscribe = app.bus.subscribe.bind(app.bus);
157 const c = { live: 0, restore: () => void (app.bus.subscribe = subscribe) };
158 app.bus.subscribe = ((...a: Parameters<typeof subscribe>) => {
159 c.live++;
160 const off = subscribe(...a);
161 return () => {
162 c.live--;
163 off();
164 };
165 }) as typeof app.bus.subscribe;
166 return c;
167 };
168 const until = async (ok: () => boolean) => {
169 for (let i = 0; i < 400 && !ok(); i++) {
170 await Bun.sleep(25); // real-timer: polls for a subscription made by another request
171 }
172 if (!ok()) throw new Error("the held request never subscribed");
173 };
174
175 test("?wait= returns the new verdict as soon as the check finishes, before the wait ends", async () => {
176 const p = addPull("w1");
177 const id = run(p.id, "w1", "running");
178 const started = performance.now();
179 const subs = countSubs();
180 try {
outdated

Add a test where the held request is aborted by the client and the place and subscription are shown released.

181 const held = fetch(`${servers.url}/alice/notes/pulls/${p.number}/check.txt?wait=30`);
182 await until(() => subs.live >= 1);
183 finish(p.id, id, "ok");
184 var res = await held;
185 } finally {
186 subs.restore();
187 }
188 expect(await res.text()).toBe("green\nhead: w1\n");
189 expect(performance.now() - started).toBeLessThan(10_000);
190 }, SLOW);
191
192 test("?wait= ends at its limit with pending when nothing changes", async () => {
193 const p = addPull("w2");
194 run(p.id, "w2", "running");
195 const started = performance.now();
196 const res = await fetch(`${servers.url}/alice/notes/pulls/${p.number}/check.txt?wait=1`);
197 expect(await res.text()).toBe("pending\nhead: w2\n");
198 expect(performance.now() - started).toBeGreaterThanOrEqual(900);
199 }, SLOW);
200
201 test("?wait= outside 1..120, or not an integer, is a 400 naming the range", async () => {
202 const p = addPull("w3");
203 for (const bad of ["0", "121", "-1", "x", "1.5", ""]) {
204 const res = await fetch(`${servers.url}/alice/notes/pulls/${p.number}/check.txt?wait=${bad}`);
205 expect(res.status).toBe(400);
206 expect(await res.text()).toContain("1 to 120");
207 }
208 });
209
210 test("a token holds at most two waits; the third is 429, and a finished wait frees its place", async () => {
211 const m = await mintContributeToken(app, { agent: "codex", origin: "t", repo: "alice/notes" }, "203.0.113.50", "https://forge.test");
212 if (!m.ok) throw new Error(JSON.stringify(m.error));
213 const v = verifyToken(app.db, m.value.token) as { ok: true; value: { user: User } };
214 const mine = createIssue(app.db, { repoId, authorId: v.value.user.id, title: "held", kind: "pull" });
215 if (!mine.ok) throw new Error(mine.error);
216 app.db
217 .query("INSERT INTO pull (issue_id, base_branch, head_branch, base_oid, head_oid) VALUES (?, 'main', 'h-w4', 'b', 'w4')")
218 .run(mine.value.id);
219 const id = run(mine.value.id, "w4", "running");
220 const go = (wait: number) =>
221 fetch(`${servers.url}/alice/notes/pulls/${mine.value.number}/check.txt?wait=${wait}`, {
222 headers: { authorization: `Bearer ${m.value.token}` },
223 });
224 const subs = countSubs();
225 try {
226 const one = go(30);
227 const two = go(30);
228 await until(() => subs.live >= 2);
229 expect((await go(30)).status).toBe(429);
230 finish(mine.value.id, id, "failed");
231 expect((await one).status).toBe(200);
232 expect((await two).status).toBe(200);
233 expect((await go(1)).status).toBe(200);
234 } finally {
235 subs.restore();
236 }
237 }, SLOW);
238
239 test("a client that goes away frees its place and its bus subscription", async () => {
240 const m = await mintContributeToken(app, { agent: "codex", origin: "t", repo: "alice/notes" }, "203.0.113.51", "https://forge.test");
241 if (!m.ok) throw new Error(JSON.stringify(m.error));
242 const v = verifyToken(app.db, m.value.token) as { ok: true; value: { user: User } };
243 const mine = createIssue(app.db, { repoId, authorId: v.value.user.id, title: "abort", kind: "pull" });
244 if (!mine.ok) throw new Error(mine.error);
245 app.db
246 .query("INSERT INTO pull (issue_id, base_branch, head_branch, base_oid, head_oid) VALUES (?, 'main', 'h-w5', 'b', 'w5')")
247 .run(mine.value.id);
248 run(mine.value.id, "w5", "running");
249 let live = 0;
250 const subscribe = app.bus.subscribe.bind(app.bus);
251 app.bus.subscribe = ((...a: Parameters<typeof subscribe>) => {
252 live++;
253 const off = subscribe(...a);
254 return () => {
255 live--;
256 off();
257 };
258 }) as typeof app.bus.subscribe;
259 try {
260 const go = (signal?: AbortSignal) =>
261 fetch(`${servers.url}/alice/notes/pulls/${mine.value.number}/check.txt?wait=30`, {
262 headers: { authorization: `Bearer ${m.value.token}` },
263 signal: signal ?? null,
264 });
265 const ctl = new AbortController();
266 const a = go(ctl.signal).catch(() => "gone");
267 const b = go(ctl.signal).catch(() => "gone");
268 await until(() => live >= 2);
269 expect(live).toBe(2);
270 ctl.abort();
271 expect(await a).toBe("gone");
272 expect(await b).toBe("gone");
273 for (let i = 0; i < 40 && live > 0; i++) {
274 await Bun.sleep(50); // real-timer: the server sees the close
275 }
276 expect(live).toBe(0);
277 // Both places are free: two new holds are held, not 429, and a third is.
278 const c = new AbortController();
279 const x = go(c.signal).catch(() => "gone");
280 const y = go(c.signal).catch(() => "gone");
281 await until(() => live >= 2);
282 expect(live).toBe(2);
283 expect((await go()).status).toBe(429);
284 c.abort();
285 await x;
286 await y;
287 } finally {
288 app.bus.subscribe = subscribe;
289 }
290 }, SLOW);
test/timers-named.test.ts +1
@@ -72,6 +72,7 @@ const ALLOWED: readonly Allowed[] = [
7272 ["src/http/flow_stream.ts", "() => { emit(\"flow.settled\""],
7373 ]),
7474 ...allow("setTimeout", `a shell session's resize debounce, ${PER_CONNECTION}`, [["src/machines/shell.ts", "() => session.size === resize"]]),
75 ...allow("setTimeout", `a check.txt wait's deadline, ${PER_CONNECTION}`, [["src/http/check_text.ts", "() => changed.resolve()"]]),
7576 ...allow("setTimeout", `an ssh login's auth deadline, ${PER_CONNECTION}`, [["src/ssh/server.ts", "() => { this.transport.disconnect("]]),
7677 ...allow("setTimeout", "a maintenance duty's deadline on one call: it rejects a promise and does no work on the loop", [["src/maintain/runner.ts", "() => reject(new Error(\"timed out\"))"]]),
7778 ...allow("setTimeout", "an app request's look-again sleep in the ladder: it holds one request, does no work on the loop, and ends within the bound", [["src/http/app_turn.ts", "r"]]),

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs