Public: anyone can read this; the crew can change it.
Duplication report: near-duplicate functions and tests clustered into one shape #14691
Check · Review · Land · Live done — Landed and live in production.
mason/forge:mason/14620-duplication-report-near-duplicate-functi → main 4 commits fork 2 threads · 2 unresolved
54 files
changed +850
−163
AGENTS.md +2
| @@ -67,6 +67,8 @@ An app token's requests go behind people's and repeat reads are 304s (`src/http/ | ||
| 67 | 67 | `src/crew/beat.ts`: `lastBeat`, when each repository's beat last ran, and `declared`, the `beat` last read from each default branch with the commit it came from (#13709); at most one entry each per repository, gone on restart (a restart beats once at once; the declaration is a file in git). |
| 68 | 68 | `src/perf/job_alloc.ts`: `counted`, called by `during` around each job's synchronous slice, adds the JS heap's growth to the job's name (at most 128 names); the minute memory line reads and empties it (`memory allocated by jobs:`, #13924), so a timer that allocates with no request running is named. `bench/idle.ts` is the idle budget on a seeded forge (heap under 20 MiB a minute, no stall over 100 ms). Gone on restart. |
| 69 | 69 | |
| 70 | `bun scripts/dupes.ts` (#14620) reads every function and test body under `src/` and `test/` as a shape (`scripts/dupes_shape.ts`: names, strings and numbers taken out), clusters three or more that share `THRESHOLD` of their 4-token runs, and prints each cluster with the shape its members all hold and the forge's duplication rate; `--tune` scores `scripts/dupes-pairs.json`, the forge's own labeled pairs, and `test/dupes.test.ts` holds `THRESHOLD` to the best of them. `bun scripts/gate.ts` says, as a note and never a refusal, when a pull adds a function at or over it to an existing one (`dupesNote`); it blocks only once its false-positive rate is measured. A test that waits on something with no event to await calls `waitFor` or `pollUntil` in `test/helpers/poll.ts`, not its own `for (…) await Bun.sleep(…)` loop. | |
| 71 | ||
| 70 | 72 | A pull that changes code under `src/` adds or edits a `test/*.test.ts` too, or says `No test: <why>` in a commit line; `bun scripts/gate.ts` refuses it otherwise ("a behaviour a pull adds has a test that fails without it", #14235). |
| 71 | 73 | |
| 72 | 74 | A pull that adds a page (`src/web/pages/*.ts`), an MCP tool (`src/http/mcp/tools/*.ts`) or a settings module (`src/settings/*.ts`) has a commit line `Replaces: <what it takes the place of>` or `Replaces: nothing, <why>` (#14249, `src/kernel/replaces.ts`); `bun scripts/gate.ts` says it as a note, never a refusal, and the reviewer holds the pull to that line. |
scripts/dupes-pairs.json added +37
| @@ -0,0 +1,37 @@ | ||
| 1 | [ | |
| 2 | {"a":"test/apps-adversarial.test.ts#test: a forge_ token and a Basic auth header are redacted on the way in","b":"test/boundary-tokens-gates.test.ts#test: the log layer blanks a forge token wherever it is written: bare, as Bearer, as a Basic header, in a URL","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 3 | {"a":"test/render-exif.test.ts#test: the page escapes values and shows the table; no EXIF means no table","b":"test/render-exif.test.ts#test: the head of a large photo is enough","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 4 | {"a":"src/actions/repos.ts#deleteRepo","b":"src/actions/accounts.ts#removeOwnedRepos","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 5 | {"a":"test/livetriage.test.ts#test: a person who unfiles a filed issue is not overruled","b":"test/livetriage.test.ts#test: at most 25 issues are filed per tick; the rest are missed, then filed","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 6 | {"a":"src/web/page_text.ts#decode","b":"src/web/render/junit.ts#decode","same":true,"why":"two copies of an HTML entity decoder"}, | |
| 7 | {"a":"test/tui-steer.test.ts#open","b":"test/tui-issue-cost.test.ts#open","same":true,"why":"two tests open a headless terminal session the same way"}, | |
| 8 | {"a":"src/story/waves.ts#readWaves","b":"src/story/waves.ts#latestWave","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 9 | {"a":"test/apps-presence.test.ts#test: a forge beside one still serving waits for its apps; once the old one stops, it waits no more","b":"test/apps-presence.test.ts#test: a torn or foreign file in the directory says nobody is there","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 10 | {"a":"test/flow-outsiders.test.ts#test: an agent that is not this repository's crew is an outsider","b":"test/flow-review-latency.test.ts#test: an even count averages the middle two, to one decimal","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 11 | {"a":"test/next-steps-ask-status.test.ts#test: a builder on it: their name and that they are building it, naming the app and issue","b":"test/next-steps-ask-status.test.ts#test: no run yet: waiting for a builder, with no number","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 12 | {"a":"test/tui-transfer-repo.test.ts#test: typing a new name in the rename prompt renames the repository","b":"test/tui-hooks.test.ts#test: a short secret is refused in MCP's words and nothing is added","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 13 | {"a":"src/tui/screens/newapp.ts#newAppScreen","b":"src/tui/screens/newapp.ts#make","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 14 | {"a":"test/home-feed.test.ts#test: a release post reads the version and first pull, links the commit, and falls back without a version","b":"test/home-feed.test.ts#mcp","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 15 | {"a":"src/origins/inventory.ts#scanAll","b":"src/origins/inventory.ts#worker","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 16 | {"a":"src/web/studio/actions.ts#updateSubmit","b":"src/web/pages/crew.ts#tryAgainPost","same":true,"why":"two page handlers open with the same repo, sign-in and form preamble"}, | |
| 17 | {"a":"test/read-api.test.ts#test: contents reads a file's text at a ref","b":"test/read-api.test.ts#test: contents lists a directory","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 18 | {"a":"test/cluster-jobs.test.ts#test: a Job fails: the container's exit code is reported, the Job is deleted","b":"test/cluster-jobs.test.ts#test: an aborted call deletes the Job and throws","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 19 | {"a":"test/via-token.test.ts#mcp","b":"test/explore-unknown-words.test.ts#call","same":true,"why":"two helpers post one MCP tools/call"}, | |
| 20 | {"a":"test/view-source.test.ts#test: a failed fetch lands in the job and in fetch_error, scrubbed","b":"test/origins.test.ts#test: mirroring the same repo twice is refused","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 21 | {"a":"src/tui/screens/bring.ts#bringWords","b":"src/actions/revert_words.ts#takeBackWords","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 22 | {"a":"test/report-receipt.test.ts#test: the MCP report tool answers the same receipt, with no token","b":"test/report-words.test.ts#test: the MCP `report` tool answers the module's sentence","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 23 | {"a":"test/mcp-economics.test.ts#test: dryRun false sets it, and the read agrees","b":"test/mcp-economics.test.ts#test: an admin agent token is told not found, to read or to set","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 24 | {"a":"src/actions/board-pins.ts#pinPanel","b":"src/actions/board-pins.ts#movePinned","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 25 | {"a":"test/ssh-tell-first-ask.test.ts#exec","b":"test/ssh-shell-cold.test.ts#shell","same":true,"why":"two harnesses open the same ssh session"}, | |
| 26 | {"a":"src/tui/screens/cloud.ts#cloudScreen","b":"src/tui/screens/pinned.ts#pinnedScreen","same":true,"why":"two terminal screens set up the same list, note and read"}, | |
| 27 | {"a":"test/sentinel.test.ts#test: a week of nothing but stop is named","b":"test/sentinel.test.ts#test: a skip is a run saved, not a park: skips alone are never named","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 28 | {"a":"test/oauth-grants.test.ts#grant","b":"test/tokens-oauth-client.test.ts#exchange","same":true,"why":"two helpers create an OAuth client the same way"}, | |
| 29 | {"a":"test/tui-finder.test.ts#test: another ref lists that ref's files","b":"test/tui-finder.test.ts#test: typing, deleting and moving never read paths again","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 30 | {"a":"test/retention-deploys.test.ts#test: a deploy with no rows gets its list from git, and none is written","b":"test/retention-deploys.test.ts#test: a deploy an older release recorded reads its rows until retention trims them","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 31 | {"a":"src/core/app_releases.ts#listAppReleases","b":"src/core/app_releases.ts#listPublicReleases","same":false,"why":"similar in style, but one function would not serve both"}, | |
| 32 | {"a":"test/tui-profile.test.ts#boot","b":"test/tui-dotfiles.test.ts#boot","same":true,"why":"the same terminal boot helper, pasted"}, | |
| 33 | {"a":"test/releases-fit.test.ts#test: no inline width and no shipped rule fixes a width over 390 px","b":"test/changelog-fit.test.ts#test: no inline width and no shipped rule fixes a width over 390 px","same":true,"why":"the same fit test, pasted for another page"}, | |
| 34 | {"a":"test/web-browse.test.ts#asUser","b":"test/web-origins.test.ts#asUser","same":true,"why":"the same signed-in fetch helper"}, | |
| 35 | {"a":"test/ssh-tell-first-ask.test.ts#exec","b":"test/ssh-tell.test.ts#exec","same":true,"why":"the same ssh exec helper"}, | |
| 36 | {"a":"test/mcp-signup.test.ts#call","b":"test/mcp-backups.test.ts#call","same":true,"why":"the same MCP call helper"} | |
| 37 | ] |
scripts/dupes.ts added +271
| @@ -0,0 +1,271 @@ | ||
| 1 | /** | |
| 2 | * Near-duplicate functions and tests, clustered (#14620). | |
| 3 | * | |
| 4 | * The cluster-and-intersect step of Aroma (arXiv 1812.01158, §3.3.3): every function under `src/` | |
| 5 | * and `test/` is reduced to its shape (`scripts/dupes_shape.ts`), two functions whose shapes share | |
| 6 | * at least `THRESHOLD` of their 4-token runs are near-duplicates, a group of three or more is a | |
| 7 | * cluster, and the shape a cluster's members all hold is what one implementation would be. Offline: | |
| 8 | * nothing here is on the checks path except `addedFunctions` and `nearDuplicates`, which the gate | |
| 9 | * reads as a note. | |
| 10 | * | |
| 11 | * bun scripts/dupes.ts the report: rate, top clusters, each with its shared shape | |
| 12 | * bun scripts/dupes.ts --tune the threshold that best splits `scripts/dupes-pairs.json` | |
| 13 | * | |
| 14 | * The threshold is tuned on the forge's own labeled pairs, not the paper's 0.71. | |
| 15 | */ | |
| 16 | ||
| 17 | import { existsSync, readFileSync } from "node:fs"; | |
| 18 | import { functionsOf, type Fn } from "./dupes_shape.ts"; | |
| 19 | ||
| 20 | /** | |
| 21 | * Chosen by `--tune` over `scripts/dupes-pairs.json`; `test/dupes.test.ts` holds it to the best | |
| 22 | * one. | |
| 23 | */ | |
| 24 | export const THRESHOLD = 0.55; | |
| 25 | const GRAM = 4; | |
| 26 | /** The longest shared shape the report computes: the intersection costs the square of this. */ | |
| 27 | const SHARED_CAP = 300; | |
| 28 | /** A run of tokens held by more functions than this is boilerplate: it nominates no pair. */ | |
| 29 | const COMMON = 30; | |
| 30 | ||
| 31 | const grams = (shape: readonly string[]): Set<string> => { | |
| 32 | const out = new Set<string>(); | |
| 33 | for (let i = 0; i + GRAM <= shape.length; i++) out.add(shape.slice(i, i + GRAM).join(" ")); | |
| 34 | return out; | |
| 35 | }; | |
| 36 | ||
| 37 | /** Jaccard similarity of two shapes' 4-token runs. */ | |
| 38 | export function similarity(a: readonly string[], b: readonly string[]): number { | |
| 39 | const x = grams(a); | |
| 40 | const y = grams(b); | |
| 41 | let both = 0; | |
| 42 | for (const g of x) if (y.has(g)) both++; | |
| 43 | const all = x.size + y.size - both; | |
| 44 | return all === 0 ? 0 : both / all; | |
| 45 | } | |
| 46 | ||
| 47 | /** | |
| 48 | * Every function and test body under `roots` of the tree at `root`; past `budgetMs` it stops | |
| 49 | * reading and `out.partial` is true, so a caller on a clock still answers. | |
| 50 | */ | |
| 51 | export function readFunctions(root: string, roots: readonly string[] = ["src", "test"], min = 40, budgetMs = Infinity): Fn[] & { partial?: true } { | |
| 52 | const out: Fn[] & { partial?: true } = []; | |
| 53 | const stop = Date.now() + budgetMs; | |
| 54 | for (const r of roots.filter((x) => existsSync(`${root}/${x}`))) | |
| 55 | for (const f of new Bun.Glob("**/*.ts").scanSync({ cwd: `${root}/${r}` })) { | |
| 56 | if (Date.now() > stop) { out.partial = true; return out; } | |
| 57 | if (!f.endsWith(".d.ts")) out.push(...functionsOf(`${r}/${f}`, readFileSync(`${root}/${r}/${f}`, "utf8"), min)); | |
| 58 | } | |
| 59 | return out; | |
| 60 | } | |
| 61 | ||
| 62 | export type Edge = { a: number; b: number; score: number }; | |
| 63 | ||
| 64 | /** | |
| 65 | * Pairs scoring at least `threshold`, found through an index of 4-token runs rather than all pairs. | |
| 66 | */ | |
| 67 | export function similarPairs(fns: readonly Fn[], threshold: number): Edge[] { | |
| 68 | const sets = fns.map((f) => grams(f.shape)); | |
| 69 | const index = new Map<string, number[]>(); | |
| 70 | sets.forEach((set, i) => { for (const g of set) (index.get(g) ?? index.set(g, []).get(g)!).push(i); }); | |
| 71 | const edges: Edge[] = []; | |
| 72 | sets.forEach((set, a) => { | |
| 73 | const shared = new Map<number, number>(); | |
| 74 | for (const g of set) { | |
| 75 | const holders = index.get(g)!; | |
| 76 | if (holders.length > COMMON) continue; | |
| 77 | for (const b of holders) if (b > a) shared.set(b, (shared.get(b) ?? 0) + 1); | |
| 78 | } | |
| 79 | // The rare runs only nominate a partner; the score counts every run. | |
| 80 | for (const [b, n] of shared) { | |
| 81 | if (n < 3) continue; | |
| 82 | let both = 0; | |
| 83 | for (const g of set) if (sets[b]!.has(g)) both++; | |
| 84 | const score = both / (set.size + sets[b]!.size - both); | |
| 85 | if (score >= threshold) edges.push({ a, b, score }); | |
| 86 | } | |
| 87 | }); | |
| 88 | return edges; | |
| 89 | } | |
| 90 | ||
| 91 | export type Cluster = { members: Fn[]; mean: number; shared: string[] }; | |
| 92 | ||
| 93 | /** | |
| 94 | * The longest run of tokens in order that both hold (capped at 300 tokens each, the cost being the | |
| 95 | * product). | |
| 96 | */ | |
| 97 | function common(a: readonly string[], b: readonly string[]): string[] { | |
| 98 | const x = a.slice(0, 300); | |
| 99 | const y = b.slice(0, 300); | |
| 100 | const w = y.length + 1; | |
| 101 | const len = new Uint16Array((x.length + 1) * w); | |
| 102 | for (let i = x.length - 1; i >= 0; i--) | |
| 103 | for (let j = y.length - 1; j >= 0; j--) | |
| 104 | len[i * w + j] = x[i] === y[j] ? len[(i + 1) * w + j + 1]! + 1 : Math.max(len[(i + 1) * w + j]!, len[i * w + j + 1]!); | |
| 105 | const out: string[] = []; | |
| 106 | for (let i = 0, j = 0; i < x.length && j < y.length; ) { | |
| 107 | if (x[i] === y[j]) { out.push(x[i]!); i++; j++; } | |
| 108 | else if (len[(i + 1) * w + j]! >= len[i * w + j + 1]!) i++; | |
| 109 | else j++; | |
| 110 | } | |
| 111 | return out; | |
| 112 | } | |
| 113 | ||
| 114 | /** | |
| 115 | * Groups of `min` or more functions joined by pairs at `threshold`, each with the shape all its | |
| 116 | * members hold. | |
| 117 | */ | |
| 118 | export function clusters(fns: readonly Fn[], threshold = THRESHOLD, min = 3): Cluster[] { | |
| 119 | const edges = similarPairs(fns, threshold); | |
| 120 | const parent = fns.map((_, i) => i); | |
| 121 | const find = (i: number): number => (parent[i] === i ? i : (parent[i] = find(parent[i]!))); | |
| 122 | for (const e of edges) parent[find(e.a)] = find(e.b); | |
| 123 | const groups = new Map<number, number[]>(); | |
| 124 | fns.forEach((_, i) => { if (edges.length) (groups.get(find(i)) ?? groups.set(find(i), []).get(find(i))!).push(i); }); | |
| 125 | const inGroup = new Set(edges.flatMap((e) => [e.a, e.b])); | |
| 126 | const out: Cluster[] = []; | |
| 127 | for (const members of groups.values()) { | |
| 128 | if (members.length < min || !members.every((m) => inGroup.has(m))) continue; | |
| 129 | const scores = edges.filter((e) => members.includes(e.a)).map((e) => e.score); | |
| 130 | const shared = members.slice(1).reduce((s, m) => common(s, fns[m]!.shape), fns[members[0]!]!.shape.slice(0, SHARED_CAP)); | |
| 131 | out.push({ members: members.map((m) => fns[m]!), mean: scores.reduce((s, x) => s + x, 0) / scores.length, shared }); | |
| 132 | } | |
| 133 | return out.sort((p, q) => weight(q) - weight(p)); | |
| 134 | } | |
| 135 | ||
| 136 | /** Tokens one implementation would save: the shared shape, once for each member past the first. */ | |
| 137 | export const weight = (c: Cluster): number => c.shared.length * (c.members.length - 1); | |
| 138 | ||
| 139 | /** | |
| 140 | * What the forge's duplication is, as a share of every token in a function and as a count of | |
| 141 | * functions. | |
| 142 | */ | |
| 143 | export function rate(fns: readonly Fn[], found: readonly Cluster[]): { tokens: number; fns: number; saved: number } { | |
| 144 | const total = fns.reduce((s, f) => s + f.shape.length, 0); | |
| 145 | return { tokens: total, fns: found.reduce((s, c) => s + c.members.length, 0), saved: found.reduce((s, c) => s + weight(c), 0) }; | |
| 146 | } | |
| 147 | ||
| 148 | /** | |
| 149 | * Each existing function a new one is at least `threshold` like: for the gate's note. It runs on | |
| 150 | * every gate, so an existing function is first counted against the new ones' 4-token runs, as | |
| 151 | * numbers; only one that holds enough of a new one's runs is scored. | |
| 152 | */ | |
| 153 | export function nearDuplicates(added: readonly Fn[], existing: readonly Fn[], threshold = THRESHOLD): { added: Fn; like: Fn; score: number }[] { | |
| 154 | const ids = new Map<string, number>(); | |
| 155 | const id = (t: string): number => ids.get(t) ?? (ids.set(t, ids.size), ids.size - 1); | |
| 156 | const keysOf = (shape: readonly string[]): number[] => { | |
| 157 | const toks = shape.map(id); | |
| 158 | const out: number[] = []; | |
| 159 | for (let i = 0; i + GRAM <= toks.length; i++) out.push(((toks[i]! * 8192 + toks[i + 1]!) * 8192 + toks[i + 2]!) * 8192 + toks[i + 3]!); | |
| 160 | return out; | |
| 161 | }; | |
| 162 | const mine = added.map((a) => new Set(keysOf(a.shape))); | |
| 163 | const holders = new Map<number, number[]>(); | |
| 164 | mine.forEach((set, a) => { for (const k of set) (holders.get(k) ?? holders.set(k, []).get(k)!).push(a); }); | |
| 165 | // Jaccard cannot reach the threshold when one shape is much longer than the other. | |
| 166 | const shortest = Math.min(...added.map((a) => a.shape.length)); | |
| 167 | const longest = Math.max(...added.map((a) => a.shape.length)); | |
| 168 | const best: ({ like: Fn; score: number } | undefined)[] = added.map(() => undefined); | |
| 169 | for (const e of existing) { | |
| 170 | if (e.kind !== "function" || e.shape.length < shortest * threshold || e.shape.length * threshold > longest) continue; | |
| 171 | const hits = new Map<number, Set<number>>(); | |
| 172 | const keys = keysOf(e.shape); | |
| 173 | for (const k of keys) for (const a of holders.get(k) ?? []) (hits.get(a) ?? hits.set(a, new Set()).get(a)!).add(k); | |
| 174 | if (hits.size === 0) continue; | |
| 175 | const unique = new Set(keys); | |
| 176 | for (const [a, shared] of hits) { | |
| 177 | const x = added[a]!; | |
| 178 | if (e.id === x.id) continue; | |
| 179 | const total = mine[a]!.size + unique.size - shared.size; | |
| 180 | const score = total === 0 ? 0 : shared.size / total; | |
| 181 | if (score >= threshold && (!best[a] || score > best[a]!.score)) best[a] = { like: e, score }; | |
| 182 | } | |
| 183 | } | |
| 184 | const out: { added: Fn; like: Fn; score: number }[] = []; | |
| 185 | added.forEach((x, a) => { if (best[a]) out.push({ added: x, ...best[a]! }); }); | |
| 186 | return out; | |
| 187 | } | |
| 188 | ||
| 189 | /** The new line ranges of each file in a `git diff -U0`. */ | |
| 190 | export function addedRanges(diff: string): Map<string, [number, number][]> { | |
| 191 | const out = new Map<string, [number, number][]>(); | |
| 192 | let file = ""; | |
| 193 | for (const line of diff.split("\n")) { | |
| 194 | if (line.startsWith("+++ ")) file = line.startsWith("+++ b/") ? line.slice(6) : ""; | |
| 195 | const m = /^@@ -\S+ \+(\d+)(?:,(\d+))? @@/.exec(line); | |
| 196 | if (!m || !file) continue; | |
| 197 | const count = m[2] === undefined ? 1 : Number(m[2]); | |
| 198 | if (count > 0) (out.get(file) ?? out.set(file, []).get(file)!).push([Number(m[1]), Number(m[1]) + count - 1]); | |
| 199 | } | |
| 200 | return out; | |
| 201 | } | |
| 202 | ||
| 203 | /** | |
| 204 | * The functions (not test bodies: sibling tests read alike by design) that a diff starts, read from | |
| 205 | * the tree at `root`. | |
| 206 | */ | |
| 207 | export function addedFunctions(diff: string, root: string): Fn[] { | |
| 208 | const out: Fn[] = []; | |
| 209 | for (const [file, ranges] of addedRanges(diff)) { | |
| 210 | if (!file.endsWith(".ts") || file.endsWith(".d.ts")) continue; | |
| 211 | let text: string; | |
| 212 | try { text = readFileSync(`${root}/${file}`, "utf8"); } catch { continue; } | |
| 213 | for (const fn of functionsOf(file, text)) if (fn.kind === "function" && ranges.some(([a, b]) => fn.line >= a && fn.line <= b)) out.push(fn); | |
| 214 | } | |
| 215 | return out; | |
| 216 | } | |
| 217 | ||
| 218 | export type Labeled = { a: string; b: string; same: boolean; why: string }; | |
| 219 | ||
| 220 | /** | |
| 221 | * Precision, recall and F1 at each threshold over labeled pairs, whose ids are looked up in `fns`. | |
| 222 | */ | |
| 223 | export function tune(fns: readonly Fn[], pairs: readonly Labeled[]): { threshold: number; precision: number; recall: number; f1: number }[] { | |
| 224 | const byId = new Map(fns.map((f) => [f.id, f])); | |
| 225 | const scored = pairs.map((p) => { | |
| 226 | const a = byId.get(p.a); | |
| 227 | const b = byId.get(p.b); | |
| 228 | if (!a || !b) throw new Error(`dupes-pairs.json: ${!a ? p.a : p.b} is not a function under src/ or test/ any more`); | |
| 229 | return { same: p.same, score: similarity(a.shape, b.shape) }; | |
| 230 | }); | |
| 231 | const out = []; | |
| 232 | for (let t = 0.3; t <= 0.951; t += 0.05) { | |
| 233 | const hit = scored.filter((s) => s.score >= t); | |
| 234 | const tp = hit.filter((s) => s.same).length; | |
| 235 | const precision = hit.length ? tp / hit.length : 1; | |
| 236 | const recall = tp / Math.max(1, scored.filter((s) => s.same).length); | |
| 237 | out.push({ threshold: Math.round(t * 100) / 100, precision, recall, f1: precision + recall ? (2 * precision * recall) / (precision + recall) : 0 }); | |
| 238 | } | |
| 239 | return out; | |
| 240 | } | |
| 241 | ||
| 242 | function report(fns: readonly Fn[], found: readonly Cluster[], top: number): string { | |
| 243 | const r = rate(fns, found); | |
| 244 | const lines = [ | |
| 245 | `# Duplication report (threshold ${THRESHOLD}, clusters of 3+)`, | |
| 246 | "", | |
| 247 | `${fns.length} functions and tests of 40+ tokens, ${r.tokens} tokens. ${found.length} clusters hold ${r.fns} of them (${((100 * r.fns) / fns.length).toFixed(1)}%).`, | |
| 248 | `One implementation each would save ${r.saved} tokens: the forge's measured duplication rate is ${((100 * r.saved) / r.tokens).toFixed(1)}%.`, | |
| 249 | `A shared shape is cut at ${SHARED_CAP} tokens: for members longer than that the saving is a lower bound.`, | |
| 250 | "", | |
| 251 | ]; | |
| 252 | for (const c of found.slice(0, top)) { | |
| 253 | lines.push(`## ${c.members.length} like \`${c.members[0]!.id}\`: shared shape ${c.shared.length} tokens, mean ${c.mean.toFixed(2)}`); | |
| 254 | for (const m of c.members.slice(0, 12)) lines.push(`- ${m.file}:${m.line} ${m.id.split("#")[1]}`); | |
| 255 | if (c.members.length > 12) lines.push(`- ... and ${c.members.length - 12} more`); | |
| 256 | lines.push("", " " + c.shared.slice(0, 60).join(" ") + (c.shared.length > 60 ? " ..." : ""), ""); | |
| 257 | } | |
| 258 | return lines.join("\n"); | |
| 259 | } | |
| 260 | ||
| 261 | if (import.meta.main) { | |
| 262 | const fns = readFunctions("."); | |
| 263 | if (process.argv.includes("--tune")) { | |
| 264 | const pairs = JSON.parse(readFileSync("scripts/dupes-pairs.json", "utf8")) as Labeled[]; | |
| 265 | for (const row of tune(fns, pairs)) console.log(`${row.threshold} precision ${row.precision.toFixed(2)} recall ${row.recall.toFixed(2)} f1 ${row.f1.toFixed(2)}`); | |
| 266 | } else { | |
| 267 | const t = Date.now(); | |
| 268 | console.log(report(fns, clusters(fns), Number(process.argv[2]) || 25)); | |
| 269 | console.error(`dupes: ${Date.now() - t} ms`); | |
| 270 | } | |
| 271 | } |
scripts/dupes_shape.ts added +219
| @@ -0,0 +1,219 @@ | ||
| 1 | /** | |
| 2 | * The shape of a function (#14620): its tokens with the names taken out. | |
| 3 | * | |
| 4 | * Aroma (arXiv 1812.01158, §3.3) compares code by its keywords and punctuation, so two functions | |
| 5 | * that do the same thing under other names look alike. This reads TypeScript into tokens, finds | |
| 6 | * each function, arrow function and test body, and keeps what makes its shape: keywords, | |
| 7 | * punctuation and member names (`.push`, `.get`), while a local name becomes `$`, a number `N`, a | |
| 8 | * string `S` and a template `T`. Nothing is parsed; a regular expression literal and a brace in a | |
| 9 | * string are the only things the scanner has to know about. | |
| 10 | */ | |
| 11 | ||
| 12 | export type Fn = { | |
| 13 | /** `path#name`, or `path#test: title` for a test. */ | |
| 14 | id: string; | |
| 15 | file: string; | |
| 16 | line: number; | |
| 17 | kind: "function" | "test"; | |
| 18 | /** The normalized tokens of the parameters and the body. */ | |
| 19 | shape: string[]; | |
| 20 | }; | |
| 21 | ||
| 22 | const KEYWORDS = new Set( | |
| 23 | ( | |
| 24 | "async await break case catch class const continue default delete do else enum export extends " + | |
| 25 | "finally for function if import in instanceof let new of return switch this throw try typeof " + | |
| 26 | "void while yield null undefined true false as type interface readonly" | |
| 27 | ).split(" "), | |
| 28 | ); | |
| 29 | ||
| 30 | /** After one of these a `/` begins a regular expression, not a division. */ | |
| 31 | const TWO = new Set(["=>", "&&", "||", "??", "?.", "==", "!=", "<=", ">=", "..."]); | |
| 32 | ||
| 33 | const BEFORE_REGEX = new Set(["(", ",", "=", ":", "[", "!", "&", "|", "?", "{", "}", ";", "return", "=>", "&&", "||"]); | |
| 34 | ||
| 35 | type Token = { t: string; w: string; line: number }; | |
| 36 | ||
| 37 | function endOfQuoted(s: string, i: number, q: string): number { | |
| 38 | for (i++; i < s.length && s[i] !== q; i++) if (s[i] === "\\") i++; | |
| 39 | return i; | |
| 40 | } | |
| 41 | ||
| 42 | /** The index of the `}` that closes the `${` opened before `i`, in a template literal. */ | |
| 43 | function endOfTemplate(s: string, i: number): number { | |
| 44 | for (i++; i < s.length && s[i] !== "`"; i++) { | |
| 45 | if (s[i] === "\\") i++; | |
| 46 | else if (s[i] === "$" && s[i + 1] === "{") { | |
| 47 | let depth = 1; | |
| 48 | for (i += 2; i < s.length && depth > 0; i++) { | |
| 49 | if (s[i] === "{") depth++; | |
| 50 | else if (s[i] === "}") depth--; | |
| 51 | else if (s[i] === "`") i = endOfTemplate(s, i); | |
| 52 | else if (s[i] === '"' || s[i] === "'") i = endOfQuoted(s, i, s[i]!); | |
| 53 | } | |
| 54 | i--; | |
| 55 | } | |
| 56 | } | |
| 57 | return i; | |
| 58 | } | |
| 59 | ||
| 60 | // Character classes by code, not by regular expression: the gate tokenizes the whole tree. | |
| 61 | const isDigit = (c: number): boolean => c >= 48 && c <= 57; | |
| 62 | const isWord = (c: number): boolean => (c >= 97 && c <= 122) || (c >= 65 && c <= 90) || isDigit(c) || c === 95 || c === 36; | |
| 63 | const isStart = (c: number): boolean => isWord(c) && !isDigit(c); | |
| 64 | ||
| 65 | /** Tokens of one source text, names normalized as the header says. */ | |
| 66 | export function tokenize(s: string): Token[] { | |
| 67 | const out: Token[] = []; | |
| 68 | let line = 1; | |
| 69 | let i = 0; | |
| 70 | const push = (t: string, w = t) => out.push({ t, w, line }); | |
| 71 | while (i < s.length) { | |
| 72 | const code = s.charCodeAt(i); | |
| 73 | if (code === 10) { line++; i++; continue; } | |
| 74 | if (code <= 32) { i++; continue; } | |
| 75 | const c = s[i]!; | |
| 76 | if (code > 127 && /\s/.test(c)) { i++; continue; } | |
| 77 | if (c === "/" && s[i + 1] === "/") { const e = s.indexOf("\n", i); i = e < 0 ? s.length : e; continue; } | |
| 78 | if (c === "/" && s[i + 1] === "*") { | |
| 79 | const end = s.indexOf("*/", i + 2); | |
| 80 | const stop = end < 0 ? s.length : end + 2; | |
| 81 | for (; i < stop; i++) if (s[i] === "\n") line++; | |
| 82 | continue; | |
| 83 | } | |
| 84 | if (c === '"' || c === "'") { const end = endOfQuoted(s, i, c); push("S", s.slice(i + 1, end)); i = end + 1; continue; } | |
| 85 | if (c === "`") { | |
| 86 | const end = endOfTemplate(s, i); | |
| 87 | for (; i <= end; i++) if (s[i] === "\n") line++; | |
| 88 | push("T"); | |
| 89 | continue; | |
| 90 | } | |
| 91 | if (c === "/" && BEFORE_REGEX.has(out[out.length - 1]?.t ?? ";")) { | |
| 92 | let j = i + 1; | |
| 93 | for (let inClass = false; j < s.length && s[j] !== "\n" && (inClass || s[j] !== "/"); j++) { | |
| 94 | if (s[j] === "\\") j++; | |
| 95 | else if (s[j] === "[") inClass = true; | |
| 96 | else if (s[j] === "]") inClass = false; | |
| 97 | } | |
| 98 | i = j + 1; | |
| 99 | while (/[a-z]/.test(s[i] ?? "")) i++; | |
| 100 | push("R"); | |
| 101 | continue; | |
| 102 | } | |
| 103 | if (isDigit(s.charCodeAt(i))) { while (i < s.length && (isWord(s.charCodeAt(i)) || s[i] === ".")) i++; push("N"); continue; } | |
| 104 | if (isStart(s.charCodeAt(i))) { | |
| 105 | let j = i + 1; | |
| 106 | while (j < s.length && isWord(s.charCodeAt(j))) j++; | |
| 107 | const word = s.slice(i, j); | |
| 108 | i = j; | |
| 109 | const afterDot = out[out.length - 1]?.t === "."; | |
| 110 | push(KEYWORDS.has(word) && !afterDot ? word : afterDot ? `.${word}` : "$", word); | |
| 111 | continue; | |
| 112 | } | |
| 113 | const two = s.slice(i, i + 2); | |
| 114 | if (TWO.has(two)) { | |
| 115 | const three = s.slice(i, i + 3); | |
| 116 | if (three === "===" || three === "!==") { i += 3; push(three); continue; } | |
| 117 | if (two !== "!=" && two !== "==") { i += 2; push(two); continue; } | |
| 118 | i += 2; push(two); continue; | |
| 119 | } | |
| 120 | i++; | |
| 121 | push(c); | |
| 122 | } | |
| 123 | return out; | |
| 124 | } | |
| 125 | ||
| 126 | /** The index of the token closing the bracket opened at `at`, or -1. */ | |
| 127 | function closing(toks: Token[], at: number): number { | |
| 128 | const open = toks[at]!.t; | |
| 129 | const close = open === "(" ? ")" : open === "{" ? "}" : "]"; | |
| 130 | let depth = 0; | |
| 131 | for (let i = at; i < toks.length; i++) { | |
| 132 | if (toks[i]!.t === open) depth++; | |
| 133 | else if (toks[i]!.t === close && --depth === 0) return i; | |
| 134 | } | |
| 135 | return -1; | |
| 136 | } | |
| 137 | ||
| 138 | /** | |
| 139 | * From the `)` of a parameter list, the `{` that opens the body: past a return type, never past a | |
| 140 | * `;`. | |
| 141 | */ | |
| 142 | function bodyOpen(toks: Token[], from: number, arrow: boolean): number { | |
| 143 | let depth = 0; | |
| 144 | for (let i = from; i < toks.length; i++) { | |
| 145 | const t = toks[i]!.t; | |
| 146 | if (t === ";" && depth === 0) return -1; | |
| 147 | if (arrow && t === "=>" && depth === 0) return toks[i + 1]?.t === "{" ? i + 1 : -1; | |
| 148 | if (t === "(" || t === "[" || t === "<") depth++; | |
| 149 | else if (t === ")" || t === "]" || t === ">") depth--; | |
| 150 | else if (t === "{" && !arrow && depth === 0 && !["|", "&", ":", "<", ","].includes(toks[i - 1]!.t)) return i; | |
| 151 | else if (t === "{") { const e = closing(toks, i); if (e < 0) return -1; i = e; } | |
| 152 | } | |
| 153 | return -1; | |
| 154 | } | |
| 155 | ||
| 156 | /** From the `)` at `params`, back to the `(` that opens the list. */ | |
| 157 | function openOf(toks: Token[], params: number): number { | |
| 158 | let depth = 0; | |
| 159 | for (let i = params; i >= 0; i--) { | |
| 160 | if (toks[i]!.t === ")") depth++; | |
| 161 | else if (toks[i]!.t === "(" && --depth === 0) return i; | |
| 162 | } | |
| 163 | return -1; | |
| 164 | } | |
| 165 | ||
| 166 | /** After the `<` at `at`, the index past its `>`. */ | |
| 167 | function pastAngle(toks: Token[], at: number): number { | |
| 168 | let depth = 0; | |
| 169 | for (let i = at; i < toks.length; i++) { | |
| 170 | if (toks[i]!.t === "<") depth++; | |
| 171 | else if (toks[i]!.t === ">" && --depth === 0) return i + 1; | |
| 172 | } | |
| 173 | return -1; | |
| 174 | } | |
| 175 | ||
| 176 | /** | |
| 177 | * Functions, arrow functions with a block body bound to a name, and `test`/`it` bodies, each at | |
| 178 | * least `min` tokens long. An arrow passed as an argument is left to the function that holds it. | |
| 179 | */ | |
| 180 | export function functionsOf(file: string, text: string, min = 40): Fn[] { | |
| 181 | const toks = tokenize(text); | |
| 182 | const found: Fn[] = []; | |
| 183 | const take = (name: string, kind: Fn["kind"], start: number, open: number) => { | |
| 184 | const end = open < 0 ? -1 : closing(toks, open); | |
| 185 | if (end < 0) return; | |
| 186 | const shape = toks.slice(start, end + 1).map((x) => x.t); | |
| 187 | if (shape.length >= min) found.push({ id: `${file}#${name}`, file, line: toks[start]!.line, kind, shape }); | |
| 188 | }; | |
| 189 | for (let i = 0; i < toks.length; i++) { | |
| 190 | const t = toks[i]!.t; | |
| 191 | if (t === "function") { | |
| 192 | let p = i + 1; | |
| 193 | if (toks[p]?.t === "*") p++; | |
| 194 | const name = toks[p]?.t === "$" ? toks[p++]!.w : "(anonymous)"; | |
| 195 | if (toks[p]?.t === "<") p = pastAngle(toks, p); | |
| 196 | if (toks[p]?.t !== "(") continue; | |
| 197 | const params = closing(toks, p); | |
| 198 | if (params > 0) take(name, "function", p, bodyOpen(toks, params + 1, false)); | |
| 199 | } else if (t === "=>" && toks[i + 1]?.t === "{") { | |
| 200 | // `const name = (a) => {` or `const name = async (a: T): R => {`. | |
| 201 | let params = -1; | |
| 202 | for (let d = 1; d < 40 && i - d > 0 && params < 0; d++) { | |
| 203 | const x = toks[i - d]!.t; | |
| 204 | if (x === ")") params = i - d; | |
| 205 | else if (x === ";" || x === "=>" || x === "{" || x === "}") break; | |
| 206 | } | |
| 207 | if (params < 0) continue; | |
| 208 | const open = openOf(toks, params); | |
| 209 | const eq = toks[open - 1]?.t === "async" ? open - 2 : open - 1; | |
| 210 | if (open < 0 || toks[eq]?.t !== "=" || toks[eq - 1]?.t !== "$") continue; | |
| 211 | take(toks[eq - 1]!.w, "function", open, i + 1); | |
| 212 | } else if (t === "$" && (toks[i]!.w === "test" || toks[i]!.w === "it") && toks[i + 1]?.t === "(" && toks[i + 2]?.t === "S") { | |
| 213 | const end = closing(toks, i + 1); | |
| 214 | const shape = end < 0 ? [] : toks.slice(i + 2, end).map((x) => x.t); | |
| 215 | if (shape.length >= min) found.push({ id: `${file}#test: ${toks[i + 2]!.w}`, file, line: toks[i]!.line, kind: "test", shape }); | |
| 216 | } | |
| 217 | } | |
| 218 | return found; | |
| 219 | } |
scripts/gate.ts +44 −40
| @@ -401,6 +401,14 @@ function git(args: string[], cwd: string = process.cwd()): { ok: boolean; out: s | ||
| 401 | 401 | return { ok: r.exitCode === 0, out: r.stdout.toString().trim() }; |
| 402 | 402 | } |
| 403 | 403 | |
| 404 | /** | |
| 405 | * What a pull is judged against: `$FORGE_CHECK_BASE`, else the merge base with main; empty when | |
| 406 | * neither resolves. | |
| 407 | */ | |
| 408 | export function baseRef(cwd: string = process.cwd()): string { | |
| 409 | return process.env.FORGE_CHECK_BASE || git(["merge-base", "origin/main", "HEAD"], cwd).out || git(["merge-base", "main", "HEAD"], cwd).out; | |
| 410 | } | |
| 411 | ||
| 404 | 412 | /** |
| 405 | 413 | * Pins the pull edited, against its base: `$FORGE_CHECK_BASE`, else the |
| 406 | 414 | * merge base with main. A pin whose migration is not on the base was never |
| @@ -570,8 +578,7 @@ function wrapSources(root: string, all: boolean, working = false): { path: strin | ||
| 570 | 578 | .filter((dir) => existsSync(join(root, dir))) |
| 571 | 579 | .flatMap((dir) => (readdirSync(join(root, dir), { recursive: true }) as string[]).map((f) => `${dir}/${f}`)); |
| 572 | 580 | } else { |
| 573 | const base = | |
| 574 | process.env.FORGE_CHECK_BASE || git(["merge-base", "origin/main", "HEAD"], root).out || git(["merge-base", "main", "HEAD"], root).out; | |
| 581 | const base = baseRef(root); | |
| 575 | 582 | if (!base) return []; |
| 576 | 583 | const changed = git(["diff", "--name-only", "--diff-filter=d", ...(working ? [base] : [`${base}..${judgedHead(root)}`]), "--", ...dirs], root); |
| 577 | 584 | const fresh = working ? git(["ls-files", "--others", "--exclude-standard", "--", ...dirs], root) : { ok: false, out: "" }; |
| @@ -600,7 +607,7 @@ export function wrapFix(root = ".", all = false): string[] { | ||
| 600 | 607 | const { rewrap } = require("../src/kernel/comment_wrap.ts") as typeof import("../src/kernel/comment_wrap.ts"); |
| 601 | 608 | const { parseAdded } = require("../src/kernel/review_rules.ts") as typeof import("../src/kernel/review_rules.ts"); |
| 602 | 609 | const { parseHunks, keepHunks } = require("../src/kernel/rewrap_noise.ts") as typeof import("../src/kernel/rewrap_noise.ts"); |
| 603 | const base = all ? "" : process.env.FORGE_CHECK_BASE || git(["merge-base", "origin/main", "HEAD"], root).out || git(["merge-base", "main", "HEAD"], root).out; | |
| 610 | const base = all ? "" : baseRef(root); | |
| 604 | 611 | // The lines the pull wrote, by file (#13913): only comments touching one are rewrapped. |
| 605 | 612 | const written = new Map<string, Set<number>>(); |
| 606 | 613 | if (!all) { |
| @@ -641,10 +648,7 @@ export function wrapFix(root = ".", all = false): string[] { | ||
| 641 | 648 | */ |
| 642 | 649 | export async function rewrapNoiseRefusal(cwd: string = process.cwd()): Promise<string[]> { |
| 643 | 650 | const { parseHunks, rewrapNoiseProblems } = await import("../src/kernel/rewrap_noise.ts"); |
| 644 | const base = | |
| 645 | process.env.FORGE_CHECK_BASE || | |
| 646 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 647 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 651 | const base = baseRef(cwd); | |
| 648 | 652 | if (!base) return []; |
| 649 | 653 | const diff = git(["diff", "-U0", "--no-color", "--no-ext-diff", `${base}..${judgedHead(cwd)}`, "--", "src", "test", "scripts"], cwd); |
| 650 | 654 | return diff.ok ? rewrapNoiseProblems(parseHunks(diff.out)) : []; |
| @@ -656,10 +660,7 @@ export async function rewrapNoiseRefusal(cwd: string = process.cwd()): Promise<s | ||
| 656 | 660 | */ |
| 657 | 661 | export async function addedRefusal(cwd: string = process.cwd()): Promise<string[]> { |
| 658 | 662 | const { addedProblems, parseAdded } = await import("../src/kernel/review_rules.ts"); |
| 659 | const base = | |
| 660 | process.env.FORGE_CHECK_BASE || | |
| 661 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 662 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 663 | const base = baseRef(cwd); | |
| 663 | 664 | if (!base) return []; |
| 664 | 665 | const diff = git(["diff", "-U0", "--no-color", "--no-ext-diff", `${base}..${judgedHead(cwd)}`, "--", "src", "test"], cwd); |
| 665 | 666 | return diff.ok ? addedProblems(parseAdded(diff.out)) : []; |
| @@ -673,10 +674,7 @@ export async function addedRefusal(cwd: string = process.cwd()): Promise<string[ | ||
| 673 | 674 | export async function tryItNote(cwd: string = process.cwd()): Promise<string[]> { |
| 674 | 675 | const { parseNameStatus } = await import("../src/kernel/pin_edits.ts"); |
| 675 | 676 | const { tryItProblems } = await import("../src/kernel/try_it.ts"); |
| 676 | const base = | |
| 677 | process.env.FORGE_CHECK_BASE || | |
| 678 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 679 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 677 | const base = baseRef(cwd); | |
| 680 | 678 | if (!base) return []; |
| 681 | 679 | const diff = git(["diff", "--name-status", "-M", `${base}..${judgedHead(cwd)}`], cwd); |
| 682 | 680 | const log = git(["log", "--format=%B%x00", `${base}..${judgedHead(cwd)}`], cwd); |
| @@ -691,10 +689,7 @@ export async function tryItNote(cwd: string = process.cwd()): Promise<string[]> | ||
| 691 | 689 | export async function replacesNote(cwd: string = process.cwd()): Promise<string[]> { |
| 692 | 690 | const { parseNameStatus } = await import("../src/kernel/pin_edits.ts"); |
| 693 | 691 | const { replacesProblems } = await import("../src/kernel/replaces.ts"); |
| 694 | const base = | |
| 695 | process.env.FORGE_CHECK_BASE || | |
| 696 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 697 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 692 | const base = baseRef(cwd); | |
| 698 | 693 | if (!base) return []; |
| 699 | 694 | const diff = git(["diff", "--name-status", "-M", `${base}..${judgedHead(cwd)}`], cwd); |
| 700 | 695 | const log = git(["log", "--format=%B%x00", `${base}..${judgedHead(cwd)}`], cwd); |
| @@ -702,14 +697,35 @@ export async function replacesNote(cwd: string = process.cwd()): Promise<string[ | ||
| 702 | 697 | return replacesProblems(parseNameStatus(diff.out), log.out.split("\0").map((m) => m.trim()).filter(Boolean)); |
| 703 | 698 | } |
| 704 | 699 | |
| 700 | /** The most time the duplicate note spends reading the tree. */ | |
| 701 | const DUPES_BUDGET_MS = 8000; | |
| 702 | ||
| 703 | /** | |
| 704 | * A function a pull adds that is close to one already there (#14620): a note naming the existing | |
| 705 | * one, never a refusal, until its false-positive rate is measured (`scripts/dupes.ts`). | |
| 706 | */ | |
| 707 | export async function dupesNote(cwd: string = process.cwd()): Promise<string[]> { | |
| 708 | const base = baseRef(cwd); | |
| 709 | if (!base) return []; | |
outdated
This reads every function in src and test on each gate run. Measure it and state the ms in the pull. It sits on the critical path of every pull. |
||
| 710 | const diff = git(["diff", "-U0", "--no-color", "--no-ext-diff", `${base}..${judgedHead(cwd)}`, "--", "src", "test"], cwd); | |
| 711 | if (!diff.ok) return []; | |
| 712 | const { addedFunctions, nearDuplicates, readFunctions } = await import("./dupes.ts"); | |
| 713 | const added = addedFunctions(diff.out, cwd); | |
| 714 | if (added.length === 0) return []; | |
| 715 | // About 4 s over the whole tree (42k functions, 50 MB; #14620), bounded at 8 s of reading. | |
| 716 | const existing = readFunctions(cwd, ["src", "test"], 40, DUPES_BUDGET_MS); | |
| 717 | const notes = nearDuplicates(added, existing).map( | |
| 718 | (n) => `${n.added.id} (line ${n.added.line}) is ${n.score.toFixed(2)} like ${n.like.id} (line ${n.like.line}): one implementation?`, | |
| 719 | ); | |
| 720 | if (existing.partial) notes.push(`the duplicate scan stopped after ${DUPES_BUDGET_MS} ms: it compared against part of the tree`); | |
| 721 | return notes; | |
| 722 | } | |
| 723 | ||
| 705 | 724 | /** A pull that adds a page adds its terminal screen and how-to too (#10628). */ |
| 706 | 725 | export async function bothDoorsRefusal(cwd: string = process.cwd()): Promise<string[]> { |
| 707 | 726 | const { parseNameStatus } = await import("../src/kernel/pin_edits.ts"); |
| 708 | 727 | const { bothDoorsProblems } = await import("../src/kernel/both_doors.ts"); |
| 709 | const base = | |
| 710 | process.env.FORGE_CHECK_BASE || | |
| 711 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 712 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 728 | const base = baseRef(cwd); | |
| 713 | 729 | if (!base) return []; |
| 714 | 730 | const diff = git(["diff", "--name-status", "-M", `${base}..${judgedHead(cwd)}`], cwd); |
| 715 | 731 | const log = git(["log", "--format=%B%x00", `${base}..${judgedHead(cwd)}`], cwd); |
| @@ -726,10 +742,7 @@ export async function bothDoorsRefusal(cwd: string = process.cwd()): Promise<str | ||
| 726 | 742 | export async function testsWithChangeRefusal(cwd: string = process.cwd()): Promise<string[]> { |
| 727 | 743 | const { parseNameStatus } = await import("../src/kernel/pin_edits.ts"); |
| 728 | 744 | const { testsWithChangeProblems } = await import("../src/kernel/tests_with_change.ts"); |
| 729 | const base = | |
| 730 | process.env.FORGE_CHECK_BASE || | |
| 731 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 732 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 745 | const base = baseRef(cwd); | |
| 733 | 746 | if (!base) return []; |
| 734 | 747 | const diff = git(["diff", "--name-status", "-M", `${base}..${judgedHead(cwd)}`], cwd); |
| 735 | 748 | const log = git(["log", "--format=%B%x00", `${base}..${judgedHead(cwd)}`], cwd); |
| @@ -746,10 +759,7 @@ export async function lookNote(cwd: string = process.cwd()): Promise<string[]> { | ||
| 746 | 759 | const { lookNotes } = await import("../src/kernel/look_rule.ts"); |
| 747 | 760 | const { pagesShowing } = await import("../src/kernel/look_pages.ts"); |
| 748 | 761 | const { readTui } = await import("../src/kernel/tui_entry.ts"); |
| 749 | const base = | |
| 750 | process.env.FORGE_CHECK_BASE || | |
| 751 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 752 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 762 | const base = baseRef(cwd); | |
| 753 | 763 | if (!base) return []; |
| 754 | 764 | const diff = git(["diff", "--name-status", "-M", `${base}..${judgedHead(cwd)}`], cwd); |
| 755 | 765 | const log = git(["log", "--format=%B%x00", `${base}..${judgedHead(cwd)}`], cwd); |
| @@ -815,10 +825,7 @@ export async function directionRefusal(cwd: string = process.cwd(), run: typeof | ||
| 815 | 825 | */ |
| 816 | 826 | export async function dependencyRefusal(cwd: string = process.cwd()): Promise<string[]> { |
| 817 | 827 | const { dependencyProblems, DEPENDENCIES_DOC } = await import("../src/kernel/dependencies.ts"); |
| 818 | const base = | |
| 819 | process.env.FORGE_CHECK_BASE || | |
| 820 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 821 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 828 | const base = baseRef(cwd); | |
| 822 | 829 | if (!base) return []; |
| 823 | 830 | const head = judgedHead(cwd); |
| 824 | 831 | const read = (path: string) => { |
| @@ -833,10 +840,7 @@ export async function dependencyRefusal(cwd: string = process.cwd()): Promise<st | ||
| 833 | 840 | /** A pull that grows README.md puts the feature's how-to in docs/features instead (#12392). */ |
| 834 | 841 | export async function readmeCapRefusal(cwd: string = process.cwd()): Promise<string[]> { |
| 835 | 842 | const { readmeCapProblems } = await import("../src/kernel/readme_cap.ts"); |
| 836 | const base = | |
| 837 | process.env.FORGE_CHECK_BASE || | |
| 838 | git(["merge-base", "origin/main", "HEAD"], cwd).out || | |
| 839 | git(["merge-base", "main", "HEAD"], cwd).out; | |
| 843 | const base = baseRef(cwd); | |
| 840 | 844 | if (!base) return []; |
| 841 | 845 | const before = git(["show", `${base}:README.md`], cwd); |
| 842 | 846 | if (!before.ok) return []; |
| @@ -868,7 +872,7 @@ export async function gateVerdict(cwd: string = process.cwd()): Promise<{ notes: | ||
| 868 | 872 | ...billingDeletesRefusal(cwd), |
| 869 | 873 | ...testHonestyRefusal(cwd), |
| 870 | 874 | ]; |
| 871 | return { notes: [...wrapNote(cwd), ...(await lookNote(cwd)), ...(await tryItNote(cwd)), ...(await replacesNote(cwd))], refusal }; | |
| 875 | return { notes: [...wrapNote(cwd), ...(await lookNote(cwd)), ...(await tryItNote(cwd)), ...(await replacesNote(cwd)), ...(await dupesNote(cwd))], refusal }; | |
| 872 | 876 | } |
| 873 | 877 | |
| 874 | 878 | // `bun scripts/gate.ts`: the audit alone, first in `forge.json`'s check, so |
scripts/test-honesty-allow.json +22 −47
| @@ -1,6 +1,5 @@ | ||
| 1 | 1 | { |
| 2 | 2 | "test/account-budget.test.ts": {"wait":1}, |
| 3 | "test/account-ceiling-reserve.test.ts": {"wait":1}, | |
| 4 | 3 | "test/accounts.test.ts": {"wait":2}, |
| 5 | 4 | "test/act-now.test.ts": {"wait":2}, |
| 6 | 5 | "test/actions-newapp-template.test.ts": {"timeout":1}, |
| @@ -17,7 +16,6 @@ | ||
| 17 | 16 | "test/app-limit.test.ts": {"timeout":1}, |
| 18 | 17 | "test/app-pull-now.test.ts": {"wait":2}, |
| 19 | 18 | "test/app-template.test.ts": {"timeout":1}, |
| 20 | "test/app-tty-sessions.test.ts": {"wait":1}, | |
| 21 | 19 | "test/app-variants.test.ts": {"wait":2}, |
| 22 | 20 | "test/app_template_folders.test.ts": {"timeout":1}, |
| 23 | 21 | "test/app_templates.test.ts": {"timeout":1}, |
| @@ -25,7 +23,6 @@ | ||
| 25 | 23 | "test/apps-adversarial-grants.test.ts": {"wait":1}, |
| 26 | 24 | "test/apps-adversarial-teardown.test.ts": {"wait":3}, |
| 27 | 25 | "test/apps-adversarial.test.ts": {"wait":2}, |
| 28 | "test/apps-archive-race.test.ts": {"wait":3}, | |
| 29 | 26 | "test/apps-build-preview.test.ts": {"wait":1}, |
| 30 | 27 | "test/apps-busy.test.ts": {"wait":2}, |
| 31 | 28 | "test/apps-handoff.test.ts": {"wait":1}, |
| @@ -62,7 +59,7 @@ | ||
| 62 | 59 | "test/big-clone.test.ts": {"wait":3}, |
| 63 | 60 | "test/billing-replay.test.ts": {"wait":2}, |
| 64 | 61 | "test/boards-healthz-gap.test.ts": {"wait":1}, |
| 65 | "test/boot-beside-peer.test.ts": {"wait":3}, | |
| 62 | "test/boot-beside-peer.test.ts": {"wait":2}, | |
| 66 | 63 | "test/boot-no-stall.test.ts": {"wait":1}, |
| 67 | 64 | "test/boot-production.test.ts": {"wait":5}, |
| 68 | 65 | "test/boundary-fork-objects.test.ts": {"wait":2}, |
| @@ -108,22 +105,19 @@ | ||
| 108 | 105 | "test/checks-one-core.test.ts": {"wait":4}, |
| 109 | 106 | "test/checks-placement.test.ts": {"wait":1}, |
| 110 | 107 | "test/checks.test.ts": {"wait":1}, |
| 111 | "test/child-credit.test.ts": {"wait":4}, | |
| 112 | "test/client-abort.test.ts": {"wait":5}, | |
| 108 | "test/child-credit.test.ts": {"wait":2}, | |
| 113 | 109 | "test/client-list.test.ts": {"wait":1}, |
| 114 | 110 | "test/codes-tui.test.ts": {"wait":1}, |
| 115 | 111 | "test/combined-check.test.ts": {"wait":2}, |
| 116 | 112 | "test/combined-fork.test.ts": {"wait":1}, |
| 117 | 113 | "test/composer-draft.test.ts": {"wait":2}, |
| 118 | "test/conflicts-tui.test.ts": {"wait":3}, | |
| 114 | "test/conflicts-tui.test.ts": {"wait":2}, | |
| 119 | 115 | "test/contain.test.ts": {"wait":8}, |
| 120 | 116 | "test/contribute-check-text.test.ts": {"wait":2}, |
| 121 | 117 | "test/conversation-confirm.test.ts": {"wait":1}, |
| 122 | 118 | "test/conversation-files.test.ts": {"wait":2}, |
| 123 | "test/conversation-queue-api.test.ts": {"wait":1}, | |
| 124 | "test/conversation-queue.test.ts": {"wait":1}, | |
| 125 | "test/conversation-talk.test.ts": {"wait":3}, | |
| 126 | "test/conversation-turn.test.ts": {"wait":7}, | |
| 119 | "test/conversation-talk.test.ts": {"wait":2}, | |
| 120 | "test/conversation-turn.test.ts": {"wait":5}, | |
| 127 | 121 | "test/conversations-etag.test.ts": {"wait":1}, |
| 128 | 122 | "test/conversations.test.ts": {"wait":1}, |
| 129 | 123 | "test/cookie-flags.test.ts": {"timeout":1}, |
| @@ -142,9 +136,9 @@ | ||
| 142 | 136 | "test/crew-lent-refresh.test.ts": {"wait":1}, |
| 143 | 137 | "test/crew-loop.test.ts": {"wait":4}, |
| 144 | 138 | "test/crew-merge-conflict.test.ts": {"wait":1}, |
| 145 | "test/crew-mirror.test.ts": {"wait":5}, | |
| 139 | "test/crew-mirror.test.ts": {"wait":3}, | |
| 146 | 140 | "test/crew-network.test.ts": {"wait":3}, |
| 147 | "test/crew-own-forge.test.ts": {"wait":3}, | |
| 141 | "test/crew-own-forge.test.ts": {"wait":1}, | |
| 148 | 142 | "test/crew-parks-done.test.ts": {"wait":3}, |
| 149 | 143 | "test/crew-parks-planner.test.ts": {"wait":5}, |
| 150 | 144 | "test/crew-proposal.test.ts": {"wait":4}, |
| @@ -159,9 +153,7 @@ | ||
| 159 | 153 | "test/crew-wrapup.test.ts": {"wait":1}, |
| 160 | 154 | "test/desk-api.test.ts": {"wait":4}, |
| 161 | 155 | "test/disk-floor.test.ts": {"wait":1}, |
| 162 | "test/door-pulls.test.ts": {"wait":1}, | |
| 163 | "test/drill-board-line.test.ts": {"wait":2}, | |
| 164 | "test/drill-budget-size.test.ts": {"wait":1}, | |
| 156 | "test/drill-board-line.test.ts": {"wait":1}, | |
| 165 | 157 | "test/drill-job.test.ts": {"wait":1}, |
| 166 | 158 | "test/drill-parts.test.ts": {"wait":1}, |
| 167 | 159 | "test/drill-run.test.ts": {"wait":2}, |
| @@ -210,20 +202,18 @@ | ||
| 210 | 202 | "test/fork-follow-conflict.test.ts": {"wait":1}, |
| 211 | 203 | "test/fork-follow-mode.test.ts": {"wait":3}, |
| 212 | 204 | "test/fork-pull-approval.test.ts": {"wait":4}, |
| 213 | "test/fork-sever.test.ts": {"wait":1}, | |
| 214 | "test/fork-standing-line.test.ts": {"wait":2}, | |
| 205 | "test/fork-standing-line.test.ts": {"wait":1}, | |
| 215 | 206 | "test/forks-ranking.test.ts": {"wait":1}, |
| 216 | 207 | "test/front.test.ts": {"wait":1}, |
| 217 | 208 | "test/full-disk.test.ts": {"timeout":1,"wait":2}, |
| 218 | 209 | "test/gate-drain.test.ts": {"wait":2}, |
| 219 | 210 | "test/gate-interrupted.test.ts": {"wait":1}, |
| 220 | "test/gate-last-slot.test.ts": {"wait":2}, | |
| 211 | "test/gate-last-slot.test.ts": {"wait":1}, | |
| 221 | 212 | "test/gate-update-pull.test.ts": {"wait":1}, |
| 222 | 213 | "test/git-pace-wiring.test.ts": {"wait":2}, |
| 223 | 214 | "test/git-ssh.test.ts": {"wait":1}, |
| 224 | 215 | "test/guest-pull-env.test.ts": {"wait":1}, |
| 225 | 216 | "test/guest-pull.test.ts": {"wait":1}, |
| 226 | "test/handover-give-back.test.ts": {"wait":1}, | |
| 227 | 217 | "test/hardening.test.ts": {"wait":2}, |
| 228 | 218 | "test/health-vitals-reuse.test.ts": {"wait":1}, |
| 229 | 219 | "test/helpers/apps-forge.ts": {"wait":2}, |
| @@ -234,7 +224,6 @@ | ||
| 234 | 224 | "test/helpers/handover-roll.ts": {"wait":6}, |
| 235 | 225 | "test/helpers/overlap-worker.ts": {"wait":2}, |
| 236 | 226 | "test/helpers/parks-engine.ts": {"wait":1}, |
| 237 | "test/helpers/poll.ts": {"wait":1}, | |
| 238 | 227 | "test/helpers/pull-goes-back.ts": {"wait":1}, |
| 239 | 228 | "test/helpers/real-child-forge.ts": {"wait":1}, |
| 240 | 229 | "test/helpers/real-move-e2e.ts": {"wait":1}, |
| @@ -274,14 +263,10 @@ | ||
| 274 | 263 | "test/mcp-org-teams-waiting.test.ts": {"wait":1}, |
| 275 | 264 | "test/mcp-restore-drill.test.ts": {"wait":1}, |
| 276 | 265 | "test/mcp-sync.test.ts": {"timeout":1}, |
| 277 | "test/media.test.ts": {"wait":2}, | |
| 278 | 266 | "test/memory-hold.test.ts": {"wait":1}, |
| 279 | "test/merge-queue-batch.test.ts": {"wait":3}, | |
| 280 | "test/merge-queue-clash.test.ts": {"wait":2}, | |
| 281 | "test/merge-queue-hold.test.ts": {"wait":2}, | |
| 282 | "test/merge-queue-parallel.test.ts": {"wait":4}, | |
| 283 | "test/merge-queue-restart.test.ts": {"wait":7}, | |
| 284 | "test/merge-queue-suspect.test.ts": {"wait":1}, | |
| 267 | "test/merge-queue-clash.test.ts": {"wait":1}, | |
| 268 | "test/merge-queue-hold.test.ts": {"wait":1}, | |
| 269 | "test/merge-queue-restart.test.ts": {"wait":4}, | |
| 285 | 270 | "test/merge-queue.test.ts": {"wait":2}, |
| 286 | 271 | "test/meter.test.ts": {"wait":2}, |
| 287 | 272 | "test/migrate-after.test.ts": {"timeout":1}, |
| @@ -317,19 +302,16 @@ | ||
| 317 | 302 | "test/origin-review-actions.test.ts": {"timeout":1}, |
| 318 | 303 | "test/origins-reconnect.test.ts": {"timeout":1,"wait":1}, |
| 319 | 304 | "test/origins-sync-settled.test.ts": {"wait":1}, |
| 320 | "test/orphan-dispatch.test.ts": {"wait":1}, | |
| 321 | 305 | "test/outsiders-member.test.ts": {"wait":1}, |
| 322 | 306 | "test/overlap.test.ts": {"wait":2}, |
| 323 | 307 | "test/overlay-host.test.ts": {"wait":3}, |
| 324 | 308 | "test/own-code.test.ts": {"wait":1}, |
| 325 | 309 | "test/pace-floor.test.ts": {"wait":1}, |
| 326 | 310 | "test/packs-mirror.test.ts": {"wait":1}, |
| 327 | "test/packs-up.test.ts": {"wait":1}, | |
| 328 | 311 | "test/parent-link.test.ts": {"timeout":1}, |
| 329 | 312 | "test/parent-releases-idle.test.ts": {"wait":1}, |
| 330 | 313 | "test/parked-close.test.ts": {"wait":1}, |
| 331 | 314 | "test/perf-duty.test.ts": {"wait":4}, |
| 332 | "test/perf-server-hook.test.ts": {"wait":1}, | |
| 333 | 315 | "test/perf-stall-blame.test.ts": {"wait":2}, |
| 334 | 316 | "test/pinned-forge-address.test.ts": {"wait":1}, |
| 335 | 317 | "test/pinned-forge-recovery.test.ts": {"wait":2}, |
| @@ -365,7 +347,6 @@ | ||
| 365 | 347 | "test/render-subtitles-shift.test.ts": {"wait":1}, |
| 366 | 348 | "test/repo-transfer-follow.test.ts": {"wait":2}, |
| 367 | 349 | "test/retention-jobs.test.ts": {"wait":3}, |
| 368 | "test/roll-write-pause.test.ts": {"wait":2}, | |
| 369 | 350 | "test/run-cancel-ended.test.ts": {"wait":1}, |
| 370 | 351 | "test/run-cancel-stopping.test.ts": {"wait":2}, |
| 371 | 352 | "test/run-cancel-tree.test.ts": {"wait":4}, |
| @@ -395,28 +376,26 @@ | ||
| 395 | 376 | "test/snapshot-outcome.test.ts": {"wait":2}, |
| 396 | 377 | "test/snapshot-restore-clones.test.ts": {"wait":2}, |
| 397 | 378 | "test/snapshot-restore-packed.test.ts": {"wait":1}, |
| 398 | "test/snapshot-restore-streamed.test.ts": {"wait":6}, | |
| 379 | "test/snapshot-restore-streamed.test.ts": {"wait":5}, | |
| 399 | 380 | "test/snapshot-slow-line.test.ts": {"wait":1}, |
| 400 | 381 | "test/snapshot-uploads.test.ts": {"wait":4}, |
| 401 | 382 | "test/spawn-app-links.test.ts": {"wait":1}, |
| 402 | 383 | "test/spawn-births.test.ts": {"wait":7}, |
| 403 | "test/spawn-cell-socket.test.ts": {"wait":1}, | |
| 404 | 384 | "test/spawn-cell-stopping.test.ts": {"wait":1}, |
| 405 | "test/spawn-cell.test.ts": {"wait":4}, | |
| 385 | "test/spawn-cell.test.ts": {"wait":2}, | |
| 406 | 386 | "test/spawn-chain.test.ts": {"wait":1}, |
| 407 | 387 | "test/spawn-child-runs-apps.test.ts": {"wait":1}, |
| 408 | 388 | "test/spawn-events.test.ts": {"wait":2}, |
| 409 | 389 | "test/spawn-limits-listed.test.ts": {"wait":1}, |
| 410 | 390 | "test/spawn-live.test.ts": {"wait":2}, |
| 411 | 391 | "test/spawn-move-cell-sizes.test.ts": {"wait":1}, |
| 412 | "test/spawn-reach-door.test.ts": {"wait":1}, | |
| 413 | 392 | "test/spawn-sleep-together.test.ts": {"wait":1}, |
| 414 | "test/spawn-stopped-restart.test.ts": {"wait":4}, | |
| 393 | "test/spawn-stopped-restart.test.ts": {"wait":1}, | |
| 415 | 394 | "test/ssh-app.test.ts": {"wait":5}, |
| 416 | 395 | "test/ssh-door-limits.test.ts": {"wait":3}, |
| 417 | 396 | "test/ssh-handover.test.ts": {"wait":1}, |
| 418 | 397 | "test/ssh-port-taken.test.ts": {"wait":1}, |
| 419 | "test/ssh-shell-cold.test.ts": {"wait":3}, | |
| 398 | "test/ssh-shell-cold.test.ts": {"wait":2}, | |
| 420 | 399 | "test/ssh-version-exchange.test.ts": {"wait":1}, |
| 421 | 400 | "test/ssh.test.ts": {"wait":3}, |
| 422 | 401 | "test/stall-in-flight.test.ts": {"wait":1}, |
| @@ -427,7 +406,7 @@ | ||
| 427 | 406 | "test/steer.test.ts": {"wait":1}, |
| 428 | 407 | "test/steps-stop-tree.test.ts": {"wait":2}, |
| 429 | 408 | "test/stranger-first-ask-journey.test.ts": {"wait":1}, |
| 430 | "test/stranger-pull.test.ts": {"wait":2}, | |
| 409 | "test/stranger-pull.test.ts": {"wait":1}, | |
| 431 | 410 | "test/stranger-three-screens.test.ts": {"wait":3}, |
| 432 | 411 | "test/stripe-hook.test.ts": {"wait":3}, |
| 433 | 412 | "test/stuck-pile-doors.test.ts": {"wait":1}, |
| @@ -435,9 +414,8 @@ | ||
| 435 | 414 | "test/support/landsim.ts": {"wait":1}, |
| 436 | 415 | "test/surfaces-adversarial.test.ts": {"wait":1}, |
| 437 | 416 | "test/talk-build.test.ts": {"wait":1}, |
| 438 | "test/talk-page.test.ts": {"wait":7}, | |
| 417 | "test/talk-page.test.ts": {"wait":5}, | |
| 439 | 418 | "test/talk-slash.test.ts": {"wait":1}, |
| 440 | "test/talk-stop.test.ts": {"wait":2}, | |
| 441 | 419 | "test/team-dispatch.test.ts": {"wait":1}, |
| 442 | 420 | "test/team-merge-gate.test.ts": {"wait":1}, |
| 443 | 421 | "test/team-queue.test.ts": {"wait":1}, |
| @@ -446,8 +424,7 @@ | ||
| 446 | 424 | "test/template-scripts.test.ts": {"timeout":1}, |
| 447 | 425 | "test/templates-boot.test.ts": {"wait":1}, |
| 448 | 426 | "test/test-budget-run.test.ts": {"wait":1}, |
| 449 | "test/thanks.test.ts": {"wait":1}, | |
| 450 | "test/timers-sync.test.ts": {"wait":2}, | |
| 427 | "test/timers-sync.test.ts": {"wait":1}, | |
| 451 | 428 | "test/toolbar-keyboard.test.ts": {"wait":1}, |
| 452 | 429 | "test/toolbar-move.test.ts": {"wait":1}, |
| 453 | 430 | "test/tree-read.test.ts": {"wait":5}, |
| @@ -476,7 +453,6 @@ | ||
| 476 | 453 | "test/tui-following.test.ts": {"wait":1}, |
| 477 | 454 | "test/tui-forgeview-live.test.ts": {"wait":2}, |
| 478 | 455 | "test/tui-forks-propose.test.ts": {"wait":1}, |
| 479 | "test/tui-forks-take.test.ts": {"wait":1}, | |
| 480 | 456 | "test/tui-hooks-deliveries.test.ts": {"wait":1}, |
| 481 | 457 | "test/tui-hooks.test.ts": {"wait":3}, |
| 482 | 458 | "test/tui-issue-flow-first.test.ts": {"wait":1}, |
| @@ -488,9 +464,8 @@ | ||
| 488 | 464 | "test/tui-milestone-watch.test.ts": {"wait":1}, |
| 489 | 465 | "test/tui-newapp.test.ts": {"wait":1}, |
| 490 | 466 | "test/tui-next-up.test.ts": {"wait":2}, |
| 491 | "test/tui-origins-forge.test.ts": {"wait":2}, | |
| 467 | "test/tui-origins-forge.test.ts": {"wait":1}, | |
| 492 | 468 | "test/tui-origins-guard.test.ts": {"wait":1}, |
| 493 | "test/tui-origins.test.ts": {"wait":1}, | |
| 494 | 469 | "test/tui-people-verify.test.ts": {"wait":1}, |
| 495 | 470 | "test/tui-resilience-drill.test.ts": {"wait":2}, |
| 496 | 471 | "test/tui-resilience-now.test.ts": {"wait":3}, |
| @@ -512,7 +487,7 @@ | ||
| 512 | 487 | "test/vitals-held-agent.test.ts": {"wait":1}, |
| 513 | 488 | "test/vitals.test.ts": {"wait":3}, |
| 514 | 489 | "test/watches.test.ts": {"wait":3}, |
| 515 | "test/web-apps.test.ts": {"wait":4}, | |
| 490 | "test/web-apps.test.ts": {"wait":3}, | |
| 516 | 491 | "test/web-origins.test.ts": {"timeout":1}, |
| 517 | 492 | "test/web-ui.test.ts": {"wait":2}, |
| 518 | 493 | "test/week-doors.test.ts": {"wait":1}, |
test/account-ceiling-reserve.test.ts +2 −1
| @@ -20,6 +20,7 @@ import { readRunLog } from "../src/runs/log.ts"; | ||
| 20 | 20 | import { setLimit } from "../src/settings/limits.ts"; |
| 21 | 21 | import { agentRunSettled, startStandInCrew, stopStandInCrew, type StandInCrew } from "./helpers/crew-standin.ts"; |
| 22 | 22 | import { promote } from "./helpers/members.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | /** The budget a run's log says it was handed, in dollars. */ |
| 25 | 26 | function handed(standIn: StandInCrew, run: Run): number { |
| @@ -71,7 +72,7 @@ describe("the daily ceiling is reserved when a run starts", () => { | ||
| 71 | 72 | } |
| 72 | 73 | // Both are running before either is let finish. |
| 73 | 74 | const running = () => listRuns(app.db, repo.value.id, { kind: "agent", limit: 10 }).filter((r) => r.state === "running").length; |
| 74 | for (let i = 0; i < 300 && running() < 2; i++) await Bun.sleep(100); | |
| 75 | await waitFor(() => running() >= 2, { every: 100, within: 30_000 }); | |
| 75 | 76 | expect(running()).toBe(2); |
| 76 | 77 | writeFileSync(join(standIn.dataDir, "release"), ""); |
| 77 | 78 | const runs = await Promise.all(settled); |
test/app-tty-sessions.test.ts +2 −1
| @@ -13,6 +13,7 @@ import { appTtysOf, openAppTty } from "../src/apps/tty_registry.ts"; | ||
| 13 | 13 | import { createUser } from "../src/core/users.ts"; |
| 14 | 14 | import { myShells, stopIdleShells, stopShell, stopShells } from "../src/machines/shell.ts"; |
| 15 | 15 | import { quietLogs } from "./helpers/globals.ts"; |
| 16 | import { waitFor } from "./helpers/poll.ts"; | |
| 16 | 17 | |
| 17 | 18 | quietLogs(); |
| 18 | 19 | |
| @@ -80,7 +81,7 @@ function age(userId: string, ms: number) { | ||
| 80 | 81 | } |
| 81 | 82 | |
| 82 | 83 | async function until(ok: () => boolean) { |
| 83 | for (let i = 0; i < 600 && !ok(); i++) await Bun.sleep(25); | |
| 84 | await waitFor(() => ok(), { every: 25, within: 15_000 }); | |
| 84 | 85 | expect(ok()).toBe(true); |
| 85 | 86 | } |
| 86 | 87 |
test/apps-archive-race.test.ts +4 −3
| @@ -13,6 +13,7 @@ import { getRun } from "../src/core/runs.ts"; | ||
| 13 | 13 | import { repoDir } from "../src/git/repos.ts"; |
| 14 | 14 | import { resolveOid } from "../src/git/merge.ts"; |
| 15 | 15 | import { startAppsForge, startHello, stopAppsForge, type AppsForge } from "./helpers/apps-forge.ts"; |
| 16 | import { waitFor } from "./helpers/poll.ts"; | |
| 16 | 17 | |
| 17 | 18 | describe("archive and wake race", () => { |
| 18 | 19 | let forge: AppsForge; |
| @@ -49,14 +50,14 @@ describe("archive and wake race", () => { | ||
| 49 | 50 | if (tip === undefined) throw new Error("no tip"); |
| 50 | 51 | const redeploy = forge.runtime.requestDeploy(record, fresh(), "production", tip, "redeploy"); |
| 51 | 52 | if (redeploy === undefined) throw new Error("no redeploy"); |
| 52 | for (let i = 0; i < 600 && getRun(forge.app.db, redeploy.id)?.state !== "running"; i++) await Bun.sleep(10); | |
| 53 | await waitFor(() => getRun(forge.app.db, redeploy.id)?.state === "running", { every: 10, within: 6_000 }); | |
| 53 | 54 | expect(getRun(forge.app.db, redeploy.id)?.state).toBe("running"); |
| 54 | 55 | const archiving = updateRepoSettings(forge.app, forge.alice, fresh(), { archived: true }); |
| 55 | 56 | const waking = updateRepoSettings(forge.app, forge.alice, fresh(), { archived: false }); |
| 56 | 57 | const [archived, woke] = await Promise.all([archiving, waking]); |
| 57 | 58 | expect(archived.ok && woke.ok).toBe(true); |
| 58 | 59 | expect(fresh().archivedAt).toBeNull(); |
| 59 | for (let i = 0; i < 300 && production() !== "running"; i++) await Bun.sleep(100); | |
| 60 | await waitFor(() => production() === "running", { every: 100, within: 30_000 }); | |
| 60 | 61 | expect(production()).toBe("running"); |
| 61 | 62 | }, 90_000); |
| 62 | 63 | |
| @@ -72,7 +73,7 @@ describe("archive and wake race", () => { | ||
| 72 | 73 | expect(archivedMoves() - before).toBe(1); |
| 73 | 74 | const woke = await updateRepoSettings(forge.app, forge.alice, fresh(), { archived: false }); |
| 74 | 75 | expect(woke.ok).toBe(true); |
| 75 | for (let i = 0; i < 600 && production() !== "running"; i++) await Bun.sleep(100); | |
| 76 | await waitFor(() => production() === "running", { every: 100, within: 60_000 }); | |
| 76 | 77 | expect(production()).toBe("running"); |
| 77 | 78 | }, 90_000); |
| 78 | 79 | }); |
test/boot-beside-peer.test.ts +2 −1
| @@ -15,6 +15,7 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises"; | ||
| 15 | 15 | import { tmpdir } from "node:os"; |
| 16 | 16 | import { join } from "node:path"; |
| 17 | 17 | import { createApp } from "../src/app.ts"; |
| 18 | import { waitFor } from "./helpers/poll.ts"; | |
| 18 | 19 | |
| 19 | 20 | // The peer's ceiling: it lets go sooner, once the forge is up and has |
| 20 | 21 | // answered HELD_PROBES probes while the lock is still held (#12397), so a slow |
| @@ -127,7 +128,7 @@ async function bootBeside() { | ||
| 127 | 128 | await Bun.sleep(200); |
| 128 | 129 | } |
| 129 | 130 | await peer.exited; |
| 130 | for (let i = 0; i < 100 && !out.includes("the other forge let go of this database"); i++) await Bun.sleep(100); | |
| 131 | await waitFor(() => out.includes("the other forge let go of this database"), { every: 100, within: 10_000 }); | |
| 131 | 132 | return { up, slow, held, out }; |
| 132 | 133 | } finally { |
| 133 | 134 | peer.kill(); |
test/child-credit.test.ts +3 −2
| @@ -43,6 +43,7 @@ import { fundedEnv, fundingOf, fundingWords, refreshFunding } from "../src/spawn | ||
| 43 | 43 | import { listPriceUsd } from "../src/spawn/list_price.ts"; |
| 44 | 44 | import { setLimit } from "../src/settings/limits.ts"; |
| 45 | 45 | import { quietLogs } from "./helpers/globals.ts"; |
| 46 | import { waitFor } from "./helpers/poll.ts"; | |
| 46 | 47 | |
| 47 | 48 | quietLogs(); |
| 48 | 49 | |
| @@ -332,7 +333,7 @@ describe("a parent funds a child's crew", () => { | ||
| 332 | 333 | const answers = await Promise.all(held.slice(MAX_IN_FLIGHT).map((p) => p)); |
| 333 | 334 | // Calls past the limit were refused at once, without reaching the provider. |
| 334 | 335 | for (const a of answers) expect(a.status).toBe(429); |
| 335 | for (let i = 0; i < 50 && upstreamSeen.length < before + MAX_IN_FLIGHT; i++) await Bun.sleep(10); | |
| 336 | await waitFor(() => upstreamSeen.length >= before + MAX_IN_FLIGHT, { every: 10, within: 500 }); | |
| 336 | 337 | expect(upstreamSeen.length - before).toBe(MAX_IN_FLIGHT); |
| 337 | 338 | const was = spent(); |
| 338 | 339 | open(); |
| @@ -402,7 +403,7 @@ describe("a parent funds a child's crew", () => { | ||
| 402 | 403 | await t.press("enter"); |
| 403 | 404 | await t.type("7.5"); |
| 404 | 405 | await t.press("enter"); |
| 405 | for (let i = 0; i < 50 && readCredit(parent, "maria")!.capUsd !== 7.5; i++) await Bun.sleep(10); | |
| 406 | await waitFor(() => readCredit(parent, "maria")!.capUsd === 7.5, { every: 10, within: 500 }); | |
| 406 | 407 | expect(readCredit(parent, "maria")!.capUsd).toBe(7.5); |
| 407 | 408 | // The same checks as the API: a sum that is not dollars sets nothing. |
| 408 | 409 | await t.palette("Lend a child"); |
test/client-abort.test.ts +6 −5
| @@ -10,6 +10,7 @@ import { accessDir, flushAccess } from "../src/http/access_log.ts"; | ||
| 10 | 10 | import * as health from "../src/http/health.ts"; |
| 11 | 11 | import { serve } from "../src/http/server.ts"; |
| 12 | 12 | import { readUsage } from "../src/core/usage_count.ts"; |
| 13 | import { waitFor } from "./helpers/poll.ts"; | |
| 13 | 14 | |
| 14 | 15 | let dataDir: string; |
| 15 | 16 | let app: App; |
| @@ -35,11 +36,11 @@ test("an aborted slow request is logged and counted as 499, and nothing stays in | ||
| 35 | 36 | const pending = fetch(`${servers.url}/readyz`, { signal: controller.signal }).catch( |
| 36 | 37 | () => "aborted", |
| 37 | 38 | ); |
| 38 | for (let i = 0; i < 100 && servers.inFlight === 0; i++) await Bun.sleep(10); | |
| 39 | await waitFor(() => servers.inFlight !== 0, { every: 10, within: 1_000 }); | |
| 39 | 40 | expect(servers.inFlight).toBe(1); |
| 40 | 41 | controller.abort(); |
| 41 | 42 | expect(await pending).toBe("aborted"); |
| 42 | for (let i = 0; i < 100 && servers.inFlight > 0; i++) await Bun.sleep(10); | |
| 43 | await waitFor(() => servers.inFlight <= 0, { every: 10, within: 1_000 }); | |
| 43 | 44 | |
| 44 | 45 | expect(servers.inFlight).toBe(0); |
| 45 | 46 | flushAccess(); |
| @@ -69,11 +70,11 @@ test("a body that arrives after the client left is cancelled", async () => { | ||
| 69 | 70 | const pending = fetch(`${servers.url}/readyz`, { signal: controller.signal }).catch( |
| 70 | 71 | () => "aborted", |
| 71 | 72 | ); |
| 72 | for (let i = 0; i < 100 && servers.inFlight === 0; i++) await Bun.sleep(10); | |
| 73 | await waitFor(() => servers.inFlight !== 0, { every: 10, within: 1_000 }); | |
| 73 | 74 | expect(servers.inFlight).toBe(1); |
| 74 | 75 | controller.abort(); |
| 75 | 76 | expect(await pending).toBe("aborted"); |
| 76 | for (let i = 0; i < 100 && servers.inFlight > 0; i++) await Bun.sleep(10); | |
| 77 | await waitFor(() => servers.inFlight <= 0, { every: 10, within: 1_000 }); | |
| 77 | 78 | if (servers.inFlight > 0) throw new Error(`never came: still waiting while servers.inFlight > 0`); |
| 78 | 79 | release( |
| 79 | 80 | new Response( |
| @@ -87,7 +88,7 @@ test("a body that arrives after the client left is cancelled", async () => { | ||
| 87 | 88 | }), |
| 88 | 89 | ), |
| 89 | 90 | ); |
| 90 | for (let i = 0; i < 100 && !cancelled; i++) await Bun.sleep(10); | |
| 91 | await waitFor(() => cancelled, { every: 10, within: 1_000 }); | |
| 91 | 92 | expect(cancelled).toBe(true); |
| 92 | 93 | expect(servers.inFlight).toBe(0); |
| 93 | 94 | } finally { |
test/conflicts-tui.test.ts +2 −1
| @@ -17,6 +17,7 @@ import { local } from "../src/tui/screens/where.ts"; | ||
| 17 | 17 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; |
| 18 | 18 | import { quietLogs } from "./helpers/globals.ts"; |
| 19 | 19 | import { settleApp } from "./helpers/tui.ts"; |
| 20 | import { waitFor } from "./helpers/poll.ts"; | |
| 20 | 21 | |
| 21 | 22 | quietLogs(); |
| 22 | 23 | |
| @@ -149,7 +150,7 @@ describe("the terminal reads a conflicted pull's blocks", () => { | ||
| 149 | 150 | const out = await pressV(alice, ["b", "f"]); |
| 150 | 151 | expect(out).not.toContain("<<<<<<<"); |
| 151 | 152 | // The commit lands off the key's own turn: wait for the branch to move. |
| 152 | for (let i = 0; i < 100 && (await tip("refs/heads/loud")) === was; i++) await Bun.sleep(50); | |
| 153 | await waitFor(async () => (await tip("refs/heads/loud")) !== was, { within: 5_000 }); | |
| 153 | 154 | if ((await tip("refs/heads/loud")) === was) throw new Error(`never came: still waiting while (await tip("refs/heads/loud")) === was`); |
| 154 | 155 | const now = await tip("refs/heads/loud"); |
| 155 | 156 | expect(now).not.toBe(was); |
test/conversation-queue-api.test.ts +2 −1
| @@ -20,6 +20,7 @@ import { serve, type Servers } from "../src/http/server.ts"; | ||
| 20 | 20 | import { forgeSshSession } from "../src/tui/ssh.ts"; |
| 21 | 21 | import { parseDuring } from "../src/tui/ssh-talk.ts"; |
| 22 | 22 | import { quietLogs } from "./helpers/globals.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | quietLogs(); |
| 25 | 26 | |
| @@ -37,7 +38,7 @@ const headers = (t: string, extra: Record<string, string> = {}) => ({ "content-t | ||
| 37 | 38 | const send = (body: object, t = rw, extra: Record<string, string> = {}) => |
| 38 | 39 | fetch(`${servers.url}/api/v1/conversation/messages`, { method: "POST", headers: headers(t, extra), body: JSON.stringify(body) }); |
| 39 | 40 | const until = async (ok: () => boolean) => { |
| 40 | for (let i = 0; i < 100 && !ok(); i++) await Bun.sleep(10); | |
| 41 | await waitFor(() => ok(), { every: 10, within: 1_000 }); | |
| 41 | 42 | if (!ok()) throw new Error(`never came: still waiting while !ok()`); |
| 42 | 43 | }; |
| 43 | 44 | /** Starts a turn that holds until stopped, without counting against the minute's messages. */ |
test/conversation-queue.test.ts +2 −1
| @@ -13,6 +13,7 @@ import { createApp, type App } from "../src/app.ts"; | ||
| 13 | 13 | import { listTurns, openConversation } from "../src/core/conversations.ts"; |
| 14 | 14 | import { createUser, type User } from "../src/core/users.ts"; |
| 15 | 15 | import { cancelQueued, queuedOf, say, STOPPED_LINE, stopTurn, turnRunning, type ConversationOptions } from "../src/crew/conversation.ts"; |
| 16 | import { waitFor } from "./helpers/poll.ts"; | |
| 16 | 17 | |
| 17 | 18 | let dir: string; |
| 18 | 19 | let app: App; |
| @@ -46,7 +47,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"partial '$ | ||
| 46 | 47 | const logged = (name: string): string[] => (existsSync(join(dir, `${name}.log`)) ? readFileSync(join(dir, `${name}.log`), "utf8").trim().split("\n") : []); |
| 47 | 48 | /** Waits on the thing itself, and fails saying so when it never comes, instead of carrying on into a wrong answer. */ |
| 48 | 49 | const until = async (ok: () => boolean, what = "the condition"): Promise<void> => { |
| 49 | for (let i = 0; i < 500 && !ok(); i++) await Bun.sleep(20); | |
| 50 | await waitFor(() => ok(), { every: 20, within: 10_000 }); | |
| 50 | 51 | if (!ok()) throw new Error(`timed out waiting for ${what}`); |
| 51 | 52 | }; |
| 52 | 53 | /** The stand-in logs a message before the forge has read its partial answer; stopping in that gap would lose the partial. */ |
test/conversation-talk.test.ts +2 −1
| @@ -27,6 +27,7 @@ import { createSession } from "../src/core/sessions.ts"; | ||
| 27 | 27 | import { conversationOptions, redo, withHistory } from "../src/crew/conversation.ts"; |
| 28 | 28 | import { exchangeActs, turnHtml } from "../src/web/pages/talk.ts"; |
| 29 | 29 | import { quietLogs } from "./helpers/globals.ts"; |
| 30 | import { waitFor } from "./helpers/poll.ts"; | |
| 30 | 31 | |
| 31 | 32 | quietLogs(); |
| 32 | 33 | |
| @@ -539,7 +540,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"Heard.","t | ||
| 539 | 540 | }); |
| 540 | 541 | const settleBob = async () => { |
| 541 | 542 | // The stand-in records the message it was started with; wait for a new one, then for the end. |
| 542 | for (let i = 0; i < 80 && readFileSync(join(dir, "first.json"), "utf8") === seen; i++) await Bun.sleep(25); | |
| 543 | await waitFor(() => readFileSync(join(dir, "first.json"), "utf8") !== seen, { every: 25, within: 2_000 }); | |
| 543 | 544 | if (readFileSync(join(dir, "first.json"), "utf8") === seen) throw new Error(`never came: still waiting while readFileSync(join(dir, "first.json"), "utf8") === seen`); |
| 544 | 545 | seen = readFileSync(join(dir, "first.json"), "utf8"); |
| 545 | 546 | for (let i = 0; i < 200; i++) { |
test/conversation-turn.test.ts +3 −2
| @@ -25,6 +25,7 @@ import { answerHeld, hold, waitingFor } from "../src/core/fence.ts"; | ||
| 25 | 25 | import { flowFor } from "../src/system/telemetry.ts"; |
| 26 | 26 | import { readTalkTurns, talkTurnsWords } from "../src/flow/talk_turns.ts"; |
| 27 | 27 | import { MCP_SERVER, mcpConfig, RESUMED_LINE, resumeInterrupted, say, STOPPED_LINE, stopTurn, turnRunning, unendedLine, TURNS_LIVE_MAX, type ConversationOptions } from "../src/crew/conversation.ts"; |
| 28 | import { waitFor } from "./helpers/poll.ts"; | |
| 28 | 29 | |
| 29 | 30 | let dir: string; |
| 30 | 31 | let app: App; |
| @@ -246,7 +247,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"Back.","to | ||
| 246 | 247 | const cut = say(app, options(hang, stop.signal), alice.id, "look at the pulls"); |
| 247 | 248 | while (!existsSync(join(dir, "cut.first"))) await Bun.sleep(20); |
| 248 | 249 | // The session is stored the moment it is named. |
| 249 | for (let i = 0; i < 100 && openConversation(app.db, alice.id).claudeSessionId !== "sess-cut"; i++) await Bun.sleep(20); | |
| 250 | await waitFor(() => openConversation(app.db, alice.id).claudeSessionId === "sess-cut", { every: 20, within: 2_000 }); | |
| 250 | 251 | expect(openConversation(app.db, alice.id).claudeSessionId).toBe("sess-cut"); |
| 251 | 252 | // Mid-turn the token is alice's own. |
| 252 | 253 | const live = verifyToken(app.db, readFileSync(join(dir, "cut.token"), "utf8")); |
| @@ -498,7 +499,7 @@ sleep 300 | ||
| 498 | 499 | chmodSync(path, 0o755); |
| 499 | 500 | const said = say(app, options(path), u.value.id, "do a slow thing"); |
| 500 | 501 | if (!said.ok || said.value.done === undefined) throw new Error("not started"); |
| 501 | for (let i = 0; i < 100 && !turns(u.value).some((t) => t === "agent:Looking into it."); i++) await Bun.sleep(50); | |
| 502 | await waitFor(() => turns(u.value).some((t) => t === "agent:Looking into it."), { within: 5_000 }); | |
| 502 | 503 | expect(turns(u.value)).toContain("agent:Looking into it."); |
| 503 | 504 | const pid = Number(readFileSync(pidFile, "utf8").trim()); |
| 504 | 505 | const via = `conversation:${openConversation(app.db, u.value.id).id}`; |
test/crew-mirror.test.ts +3 −2
| @@ -13,6 +13,7 @@ import { createRepo, deleteRepo } from "../src/actions/repos.ts"; | ||
| 13 | 13 | import { signUpProven as signUp } from "./helpers/verified.ts"; |
| 14 | 14 | import { cloneWithMirror, mirrorDir } from "../src/crew/mirror.ts"; |
| 15 | 15 | import { sandboxPolicy } from "../src/crew/sandbox.ts"; |
| 16 | import { waitFor } from "./helpers/poll.ts"; | |
| 16 | 17 | |
| 17 | 18 | let root: string; |
| 18 | 19 | let origin: string; |
| @@ -168,7 +169,7 @@ test("deleting a repository deletes its mirror without waiting for a running clo | ||
| 168 | 169 | // The delete did not wait for the clone holding the lock. |
| 169 | 170 | expect(running.settled()).toBe(false); |
| 170 | 171 | expect((await running.done).ok).toBe(true); |
| 171 | for (let i = 0; i < 60 && existsSync(mirrorDir(dataDir, made.value.id)); i++) await Bun.sleep(50); | |
| 172 | await waitFor(() => !existsSync(mirrorDir(dataDir, made.value.id)), { within: 3_000 }); | |
| 172 | 173 | expect(existsSync(mirrorDir(dataDir, made.value.id))).toBe(false); |
| 173 | 174 | } finally { |
| 174 | 175 | app.close(); |
| @@ -238,7 +239,7 @@ test("removing an account deletes its repositories' mirrors without waiting for | ||
| 238 | 239 | expect(removed.ok).toBe(true); |
| 239 | 240 | expect(running.settled()).toBe(false); |
| 240 | 241 | expect((await running.done).ok).toBe(true); |
| 241 | for (let i = 0; i < 60 && existsSync(mirrorDir(dataDir, made.value.id)); i++) await Bun.sleep(50); | |
| 242 | await waitFor(() => !existsSync(mirrorDir(dataDir, made.value.id)), { within: 3_000 }); | |
| 242 | 243 | expect(existsSync(mirrorDir(dataDir, made.value.id))).toBe(false); |
| 243 | 244 | } finally { |
| 244 | 245 | app.close(); |
test/crew-own-forge.test.ts +3 −2
| @@ -23,6 +23,7 @@ import { wokeWords } from "../src/crew/wake.ts"; | ||
| 23 | 23 | import { readRunLog } from "../src/runs/log.ts"; |
| 24 | 24 | import { promote } from "./helpers/members.ts"; |
| 25 | 25 | import { agentRunSettled, startStandInCrew, stopStandInCrew, type StandInCrew } from "./helpers/crew-standin.ts"; |
| 26 | import { waitFor } from "./helpers/poll.ts"; | |
| 26 | 27 | |
| 27 | 28 | describe("a forge of its own: its key, its limits, and the work picked back up", () => { |
| 28 | 29 | let standIn: StandInCrew; |
| @@ -72,7 +73,7 @@ describe("a forge of its own: its key, its limits, and the work picked back up", | ||
| 72 | 73 | expect(first.summary).toContain("This forge has no model key"); |
| 73 | 74 | expect(first.summary).not.toContain("ask an admin"); |
| 74 | 75 | expect(existsSync(standIn.argsFile)).toBe(false); |
| 75 | for (let i = 0; i < 100 && parkedCount(app.db, repo, issue.value.id) === 0; i++) await Bun.sleep(100); | |
| 76 | await waitFor(() => parkedCount(app.db, repo, issue.value.id) !== 0, { every: 100, within: 10_000 }); | |
| 76 | 77 | expect(parkedCount(app.db, repo, issue.value.id)).toBe(1); |
| 77 | 78 | |
| 78 | 79 | app.db.query("UPDATE user SET is_admin = 1 WHERE id = ?1").run(alice.id); |
| @@ -110,7 +111,7 @@ describe("a forge of its own: its key, its limits, and the work picked back up", | ||
| 110 | 111 | expect(stopped.summary).toContain("An admin of this forge can raise it."); |
| 111 | 112 | expect(wayIn(app, undefined, stopped.summary)?.setting).toBe("crew_limit"); |
| 112 | 113 | expect(existsSync(standIn.argsFile)).toBe(false); |
| 113 | for (let i = 0; i < 100 && parkedCount(app.db, repo, issue.value.id) === 0; i++) await Bun.sleep(100); | |
| 114 | await waitFor(() => parkedCount(app.db, repo, issue.value.id) !== 0, { every: 100, within: 10_000 }); | |
| 114 | 115 | |
| 115 | 116 | expect(saveCrewBudgets(app, admin, { run: 5 })).toEqual({ ok: true, woken: 1 }); |
| 116 | 117 | const again = await settledNth(issue.value.id, 2); |
test/door-pulls.test.ts +2 −1
| @@ -15,6 +15,7 @@ import type { Repo } from "../src/core/repos.ts"; | ||
| 15 | 15 | import { createUser, type User } from "../src/core/users.ts"; |
| 16 | 16 | import { forgeSshSession } from "../src/tui/ssh.ts"; |
| 17 | 17 | import { promote } from "./helpers/members.ts"; |
| 18 | import { waitFor } from "./helpers/poll.ts"; | |
| 18 | 19 | |
| 19 | 20 | const KEYS = [ |
| 20 | 21 | "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB2S7bF0R4NmuTUvVJXPPjdI1AF8XHXeYSNfr8DJ5eDo bob", |
| @@ -104,7 +105,7 @@ describe("an app's open pulls over ssh", () => { | ||
| 104 | 105 | drained: async () => {}, |
| 105 | 106 | }; |
| 106 | 107 | forgeSshSession(app)(session as never); |
| 107 | for (let i = 0; i < 50 && code === undefined; i++) await Bun.sleep(10); | |
| 108 | await waitFor(() => code !== undefined, { every: 10, within: 500 }); | |
| 108 | 109 | return { out, err, code }; |
| 109 | 110 | } |
| 110 | 111 |
test/drill-board-line.test.ts +2 −1
| @@ -15,6 +15,7 @@ import { vitals } from "../src/kernel/vitals.ts"; | ||
| 15 | 15 | import type { DrillResult } from "../src/snapshot/drill.ts"; |
| 16 | 16 | import { DRILL_STALE_MS, recordVerdict, seedFirstSeen } from "../src/snapshot/verdict.ts"; |
| 17 | 17 | import { forgeNeeds } from "../src/actions/health.ts"; |
| 18 | import { waitFor } from "./helpers/poll.ts"; | |
| 18 | 19 | |
| 19 | 20 | const DAY = 86_400_000; |
| 20 | 21 | const T0 = Date.parse("2026-10-01T00:00:00Z"); |
| @@ -150,7 +151,7 @@ describe("the drill's board line", () => { | ||
| 150 | 151 | await n.settled(); |
| 151 | 152 | expect(sent).toHaveLength(0); |
| 152 | 153 | clock = T0 + 10 * DAY + DRILL_STALE_MS + 1; |
| 153 | for (let i = 0; i < 100 && sent.length === 0; i++) await Bun.sleep(10); | |
| 154 | await waitFor(() => sent.length !== 0, { every: 10, within: 1_000 }); | |
| 154 | 155 | if (sent.length === 0) throw new Error(`never came: still waiting while sent.length === 0`); |
| 155 | 156 | await n.settled(); |
| 156 | 157 | await Bun.sleep(50); |
test/drill-budget-size.test.ts +2 −1
| @@ -20,6 +20,7 @@ import { restoreSnapshot } from "../src/snapshot/restore.ts"; | ||
| 20 | 20 | import { listSnapshots, takeSnapshot } from "../src/snapshot/snapshot.ts"; |
| 21 | 21 | import { CALL_DEADLINE_MS, dirStore, withRetries, type Store } from "../src/snapshot/store.ts"; |
| 22 | 22 | import { quietLogs } from "./helpers/globals.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | quietLogs(); |
| 25 | 26 | |
| @@ -156,7 +157,7 @@ describe("a 400 MB snapshot's drill", () => { | ||
| 156 | 157 | expect(r.ms).toBeLessThan(r.budgetMs + 40); |
| 157 | 158 | // Five unbounded 180 s attempts from the stall (~300 s in) would end |
| 158 | 159 | // past 1200 s; held inside what is left, the get gives up by 1000. |
| 159 | for (let i = 0; i < 50 && failed.at === undefined; i++) await Bun.sleep(10); | |
| 160 | await waitFor(() => failed.at !== undefined, { every: 10, within: 500 }); | |
| 160 | 161 | expect(failed.message).toMatch(/get snapshots\/2026-10-19T08-00-00Z\/files\/blob-3\d failed after \d attempts?/); |
| 161 | 162 | expect(failed.at! - started).toBeLessThan(r.budgetMs + 40); |
| 162 | 163 | // The card says the budget in minutes. |
test/dupes.test.ts added +118
| @@ -0,0 +1,118 @@ | ||
| 1 | /** The duplication report (#14620): shapes, clusters, the tuned threshold, and the gate's note. */ | |
| 2 | ||
| 3 | import { afterAll, beforeAll, expect, test } from "bun:test"; | |
| 4 | import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; | |
| 5 | import { tmpdir } from "node:os"; | |
| 6 | import { join } from "node:path"; | |
| 7 | import { dupesNote } from "../scripts/gate.ts"; | |
| 8 | import { addedRanges, clusters, nearDuplicates, similarity, THRESHOLD, tune, type Labeled } from "../scripts/dupes.ts"; | |
| 9 | import { functionsOf } from "../scripts/dupes_shape.ts"; | |
| 10 | ||
| 11 | const sum = (name: string, rows: string, key: string) => `export function ${name}(${rows}: number[]): number { | |
| 12 | let total = 0; | |
| 13 | for (const row of ${rows}) { | |
| 14 | if (row > 0 && row !== ${key}) total += row * 2; | |
| 15 | else if (row === ${key}) total -= 1; | |
| 16 | } | |
| 17 | const label = \`total of \${${rows}.length}\`; | |
| 18 | return total > 100 ? 100 : label.length + total; | |
| 19 | } | |
| 20 | `; | |
| 21 | ||
| 22 | test("a function's shape ignores its names, strings and numbers, and keeps its keywords and members", () => { | |
| 23 | const [a] = functionsOf("a.ts", sum("alpha", "rows", "7"), 10); | |
| 24 | const [b] = functionsOf("b.ts", sum("beta", "items", "9"), 10); | |
| 25 | expect(a!.id).toBe("a.ts#alpha"); | |
| 26 | expect(a!.shape).toEqual(b!.shape); | |
| 27 | expect(a!.shape).toContain("for"); | |
| 28 | expect(a!.shape).toContain(".length"); | |
| 29 | expect(a!.shape).not.toContain("rows"); | |
| 30 | expect(similarity(a!.shape, b!.shape)).toBe(1); | |
| 31 | }); | |
| 32 | ||
| 33 | test("a brace in a string, a comment or a regex does not end a function early", () => { | |
| 34 | const text = `function f(a: string) {\n const s = "}"; // }\n /* } */\n const r = /}/.test(a);\n return \`\${a}}\`;\n}\nconst after = 1;\n`; | |
| 35 | const [f] = functionsOf("x.ts", text, 5); | |
| 36 | expect(f!.shape.at(-1)).toBe("}"); | |
| 37 | expect(f!.shape.filter((t) => t === "}").length).toBe(f!.shape.filter((t) => t === "{").length + 0); | |
| 38 | }); | |
| 39 | ||
| 40 | test("arrow functions with a block body and test bodies are found, small ones are not", () => { | |
| 41 | const text = `const make = async (a: number): Promise<number> => {\n const b = a + 1;\n if (b > 2) return b * 2;\n return b;\n};\ntest("it does a thing", () => {\n const x = make(1);\n expect(x).toBe(2);\n});\nfunction tiny() { return 1; }\n`; | |
| 42 | expect(functionsOf("t.ts", text, 8).map((f) => `${f.kind}:${f.id}`)).toEqual(["function:t.ts#make", "test:t.ts#test: it does a thing"]); | |
| 43 | }); | |
| 44 | ||
| 45 | test("three near-duplicates are one cluster and the shared shape is what they all hold", () => { | |
| 46 | const fns = [ | |
| 47 | ...functionsOf("a.ts", sum("alpha", "rows", "7"), 10), | |
| 48 | ...functionsOf("b.ts", sum("beta", "items", "9"), 10), | |
| 49 | ...functionsOf("c.ts", sum("gamma", "xs", "3").replace("total -= 1", "total -= 2"), 10), | |
| 50 | ...functionsOf("d.ts", `export function other(a: string): string {\n try {\n return JSON.parse(a).name ?? "";\n } catch {\n return "";\n }\n}\n`, 5), | |
| 51 | ]; | |
| 52 | const found = clusters(fns); | |
| 53 | expect(found.length).toBe(1); | |
| 54 | expect(found[0]!.members.map((m) => m.id)).toEqual(["a.ts#alpha", "b.ts#beta", "c.ts#gamma"]); | |
| 55 | expect(found[0]!.shared.length).toBeGreaterThan(40); | |
| 56 | expect(clusters(fns.slice(0, 2))).toEqual([]); | |
| 57 | }); | |
| 58 | ||
| 59 | test("a new function is named with the existing one it is near, and never a test body or itself", () => { | |
| 60 | const old = functionsOf("src/old.ts", sum("alpha", "rows", "7"), 10); | |
| 61 | const added = functionsOf("src/new.ts", sum("beta", "items", "9"), 10); | |
| 62 | const [hit] = nearDuplicates(added, [...old, ...added]); | |
| 63 | expect(hit!.like.id).toBe("src/old.ts#alpha"); | |
| 64 | expect(hit!.score).toBeGreaterThanOrEqual(THRESHOLD); | |
| 65 | expect(nearDuplicates(added, added)).toEqual([]); | |
| 66 | expect(nearDuplicates(added, old.map((f) => ({ ...f, kind: "test" as const })))).toEqual([]); | |
| 67 | }); | |
| 68 | ||
| 69 | test("a diff's added line ranges are read per file", () => { | |
| 70 | const diff = `+++ b/src/a.ts\n@@ -1,2 +3,4 @@\n+x\n@@ -9 +20 @@\n+y\n@@ -30,2 +0,0 @@\n-gone\n+++ b/test/b.test.ts\n@@ -0,0 +1,2 @@\n`; | |
| 71 | expect(addedRanges(diff)).toEqual(new Map([["src/a.ts", [[3, 6], [20, 20]]], ["test/b.test.ts", [[1, 2]]]])); | |
| 72 | }); | |
| 73 | ||
| 74 | test("the threshold is the one that best splits the forge's own labeled pairs", () => { | |
| 75 | const pairs = JSON.parse(readFileSync("scripts/dupes-pairs.json", "utf8")) as Labeled[]; | |
| 76 | expect(pairs.length).toBeGreaterThanOrEqual(30); | |
| 77 | expect(pairs.some((p) => p.same) && pairs.some((p) => !p.same)).toBe(true); | |
| 78 | const fns = [...new Set(pairs.flatMap((p) => [p.a, p.b].map((id) => id.split("#")[0]!)))].flatMap((file) => functionsOf(file, readFileSync(file, "utf8"))); | |
| 79 | const rows = tune(fns, pairs); | |
| 80 | const best = Math.max(...rows.map((r) => r.f1)); | |
| 81 | expect(rows.find((r) => r.threshold === THRESHOLD)!.f1).toBe(best); | |
| 82 | }); | |
| 83 | ||
| 84 | let dir: string; | |
| 85 | const git = (...args: string[]) => { | |
| 86 | const r = Bun.spawnSync(["git", ...args], { cwd: dir, stderr: "pipe" }); | |
| 87 | if (r.exitCode !== 0) throw new Error(r.stderr.toString()); | |
| 88 | return r.stdout.toString().trim(); | |
| 89 | }; | |
| 90 | ||
| 91 | beforeAll(() => { | |
| 92 | dir = mkdtempSync(join(tmpdir(), "dupes-")); | |
| 93 | git("init", "-q", "-b", "main"); | |
| 94 | git("config", "user.email", "t@t"); | |
| 95 | git("config", "user.name", "t"); | |
| 96 | mkdirSync(join(dir, "src")); | |
| 97 | writeFileSync(join(dir, "src/old.ts"), sum("alpha", "rows", "7")); | |
| 98 | git("add", "-A"); | |
| 99 | git("commit", "-qm", "base"); | |
| 100 | }); | |
| 101 | afterAll(() => rmSync(dir, { recursive: true, force: true })); | |
| 102 | ||
| 103 | test("the gate says, and does not refuse, when a pull adds a function near an existing one", async () => { | |
| 104 | const base = git("rev-parse", "HEAD"); | |
| 105 | writeFileSync(join(dir, "src/new.ts"), sum("beta", "items", "9")); | |
| 106 | writeFileSync(join(dir, "src/other.ts"), `export function lone(a: string): string {\n try {\n return JSON.parse(a).name ?? "";\n } catch {\n return "";\n }\n return a + "x" + a + "y" + a;\n}\n`); | |
| 107 | git("add", "-A"); | |
| 108 | git("commit", "-qm", "add"); | |
| 109 | process.env.FORGE_CHECK_BASE = base; | |
| 110 | try { | |
| 111 | const note = await dupesNote(dir); | |
| 112 | expect(note.length).toBe(1); | |
| 113 | expect(note[0]).toContain("src/new.ts#beta"); | |
| 114 | expect(note[0]).toContain("src/old.ts#alpha"); | |
| 115 | } finally { | |
| 116 | delete process.env.FORGE_CHECK_BASE; | |
| 117 | } | |
| 118 | }); |
test/fork-sever.test.ts +2 −1
| @@ -18,6 +18,7 @@ import { createUser, type User } from "../src/core/users.ts"; | ||
| 18 | 18 | import { runGit } from "../src/git/env.ts"; |
| 19 | 19 | import { repoDir } from "../src/git/repos.ts"; |
| 20 | 20 | import { quietLogs } from "./helpers/globals.ts"; |
| 21 | import { waitFor } from "./helpers/poll.ts"; | |
| 21 | 22 | |
| 22 | 23 | quietLogs(); |
| 23 | 24 | |
| @@ -77,7 +78,7 @@ test("a severed fork is told nothing when its upstream changes; an attached sibl | ||
| 77 | 78 | const before = await tip(severed); |
| 78 | 79 | await commit(parent, "upstream.txt"); |
| 79 | 80 | // The follow-on runs off the event; give it the same turns it needs for the sibling. |
| 80 | for (let i = 0; i < 100 && !diverged.includes(kept.id); i++) await Bun.sleep(20); // real-timer: polls for a git event follow-on that has no injectable clock | |
| 81 | await waitFor(() => diverged.includes(kept.id), { every: 20, within: 2_000 }); // real-timer: polls for a git event follow-on that has no injectable clock | |
| 81 | 82 | expect(diverged).toContain(kept.id); |
| 82 | 83 | expect(diverged).not.toContain(severed.id); |
| 83 | 84 | expect(await tip(severed)).toBe(before); |
test/fork-standing-line.test.ts +2 −1
| @@ -16,6 +16,7 @@ import { getRepo } from "../src/core/repos.ts"; | ||
| 16 | 16 | import { createUser } from "../src/core/users.ts"; |
| 17 | 17 | import { serve, type Servers } from "../src/http/server.ts"; |
| 18 | 18 | import { promote } from "./helpers/members.ts"; |
| 19 | import { waitFor } from "./helpers/poll.ts"; | |
| 19 | 20 | |
| 20 | 21 | let dataDir: string; |
| 21 | 22 | let app: App; |
| @@ -114,7 +115,7 @@ test("real takes set 'last took': an automatic fast-forward, a merge by take_upd | ||
| 114 | 115 | // Nothing of bob's own: the parent's push fast-forwards him by itself. |
| 115 | 116 | const one = await commitToBranch(app, alice, lib, { message: "One", files: [{ path: "one.txt", content: "1\n" }] }); |
| 116 | 117 | expect(one.ok).toBe(true); |
| 117 | for (let i = 0; i < 50 && lastTookAt(app.db, bob.id) === null; i++) await Bun.sleep(100); | |
| 118 | await waitFor(() => lastTookAt(app.db, bob.id) !== null, { every: 100, within: 5_000 }); | |
| 118 | 119 | expect(lastTookAt(app.db, bob.id)).not.toBeNull(); |
| 119 | 120 | |
| 120 | 121 | // Work on both sides: a merge, taken through the real MCP tool. |
test/gate-last-slot.test.ts +2 −1
| @@ -20,6 +20,7 @@ import type { Repo } from "../src/core/repos.ts"; | ||
| 20 | 20 | import { cancelRun, createRun, getRun, type Run } from "../src/core/runs.ts"; |
| 21 | 21 | import { createUser, type User } from "../src/core/users.ts"; |
| 22 | 22 | import { serve, type Servers } from "../src/http/server.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | let dataDir: string; |
| 25 | 26 | let app: App; |
| @@ -90,7 +91,7 @@ function formingGate(pull: Pull): Promise<boolean[]> { | ||
| 90 | 91 | |
| 91 | 92 | /** Poll until `ok`, giving up at a generous deadline rather than asserting a speed. */ |
| 92 | 93 | async function until(ok: () => boolean): Promise<boolean> { |
| 93 | for (let i = 0; i < 2000 && !ok(); i++) await Bun.sleep(5); | |
| 94 | await waitFor(() => ok(), { every: 5, within: 10_000 }); | |
| 94 | 95 | return ok(); |
| 95 | 96 | } |
| 96 | 97 |
test/handover-give-back.test.ts +2 −1
| @@ -27,6 +27,7 @@ import { headlessTerminal } from "../src/tui/term/terminal.ts"; | ||
| 27 | 27 | import { settleApp } from "./helpers/tui.ts"; |
| 28 | 28 | import { quietLogs } from "./helpers/globals.ts"; |
| 29 | 29 | import { promote } from "./helpers/members.ts"; |
| 30 | import { waitFor } from "./helpers/poll.ts"; | |
| 30 | 31 | |
| 31 | 32 | quietLogs(); |
| 32 | 33 | |
| @@ -263,7 +264,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"ok","total | ||
| 263 | 264 | say(reviewer, two, "Planner, one more round: please"); |
| 264 | 265 | say(planner.value, plain, "Re-scoped; nothing to add."); |
| 265 | 266 | say(planner.value, person, "Planner, one more round: go"); |
| 266 | for (let i = 0; i < 100 && runsOn(one) === 0; i++) await Bun.sleep(50); | |
| 267 | await waitFor(() => runsOn(one) !== 0, { within: 5_000 }); | |
| 267 | 268 | expect(getIssueById(app.db, one)!.assigneeId).toBe(builder.id); |
| 268 | 269 | expect(runsOn(one)).toBeGreaterThan(0); |
| 269 | 270 | // The planner is an agent, not a director: the hand-over stands. |
test/helpers/poll.ts +21 −6
| @@ -1,8 +1,23 @@ | ||
| 1 | /** Waits until `done` holds, polling; for work that runs on its own pace in the same process. */ | |
| 2 | export async function pollUntil(done: () => boolean, what: string, timeoutMs = 30_000): Promise<void> { | |
| 3 | const deadline = Date.now() + timeoutMs; | |
| 4 | while (!done()) { | |
| 5 | if (Date.now() > deadline) throw new Error(`never came: ${what}`); | |
| 6 | await Bun.sleep(50); | |
| 1 | /** | |
| 2 | * Waits until `done` holds, polling every `every` ms for at most `within` ms; says whether it held. | |
| 3 | * For work that runs on its own pace (a child process, a timer in the code under test) and has no | |
| 4 | * event to await: one wait in one place, so the fixed-wait ratchet (#14321) counts it once, not at | |
| 5 | * every test that polls (#14620: ~90 `for (let i = 0; i < N && cond; i++) await Bun.sleep(M)`). | |
| 6 | */ | |
| 7 | export async function waitFor(done: () => boolean | Promise<boolean>, { every = 50, within = 30_000 } = {}): Promise<boolean> { | |
| 8 | // Counted in polls, as the loops it replaced were, not against the clock: a loaded machine | |
| 9 | // stretches each sleep, and a wall-clock deadline would then give up sooner than they did. | |
| 10 | for (let polls = Math.ceil(within / every); polls > 0; polls--) { | |
| 11 | if (await done()) return true; | |
| 12 | await Bun.sleep(every); | |
| 7 | 13 | } |
| 14 | return await done(); | |
| 15 | } | |
| 16 | ||
| 17 | /** | |
| 18 | * Waits until `done` holds, polling; throws `never came: <what>` when it does not within | |
| 19 | * `timeoutMs`. | |
| 20 | */ | |
| 21 | export async function pollUntil(done: () => boolean | Promise<boolean>, what: string, timeoutMs = 30_000): Promise<void> { | |
| 22 | if (!(await waitFor(done, { within: timeoutMs }))) throw new Error(`never came: ${what}`); | |
| 8 | 23 | } |
test/media.test.ts +3 −2
| @@ -21,6 +21,7 @@ import { cachedMedia } from "../src/web/pages/tree_cache.ts"; | ||
| 21 | 21 | import { MEDIA_POLICY, PDF_POLICY } from "../src/web/pages/media.ts"; |
| 22 | 22 | import { seedAdmin } from "./helpers/admin.ts"; |
| 23 | 23 | import { quietLogs } from "./helpers/globals.ts"; |
| 24 | import { waitFor } from "./helpers/poll.ts"; | |
| 24 | 25 | |
| 25 | 26 | quietLogs(); |
| 26 | 27 | |
| @@ -204,7 +205,7 @@ describe("the media route", () => { | ||
| 204 | 205 | const reader = res.body!.getReader(); |
| 205 | 206 | expect((await reader.read()).done).toBe(false); |
| 206 | 207 | await reader.cancel(); |
| 207 | for (let i = 0; i < 200 && mediaGate.stats().active > 0; i++) await Bun.sleep(10); | |
| 208 | await waitFor(() => mediaGate.stats().active <= 0, { every: 10, within: 2_000 }); | |
| 208 | 209 | expect(mediaGate.stats().active).toBe(0); |
| 209 | 210 | }); |
| 210 | 211 | |
| @@ -283,7 +284,7 @@ async function holdAll(): Promise<(() => void)[]> { | ||
| 283 | 284 | } |
| 284 | 285 | |
| 285 | 286 | const settled = async (): Promise<number> => { |
| 286 | for (let i = 0; i < 300 && mediaGate.stats().active > 0; i++) await Bun.sleep(10); | |
| 287 | await waitFor(() => mediaGate.stats().active <= 0, { every: 10, within: 3_000 }); | |
| 287 | 288 | return mediaGate.stats().active; |
| 288 | 289 | }; |
| 289 | 290 |
test/merge-queue-batch.test.ts +4 −3
| @@ -14,6 +14,7 @@ import { join } from "node:path"; | ||
| 14 | 14 | import { batchWords, type Candidate, enqueue, type Expect, type Queued } from "../src/checks/queue.ts"; |
| 15 | 15 | import type { LandVerdict } from "../src/checks/tiers.ts"; |
| 16 | 16 | import type { Pull } from "../src/core/pulls.ts"; |
| 17 | import { waitFor } from "./helpers/poll.ts"; | |
| 17 | 18 | |
| 18 | 19 | describe("the merge queue's batch size", () => { |
| 19 | 20 | let dir: string; |
| @@ -118,7 +119,7 @@ describe("the merge queue's batch size", () => { | ||
| 118 | 119 | const seen: number[][][] = []; |
| 119 | 120 | const gate = (candidate: Candidate) => new Promise<LandVerdict>((go) => pending.push({ candidate, go })); |
| 120 | 121 | const letGo = async (n: number) => { |
| 121 | for (let i = 0; i < 2000 && pending.length < n; i++) await Bun.sleep(5); | |
| 122 | await waitFor(() => pending.length >= n, { every: 5, within: 10_000 }); | |
| 122 | 123 | expect(pending).toHaveLength(n); |
| 123 | 124 | const going = pending; |
| 124 | 125 | pending = []; |
| @@ -131,11 +132,11 @@ describe("the merge queue's batch size", () => { | ||
| 131 | 132 | await letGo(1); // the sixteen: red |
| 132 | 133 | await letGo(3); // round one: 4, 8 and 12 deep; the first four land, the red run is 5–8 |
| 133 | 134 | expect(seen[1]!.map((c) => c.length)).toEqual([4, 8, 12]); |
| 134 | for (let i = 0; i < 2000 && pending.length < 4; i++) await Bun.sleep(5); | |
| 135 | await waitFor(() => pending.length >= 4, { every: 5, within: 10_000 }); | |
| 135 | 136 | expect(landings).toBe(4); |
| 136 | 137 | await letGo(4); // round two: everything left but one of 5–8 each; the one without 6 lands |
| 137 | 138 | expect(seen[2]!.map((c) => c.length)).toEqual([11, 11, 11, 11]); |
| 138 | for (let i = 0; i < 2000 && landings < 15; i++) await Bun.sleep(5); | |
| 139 | await waitFor(() => landings >= 15, { every: 5, within: 10_000 }); | |
| 139 | 140 | expect(landings).toBe(15); |
| 140 | 141 | expect(git("ls-tree", "--name-only", "refs/heads/main").split("\n").filter((f) => /^r\d+\.txt$/.test(f))).toHaveLength(15); |
| 141 | 142 | await letGo(1); // the one left out, alone: red, handed back |
test/merge-queue-clash.test.ts +2 −1
| @@ -13,6 +13,7 @@ import { tmpdir } from "node:os"; | ||
| 13 | 13 | import { join } from "node:path"; |
| 14 | 14 | import { batchWords, type Candidate, enqueue, type Expect, queueClashes, type Queued, redBatchRate, redMark } from "../src/checks/queue.ts"; |
| 15 | 15 | import type { Pull } from "../src/core/pulls.ts"; |
| 16 | import { waitFor } from "./helpers/poll.ts"; | |
| 16 | 17 | |
| 17 | 18 | describe("the merge queue remembers clashes", () => { |
| 18 | 19 | let dir: string; |
| @@ -127,7 +128,7 @@ describe("the merge queue remembers clashes", () => { | ||
| 127 | 128 | }; |
| 128 | 129 | const options = { depth: 2, room: () => 4 }; |
| 129 | 130 | const first = enqueue(repoId, [b], holding, options); |
| 130 | for (let i = 0; i < 2000 && open < 1; i++) await Bun.sleep(5); | |
| 131 | await waitFor(() => open >= 1, { every: 5, within: 10_000 }); | |
| 131 | 132 | if (open < 1) throw new Error(`never came: still waiting while open < 1`); |
| 132 | 133 | const f = entry(6, await headAdding("f", "f.txt")); |
| 133 | 134 | const second = enqueue(repoId, [f], holding, options); |
test/merge-queue-hold.test.ts +2 −1
| @@ -20,6 +20,7 @@ import { getRepoById, type Repo } from "../src/core/repos.ts"; | ||
| 20 | 20 | import { claimRun, createRun, finishRun } from "../src/core/runs.ts"; |
| 21 | 21 | import { createUser, type User } from "../src/core/users.ts"; |
| 22 | 22 | import { landIfGreen, pruneLine } from "../src/crew/autonomy.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | describe("a hold and the merge queue's line", () => { |
| 25 | 26 | let dataDir: string; |
| @@ -107,7 +108,7 @@ describe("a hold and the merge queue's line", () => { | ||
| 107 | 108 | } |
| 108 | 109 | |
| 109 | 110 | async function gating(n: number): Promise<void> { |
| 110 | for (let i = 0; i < 2000 && running.size < n; i++) await Bun.sleep(5); | |
| 111 | await waitFor(() => running.size >= n, { every: 5, within: 10_000 }); | |
| 111 | 112 | expect(running.size).toBe(n); |
| 112 | 113 | } |
| 113 | 114 |
test/merge-queue-parallel.test.ts +5 −4
| @@ -27,6 +27,7 @@ import { readQueues } from "../src/flow/queue.ts"; | ||
| 27 | 27 | import { repoDir } from "../src/git/repos.ts"; |
| 28 | 28 | import { flowLines } from "../src/tui/screens/flow.ts"; |
| 29 | 29 | import { serve, type Servers } from "../src/http/server.ts"; |
| 30 | import { waitFor } from "./helpers/poll.ts"; | |
| 30 | 31 | |
| 31 | 32 | /** Check slots the stand-in service has. */ |
| 32 | 33 | const CAPACITY = 12; |
| @@ -121,7 +122,7 @@ describe("the merge queue in parallel", () => { | ||
| 121 | 122 | |
| 122 | 123 | /** Wait until `n` gates are open at once, then let them all go: one wave. */ |
| 123 | 124 | async function letGo(n: number): Promise<void> { |
| 124 | for (let i = 0; i < 2000 && pending.length < n; i++) await Bun.sleep(5); | |
| 125 | await waitFor(() => pending.length >= n, { every: 5, within: 10_000 }); | |
| 125 | 126 | expect(pending).toHaveLength(n); |
| 126 | 127 | wave++; |
| 127 | 128 | const going = pending; |
| @@ -137,7 +138,7 @@ describe("the merge queue in parallel", () => { | ||
| 137 | 138 | const pulls = []; |
| 138 | 139 | for (let i = 0; i < 16; i++) pulls.push(await greenPull(repo, `p${i}.ts`)); |
| 139 | 140 | const sweep = sweepGreen(app, repo); |
| 140 | for (let i = 0; i < 2000 && pending.length < 2; i++) await Bun.sleep(5); | |
| 141 | await waitFor(() => pending.length >= 2, { every: 5, within: 10_000 }); | |
| 141 | 142 | expect(queueStatus().find((s) => s.repoId === repo.id)).toMatchObject({ gating: 16, gates: 2, queued: 0 }); |
| 142 | 143 | expect(queueWords(repo.id)).toMatch(/^gating 16 pulls since \d\d:\d\d in 2 gates$/); |
| 143 | 144 | await letGo(2); |
| @@ -291,11 +292,11 @@ describe("the queue's extra gates and the pull checks' slots", () => { | ||
| 291 | 292 | const repoId = `slots-${concurrency}`; |
| 292 | 293 | const [first, second] = [await entry(1), await entry(2)]; |
| 293 | 294 | const one = enqueue(repoId, [first], gate, options); |
| 294 | for (let i = 0; i < 2000 && open < 1; i++) await Bun.sleep(5); | |
| 295 | await waitFor(() => open >= 1, { every: 5, within: 10_000 }); | |
| 295 | 296 | if (open < 1) throw new Error(`never came: still waiting while open < 1`); |
| 296 | 297 | const two = enqueue(repoId, [second], gate, options); |
| 297 | 298 | // Either it gates beside the first, or it still waits in line for it. |
| 298 | for (let i = 0; i < 100 && open < 2; i++) await Bun.sleep(5); | |
| 299 | await waitFor(() => open >= 2, { every: 5, within: 500 }); | |
| 299 | 300 | if (open < 2) expect(queueStatus().find((s) => s.repoId === repoId)?.queued).toBe(1); |
| 300 | 301 | release(); |
| 301 | 302 | expect([...(await one), ...(await two)]).toEqual([false, false]); |
test/merge-queue-restart.test.ts +4 −3
| @@ -24,6 +24,7 @@ import { claimRun, createRun, finishRun } from "../src/core/runs.ts"; | ||
| 24 | 24 | import { createUser, type User } from "../src/core/users.ts"; |
| 25 | 25 | import { sweepGreen } from "../src/crew/autonomy.ts"; |
| 26 | 26 | import { repoDir } from "../src/git/repos.ts"; |
| 27 | import { waitFor } from "./helpers/poll.ts"; | |
| 27 | 28 | |
| 28 | 29 | describe("the merge queue across a restart", () => { |
| 29 | 30 | let dataDir: string; |
| @@ -115,7 +116,7 @@ describe("the merge queue across a restart", () => { | ||
| 115 | 116 | } |
| 116 | 117 | |
| 117 | 118 | async function gating(n: number): Promise<void> { |
| 118 | for (let i = 0; i < 2000 && running.size < n; i++) await Bun.sleep(5); | |
| 119 | await waitFor(() => running.size >= n, { every: 5, within: 10_000 }); | |
| 119 | 120 | expect(running.size).toBe(n); |
| 120 | 121 | } |
| 121 | 122 | |
| @@ -164,7 +165,7 @@ describe("the merge queue across a restart", () => { | ||
| 164 | 165 | const { repo, pulls, tip, record } = await killedMidGate("restart-verdict"); |
| 165 | 166 | // The stopped forge's gate answers on its drain: kept in the record. |
| 166 | 167 | running.get(tip)!.go({ ok: true, error: "", lines: [] }); |
| 167 | for (let i = 0; i < 200 && gateRecords(app.db, repo.id)[0]?.verdict === undefined; i++) await Bun.sleep(5); | |
| 168 | await waitFor(() => gateRecords(app.db, repo.id)[0]?.verdict !== undefined, { every: 5, within: 1_000 }); | |
| 168 | 169 | expect(gateRecords(app.db, repo.id)[0]?.verdict).toEqual({ lands: true }); |
| 169 | 170 | lost = true; |
| 170 | 171 | expect(await sweepGreen(app, repo)).toBe(3); |
| @@ -194,7 +195,7 @@ describe("the merge queue across a restart", () => { | ||
| 194 | 195 | const { repo, pulls, tip, record } = await killedMidGate("restart-red-check"); |
| 195 | 196 | // The roll's drain answers red with no test named: kept, not acted on. |
| 196 | 197 | running.get(tip)!.go({ ok: false, error: "check exited with code 1", lines: [], stage: "check" }); |
| 197 | for (let i = 0; i < 200 && gateRecords(app.db, repo.id)[0]?.verdict === undefined; i++) await Bun.sleep(5); | |
| 198 | await waitFor(() => gateRecords(app.db, repo.id)[0]?.verdict !== undefined, { every: 5, within: 1_000 }); | |
| 198 | 199 | expect(gateRecords(app.db, repo.id)[0]?.verdict).toMatchObject({ lands: false, reason: { kind: "check" } }); |
| 199 | 200 | lost = true; |
| 200 | 201 | const again = sweepGreen(app, repo); |
test/merge-queue-suspect.test.ts +2 −1
| @@ -12,6 +12,7 @@ import { join } from "node:path"; | ||
| 12 | 12 | import { type Candidate, enqueue, type Expect, type Queued } from "../src/checks/queue.ts"; |
| 13 | 13 | import type { LandVerdict } from "../src/checks/tiers.ts"; |
| 14 | 14 | import type { Pull } from "../src/core/pulls.ts"; |
| 15 | import { waitFor } from "./helpers/poll.ts"; | |
| 15 | 16 | |
| 16 | 17 | describe("the merge queue judges a suspect alone", () => { |
| 17 | 18 | let dir: string; |
| @@ -75,7 +76,7 @@ describe("the merge queue judges a suspect alone", () => { | ||
| 75 | 76 | const gate = (candidate: Candidate) => |
| 76 | 77 | new Promise<LandVerdict>((go) => pending.push({ candidate, go })); |
| 77 | 78 | const letGo = async (n: number) => { |
| 78 | for (let i = 0; i < 2000 && pending.length < n; i++) await Bun.sleep(5); | |
| 79 | await waitFor(() => pending.length >= n, { every: 5, within: 10_000 }); | |
| 79 | 80 | expect(pending).toHaveLength(n); |
| 80 | 81 | const going = pending; |
| 81 | 82 | pending = []; |
test/orphan-dispatch.test.ts +2 −1
| @@ -5,6 +5,7 @@ import { tmpdir } from "node:os"; | ||
| 5 | 5 | import { join } from "node:path"; |
| 6 | 6 | import { createApp } from "../src/app.ts"; |
| 7 | 7 | import { startCrew } from "../src/crew/dispatch.ts"; |
| 8 | import { waitFor } from "./helpers/poll.ts"; | |
| 8 | 9 | |
| 9 | 10 | describe("the crew sweeps orphan clones", () => { |
| 10 | 11 | test("at start it removes a leftover, and stop() clears the hourly timer", async () => { |
| @@ -38,7 +39,7 @@ describe("the crew sweeps orphan clones", () => { | ||
| 38 | 39 | networkRetryScale: 0, |
| 39 | 40 | reviewBudgetMs: 1000, |
| 40 | 41 | }); |
| 41 | for (let i = 0; i < 100 && existsSync(orphan); i++) await Bun.sleep(20); | |
| 42 | await waitFor(() => !existsSync(orphan), { every: 20, within: 2_000 }); | |
| 42 | 43 | expect(existsSync(orphan)).toBe(false); |
| 43 | 44 | expect(hourly.length).toBe(1); |
| 44 | 45 | await crew.stop(); |
test/packs-up.test.ts +2 −1
| @@ -21,6 +21,7 @@ import { deployCommands } from "../src/packs/words.ts"; | ||
| 21 | 21 | import { repoDir } from "../src/git/repos.ts"; |
| 22 | 22 | import { realHands, up } from "../scripts/pack.ts"; |
| 23 | 23 | import { quietLogs } from "./helpers/globals.ts"; |
| 24 | import { waitFor } from "./helpers/poll.ts"; | |
| 24 | 25 | |
| 25 | 26 | quietLogs(); |
| 26 | 27 | |
| @@ -239,7 +240,7 @@ describe("up() itself", () => { | ||
| 239 | 240 | const running = up(pack, { port, dataDir: d, cloneFrom: (s) => fake.get(s)!, guard: PUBLIC, say: (l) => lines.push(l), signal: stop.signal, env }); |
| 240 | 241 | let ended: number | undefined; |
| 241 | 242 | void running.then((c) => (ended = c)); |
| 242 | for (let i = 0; i < 700 && ended === undefined && !lines.some((l) => l.startsWith("Your forge is serving")); i++) await Bun.sleep(100); | |
| 243 | await waitFor(() => !(ended === undefined && !lines.some((l) => l.startsWith("Your forge is serving"))), { every: 100, within: 70_000 }); | |
| 243 | 244 | if (ended !== undefined) throw new Error(`up() ended with ${ended} before serving: ${lines.join(" | ")}`); |
| 244 | 245 | const said = lines.join("\n"); |
| 245 | 246 | expect(said).toContain(`Your forge is serving http://localhost:${port}/`); |
test/perf-server-hook.test.ts +2 −1
| @@ -7,6 +7,7 @@ import { serve, type Servers } from "../src/http/server.ts"; | ||
| 7 | 7 | import { watchStalls } from "../src/kernel/loop_lag.ts"; |
| 8 | 8 | import { liveOf } from "../src/perf/live.ts"; |
| 9 | 9 | import { quietLogs } from "./helpers/globals.ts"; |
| 10 | import { waitFor } from "./helpers/poll.ts"; | |
| 10 | 11 | |
| 11 | 12 | quietLogs(); |
| 12 | 13 | |
| @@ -39,7 +40,7 @@ test("watchStalls hands every wake's lateness to onStall", async () => { | ||
| 39 | 40 | const quiet = { debug() {}, info() {}, warn() {}, error() {} }; |
| 40 | 41 | const stop = watchStalls(quiet, 1e9, 10, undefined, (late) => void seen.push(late)); |
| 41 | 42 | // Wait for the condition (a wake), not for a fixed 80 ms a loaded shard can overrun. |
| 42 | for (let i = 0; i < 3000 && seen.length === 0; i++) await Bun.sleep(10); // real-timer: polls a real watchdog interval | |
| 43 | await waitFor(() => seen.length !== 0, { every: 10, within: 30_000 }); // real-timer: polls a real watchdog interval | |
| 43 | 44 | stop(); |
| 44 | 45 | expect(seen.length).toBeGreaterThan(0); |
| 45 | 46 | }); |
test/roll-write-pause.test.ts +3 −2
| @@ -11,6 +11,7 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises"; | ||
| 11 | 11 | import { tmpdir } from "node:os"; |
| 12 | 12 | import { join, resolve } from "node:path"; |
| 13 | 13 | import { createApp } from "../src/app.ts"; |
| 14 | import { waitFor } from "./helpers/poll.ts"; | |
| 14 | 15 | |
| 15 | 16 | const SRC = resolve(import.meta.dir, "../src"); |
| 16 | 17 | |
| @@ -88,12 +89,12 @@ test("the new forge starts its queue work within 30 s of SIGTERM to the old, wit | ||
| 88 | 89 | }; |
| 89 | 90 | void collect(forge.stdout as ReadableStream<Uint8Array>); |
| 90 | 91 | void collect(forge.stderr as ReadableStream<Uint8Array>); |
| 91 | for (let i = 0; i < 300 && !out.includes("another forge still works this database"); i++) await Bun.sleep(100); | |
| 92 | await waitFor(() => out.includes("another forge still works this database"), { every: 100, within: 30_000 }); | |
| 92 | 93 | expect(out).toContain("another forge still works this database"); |
| 93 | 94 | |
| 94 | 95 | const signalled = Date.now(); |
| 95 | 96 | old.kill("SIGTERM"); |
| 96 | for (let i = 0; i < 300 && !out.includes("the other forge let go of this database"); i++) await Bun.sleep(100); | |
| 97 | await waitFor(() => out.includes("the other forge let go of this database"), { every: 100, within: 30_000 }); | |
| 97 | 98 | if (!out.includes("the other forge let go of this database")) throw new Error(`never came: still waiting while !out.includes("the other forge let go of this database")`); |
| 98 | 99 | const pause = Date.now() - signalled; |
| 99 | 100 | if (pause >= 30_000) console.log(out); |
test/snapshot-restore-streamed.test.ts +2 −1
| @@ -14,6 +14,7 @@ import { partPath, unpacker, type PackedFile } from "../src/snapshot/parts.ts"; | ||
| 14 | 14 | import { pack } from "./helpers/snapshot-fixtures.ts"; |
| 15 | 15 | import { restoreSnapshot } from "../src/snapshot/restore.ts"; |
| 16 | 16 | import { dirStore, type Store } from "../src/snapshot/store.ts"; |
| 17 | import { waitFor } from "./helpers/poll.ts"; | |
| 17 | 18 | |
| 18 | 19 | const NAME = "2026-10-19T08-00-00Z"; |
| 19 | 20 | const prefix = `snapshots/${NAME}/`; |
| @@ -78,7 +79,7 @@ describe("a packed file unpacks as its parts arrive", () => { | ||
| 78 | 79 | peak = Math.max(peak, diskBytes(dataDir)); |
| 79 | 80 | if (key !== last) return inner.get(key); |
| 80 | 81 | // Everything else has landed; wait for the file to show it was unpacked on arrival. |
| 81 | for (let i = 0; i < 100 && diskBytes(dataDir) === 0; i++) await Bun.sleep(20); | |
| 82 | await waitFor(() => diskBytes(dataDir) !== 0, { every: 20, within: 2_000 }); | |
| 82 | 83 | if (diskBytes(dataDir) === 0) throw new Error(`never came: still waiting while diskBytes(dataDir) === 0`); |
| 83 | 84 | grewBeforeLast = diskBytes(dataDir); |
| 84 | 85 | await gate; |
test/spawn-cell-socket.test.ts +2 −1
| @@ -12,6 +12,7 @@ import { createApp, type App } from "../src/app.ts"; | ||
| 12 | 12 | import { serve, type Servers } from "../src/http/server.ts"; |
| 13 | 13 | import { cellChildren } from "../src/spawn/cell.ts"; |
| 14 | 14 | import { quietLogs } from "./helpers/globals.ts"; |
| 15 | import { waitFor } from "./helpers/poll.ts"; | |
| 15 | 16 | |
| 16 | 17 | quietLogs(); |
| 17 | 18 | |
| @@ -68,7 +69,7 @@ const open = (host: string, path = "/ws?x=1", protocols: string[] = ["chat"]) => | ||
| 68 | 69 | }); |
| 69 | 70 | |
| 70 | 71 | const until = async (ok: () => boolean) => { |
| 71 | for (let i = 0; i < 100 && !ok(); i++) await Bun.sleep(20); | |
| 72 | await waitFor(() => ok(), { every: 20, within: 2_000 }); | |
| 72 | 73 | if (!ok()) throw new Error(`never came: still waiting while !ok()`); |
| 73 | 74 | }; |
| 74 | 75 |
test/spawn-cell.test.ts +3 −2
| @@ -21,6 +21,7 @@ import { CAPACITY, childModelHosts, childPaymentHosts, childRegistryHosts, | ||
| 21 | 21 | childOriginHosts, setCapacity } from "../src/core/capacity.ts"; |
| 22 | 22 | import { openBeatDoor, parentLocal } from "../src/spawn/beat-door.ts"; |
| 23 | 23 | import { sendBeat } from "../src/spawn/heartbeat.ts"; |
| 24 | import { waitFor } from "./helpers/poll.ts"; | |
| 24 | 25 | |
| 25 | 26 | const plan = (name: string): SpawnPlan => ({ name, org: "o", apps: [], briefs: [], subscriptions: [], direction: "", keepAccess: false, budgetUsd: null }); |
| 26 | 27 | const tmp = mkdtempSync(join(tmpdir(), "spawn-cell-")); |
| @@ -332,7 +333,7 @@ describe("a child forge's crew inside its cell", () => { | ||
| 332 | 333 | expect(await sleepIdleChildren(parent, 0)).toEqual(["kid"]); |
| 333 | 334 | await Bun.sleep(Math.max(0, at - Date.now() - 300)); |
| 334 | 335 | expect(wokeAt).toBeUndefined(); |
| 335 | for (let i = 0; i < 100 && wokeAt === undefined; i++) await Bun.sleep(50); | |
| 336 | await waitFor(() => wokeAt !== undefined, { within: 5_000 }); | |
| 336 | 337 | expect(wokeAt).toBeGreaterThanOrEqual(at); |
| 337 | 338 | expect(crewAtWork(dir)).toBe(true); |
| 338 | 339 | } finally { |
| @@ -593,7 +594,7 @@ describe("a child whose cell will not come up", () => { | ||
| 593 | 594 | expect(first.status).toBe(503); |
| 594 | 595 | expect(first.headers.get("retry-after")).toBe("5"); |
| 595 | 596 | // The start it set off fails here (no cells), and the ledger turns hosted. |
| 596 | for (let i = 0; i < 50 && JSON.parse(readFileSync(file, "utf8")).placement !== "hosted"; i++) await Bun.sleep(10); | |
| 597 | await waitFor(() => JSON.parse(readFileSync(file, "utf8")).placement === "hosted", { every: 10, within: 500 }); | |
| 597 | 598 | expect(JSON.parse(readFileSync(file, "utf8")).placement).toBe("hosted"); |
| 598 | 599 | const next = await resolveHostedChild(parent, request()); |
| 599 | 600 | expect(next).not.toBe("cell"); |
test/spawn-reach-door.test.ts +2 −1
| @@ -20,6 +20,7 @@ import { usePlacement, useProbe } from "../src/spawn/door.ts"; | ||
| 20 | 20 | import { childAwake } from "../src/spawn/host.ts"; |
| 21 | 21 | import { reachAlarms, reachOf, useReach } from "../src/spawn/reach.ts"; |
| 22 | 22 | import { quietLogs } from "./helpers/globals.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | quietLogs(); |
| 25 | 26 | |
| @@ -79,7 +80,7 @@ test("the spawn result lists each address with its result; a standby page is a f | ||
| 79 | 80 | { label: "notes", url: "https://notes--kid--forge.test/", ok: false, status: 200, says: "answered 200, not by the app", seen: { server: "", via: "", "x-forge-app": "" } }, |
| 80 | 81 | ]); |
| 81 | 82 | // The never-answering app is the alarm, once its time to come up is over. |
| 82 | for (let i = 0; i < 100 && reachAlarms(app).length === 0; i++) await Bun.sleep(20); | |
| 83 | await waitFor(() => reachAlarms(app).length !== 0, { every: 20, within: 2_000 }); | |
| 83 | 84 | expect(reachAlarms(app).map((a) => a.key)).toEqual(["unreachable"]); |
| 84 | 85 | expect(listEvents(app.db, { kind: "forge.child_unreachable", limit: 5 })).toHaveLength(1); |
| 85 | 86 | expect(reachOf(app, "kid")?.alarmed).toBe(true); |
test/spawn-stopped-restart.test.ts +4 −3
| @@ -22,6 +22,7 @@ import { cellChildren, type CellChild } from "../src/spawn/cell.ts"; | ||
| 22 | 22 | import { closeHostedChildren, sleepIdleChildren } from "../src/spawn/host.ts"; |
| 23 | 23 | import { backoffMs, CLEAN_MS, cancelRestarts, lastStop, markStopped, noteStop, RESTART_MAX_MS, stopSentence } from "../src/spawn/stopped.ts"; |
| 24 | 24 | import { quietLogs } from "./helpers/globals.ts"; |
| 25 | import { waitFor } from "./helpers/poll.ts"; | |
| 25 | 26 | |
| 26 | 27 | quietLogs(); |
| 27 | 28 | |
| @@ -85,7 +86,7 @@ test("a stopped forge is started again at once when the ceiling is full of forge | ||
| 85 | 86 | try { |
| 86 | 87 | // The first start is a due run's, which is also what tells the host how to restart a stopped forge. |
| 87 | 88 | await sleepIdleChildren(parent, 0); |
| 88 | for (let i = 0; i < 100 && starts.length === 0; i++) await Bun.sleep(50); | |
| 89 | await waitFor(() => starts.length !== 0, { within: 5_000 }); | |
| 89 | 90 | expect(starts.length).toBeGreaterThan(0); |
| 90 | 91 | expect(starts).toEqual(["kid"]); |
| 91 | 92 | // It dies of memory while another forge, quiet and with no crew at work, holds the only slot. |
| @@ -93,7 +94,7 @@ test("a stopped forge is started again at once when the ceiling is full of forge | ||
| 93 | 94 | awake.add("cell-idle"); |
| 94 | 95 | mode = "evict"; |
| 95 | 96 | noteStop(parent, "cell-kid", "kid", "used 3000 MiB of memory, over its 2048 MiB limit", Date.now(), 10); |
| 96 | for (let i = 0; i < 100 && starts.length < 2; i++) await Bun.sleep(20); | |
| 97 | await waitFor(() => starts.length >= 2, { every: 20, within: 2_000 }); | |
| 97 | 98 | expect(starts).toEqual(["kid", "kid"]); |
| 98 | 99 | expect(awake.has("cell-idle")).toBe(false); |
| 99 | 100 | } finally { |
| @@ -129,7 +130,7 @@ test("a restart turned away for room says the time it will try again", async () | ||
| 129 | 130 | cells.sleepIdle = async () => (arm ? ["kid"] : []); |
| 130 | 131 | try { |
| 131 | 132 | await sleepIdleChildren(parent, 0); |
| 132 | for (let i = 0; i < 100 && starts.length === 0; i++) await Bun.sleep(50); | |
| 133 | await waitFor(() => starts.length !== 0, { within: 5_000 }); | |
| 133 | 134 | expect(starts.length).toBeGreaterThan(0); |
| 134 | 135 | awake.delete("cell-kid"); |
| 135 | 136 | awake.add("cell-busy"); |
test/ssh-shell-cold.test.ts +2 −1
| @@ -14,6 +14,7 @@ import { forgeSshSession } from "../src/tui/ssh.ts"; | ||
| 14 | 14 | import type { createForgeApp as CreateForgeApp } from "../src/tui/forge.ts"; |
| 15 | 15 | import type { SshSession } from "../src/ssh/server.ts"; |
| 16 | 16 | import { quietLogs } from "./helpers/globals.ts"; |
| 17 | import { waitFor } from "./helpers/poll.ts"; | |
| 17 | 18 | |
| 18 | 19 | quietLogs(); |
| 19 | 20 | |
| @@ -114,7 +115,7 @@ test("a failing import ends the session with exit 1 and a log", async () => { | ||
| 114 | 115 | test("the real import is reached on a shell: the terminal UI draws its first frame", async () => { |
| 115 | 116 | const { session, seen } = shell(); |
| 116 | 117 | forgeSshSession(app)(session); |
| 117 | for (let i = 0; i < 100 && seen.out === ""; i++) await Bun.sleep(50); // real-timer: the real dynamic import has no hook to await | |
| 118 | await waitFor(() => seen.out !== "", { within: 5_000 }); // real-timer: the real dynamic import has no hook to await | |
| 118 | 119 | expect(seen.out).not.toBe(""); |
| 119 | 120 | expect(seen.code).toBeUndefined(); |
| 120 | 121 | }); |
test/stranger-pull.test.ts +2 −1
| @@ -43,6 +43,7 @@ import { settleApp } from "./helpers/tui.ts"; | ||
| 43 | 43 | import { setLimit } from "../src/settings/limits.ts"; |
| 44 | 44 | import { quietLogs } from "./helpers/globals.ts"; |
| 45 | 45 | import { seatAs } from "./helpers/seat.ts"; |
| 46 | import { waitFor } from "./helpers/poll.ts"; | |
| 46 | 47 | |
| 47 | 48 | quietLogs(); |
| 48 | 49 | |
| @@ -195,7 +196,7 @@ describe("a stranger's pull", () => { | ||
| 195 | 196 | const opened = listEvents(app.db, { repoId: auto.id, kind: "pull.opened" }).find((e) => e.payload["number"] === pull.number); |
| 196 | 197 | expect(opened).toBeDefined(); |
| 197 | 198 | route(app, opened!); |
| 198 | for (let i = 0; i < 200 && reviewRuns(pull) === 0; i++) await Bun.sleep(10); | |
| 199 | await waitFor(() => reviewRuns(pull) !== 0, { every: 10, within: 2_000 }); | |
| 199 | 200 | expect(reviewRuns(pull)).toBe(1); |
| 200 | 201 | finishReviews(); |
| 201 | 202 | }); |
test/talk-page.test.ts +3 −2
| @@ -29,6 +29,7 @@ import { createUser, type User } from "../src/core/users.ts"; | ||
| 29 | 29 | import { serve, type Servers } from "../src/http/server.ts"; |
| 30 | 30 | import { promote } from "./helpers/members.ts"; |
| 31 | 31 | import { quietLogs } from "./helpers/globals.ts"; |
| 32 | import { waitFor } from "./helpers/poll.ts"; | |
| 32 | 33 | |
| 33 | 34 | quietLogs(); |
| 34 | 35 | |
| @@ -1164,7 +1165,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"Heard.","t | ||
| 1164 | 1165 | expect((await post("/talk", who.value, { text: STARTER_PROMPTS[1] }, { origin: "https://elsewhere.example" })).status).toBe(403); |
| 1165 | 1166 | expect(existsSync(join(dir, "first.json"))).toBe(false); |
| 1166 | 1167 | expect((await post("/talk", who.value, { text: STARTER_PROMPTS[1] })).status).toBe(303); |
| 1167 | for (let i = 0; i < 400 && !existsSync(join(dir, "first.json")); i++) await Bun.sleep(50); // real-timer: the agent runs in a child process behind the HTTP post; no handle to await | |
| 1168 | await waitFor(() => existsSync(join(dir, "first.json")), { within: 20_000 }); // real-timer: the agent runs in a child process behind the HTTP post; no handle to await | |
| 1168 | 1169 | expect(existsSync(join(dir, "first.json"))).toBe(true); |
| 1169 | 1170 | expect(input()).toContain(STARTER_PROMPTS[1]!); |
| 1170 | 1171 | const turns = listTurns(app.db, who.value.id, c); |
| @@ -1211,7 +1212,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"Heard.","t | ||
| 1211 | 1212 | const session = { write() {}, writeStderr() {}, exit() {}, onData() {}, onEof() {}, onClose() {} }; |
| 1212 | 1213 | fresh(alice); // the latest conversation is the one the terminal speaks in |
| 1213 | 1214 | serveTalk(app, session as never, alice, `--about alice/ctx-tty#${issue.value.number} why?`); |
| 1214 | for (let i = 0; i < 100 && !existsSync(join(dir, "first.json")); i++) await Bun.sleep(50); | |
| 1215 | await waitFor(() => existsSync(join(dir, "first.json")), { within: 5_000 }); | |
| 1215 | 1216 | expect(input()).toContain("title: Tty subject"); |
| 1216 | 1217 | expect(input()).toContain("why?"); |
| 1217 | 1218 | }); |
test/talk-stop.test.ts +3 −2
| @@ -19,6 +19,7 @@ import { partialCostUsd } from "../src/crew/claude.ts"; | ||
| 19 | 19 | import { resumeInterrupted, say, STOPPED_LINE, stopTurn } from "../src/crew/conversation.ts"; |
| 20 | 20 | import { lastTurnWords } from "../src/tui/ssh-talk.ts"; |
| 21 | 21 | import { turnHtml } from "../src/web/pages/talk.ts"; |
| 22 | import { waitFor } from "./helpers/poll.ts"; | |
| 22 | 23 | |
| 23 | 24 | let dir: string; |
| 24 | 25 | let app: App; |
| @@ -92,7 +93,7 @@ test("a stopped answer keeps its text, says stopped with dollars, and is recorde | ||
| 92 | 93 | const l = listTurns(app.db, user.id, conversation.id, {}); |
| 93 | 94 | return l.ok ? l.value.map((t) => t.text) : []; |
| 94 | 95 | }; |
| 95 | for (let i = 0; i < 100 && !texts().includes("Half an answer."); i++) await Bun.sleep(50); // real-timer: polls a child process the test cannot step | |
| 96 | await waitFor(() => texts().includes("Half an answer."), { within: 5_000 }); // real-timer: polls a child process the test cannot step | |
| 96 | 97 | expect(texts()).toContain("Half an answer."); |
| 97 | 98 | expect(stopTurn(app, user.id)).toBe(true); |
| 98 | 99 | await said.value.done; |
| @@ -133,7 +134,7 @@ test("a turn cut by a restart is charged its partial spend once; the resumed tur | ||
| 133 | 134 | return l.ok ? l.value.filter((t) => t.text === "Half an answer.").length : 0; |
| 134 | 135 | }; |
| 135 | 136 | const halvesBefore = halves(); |
| 136 | for (let i = 0; i < 100 && halves() === halvesBefore; i++) await Bun.sleep(50); // real-timer: polls a child process the test cannot step | |
| 137 | await waitFor(() => halves() !== halvesBefore, { within: 5_000 }); // real-timer: polls a child process the test cannot step | |
| 137 | 138 | restart.abort(); // the forge stops: the row stays running, charged what it streamed |
| 138 | 139 | await said.value.done; |
| 139 | 140 | expect(runs()).toEqual([{ cost_usd: expect.closeTo(0.006, 9) as unknown as number, state: "running", resumes: 0 }]); |
test/thanks.test.ts +2 −1
| @@ -34,6 +34,7 @@ import { thankAuthor, thanksNote, whomToThank } from "../src/origins/thanks.ts"; | ||
| 34 | 34 | import { staticCredential } from "../src/origins/types.ts"; |
| 35 | 35 | import { newFakeState, serveFakeForge, type FakeForge, type FakeState } from "./fake-origin.ts"; |
| 36 | 36 | import { quietLogs } from "./helpers/globals.ts"; |
| 37 | import { waitFor } from "./helpers/poll.ts"; | |
| 37 | 38 | |
| 38 | 39 | quietLogs(); |
| 39 | 40 | |
| @@ -181,7 +182,7 @@ describe("a landed mirror pull thanks its author once", () => { | ||
| 181 | 182 | |
| 182 | 183 | // The mirror's copy goes out after the merge answers. |
| 183 | 184 | const onMirror = () => (state.comments.get("trevato-apps/forge") ?? []).filter((c) => c.body.startsWith("Thank you")); |
| 184 | for (let i = 0; i < 50 && onMirror().length === 0; i++) await Bun.sleep(20); | |
| 185 | await waitFor(() => onMirror().length !== 0, { every: 20, within: 1_000 }); | |
| 185 | 186 | expect(onMirror()).toHaveLength(1); |
| 186 | 187 | expect(onMirror()[0]!.issueNumber).toBe(1); |
| 187 | 188 | expect(onMirror()[0]!.body).toBe(here[0]!); |
test/timers-sync.test.ts +2 −1
| @@ -20,6 +20,7 @@ import { getUserByName } from "../src/core/users.ts"; | ||
| 20 | 20 | import { scheduleTimers, syncedCommit } from "../src/core/timers.ts"; |
| 21 | 21 | import { serve, type Servers } from "../src/http/server.ts"; |
| 22 | 22 | import { seedAdmin } from "./helpers/admin.ts"; |
| 23 | import { waitFor } from "./helpers/poll.ts"; | |
| 23 | 24 | |
| 24 | 25 | let dataDir: string; |
| 25 | 26 | let app: App; |
| @@ -56,7 +57,7 @@ test("the schedules a production commit declares become rows once per commit; a | ||
| 56 | 57 | if (!committed.ok) throw new Error(JSON.stringify(committed.error)); |
| 57 | 58 | const tip = (await resolveOid(repoDir(app.reposRoot, "alice", "paper"), `refs/heads/${repo.defaultBranch}`))!; |
| 58 | 59 | const production = () => runtime.status(record).find((s) => s.env === "production"); |
| 59 | for (let i = 0; i < 100 && production()?.commitOid !== tip; i++) await Bun.sleep(100); | |
| 60 | await waitFor(() => production()?.commitOid === tip, { every: 100, within: 10_000 }); | |
| 60 | 61 | expect(production()?.commitOid).toBe(tip); |
| 61 | 62 | await syncAll(app); |
| 62 | 63 | const rows = scheduleTimers(app.db, record.id); |
test/tui-forks-take.test.ts +2 −1
| @@ -15,6 +15,7 @@ import { forksScreen, takeOffer } from "../src/tui/screens/forks.ts"; | ||
| 15 | 15 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; |
| 16 | 16 | import { settleApp } from "./helpers/tui.ts"; |
| 17 | 17 | import { promote } from "./helpers/members.ts"; |
| 18 | import { waitFor } from "./helpers/poll.ts"; | |
| 18 | 19 | |
| 19 | 20 | const must = <T>(r: { ok: true; value: T } | { ok: false; error: unknown }): T => { |
| 20 | 21 | if (!r.ok) throw new Error(JSON.stringify(r.error)); |
| @@ -114,7 +115,7 @@ describe("the terminal's forks screen takes the parent's updates with t", () => | ||
| 114 | 115 | }), |
| 115 | 116 | ); |
| 116 | 117 | // Bob's fork follows by itself; put it back where it was so it is behind only. |
| 117 | for (let i = 0; i < 100 && (await behind("bob")) !== 0; i++) await Bun.sleep(50); | |
| 118 | await waitFor(async () => (await behind("bob")) === 0, { within: 5_000 }); | |
| 118 | 119 | if ((await behind("bob")) !== 0) throw new Error(`never came: still waiting while (await behind("bob")) !== 0`); |
| 119 | 120 | await git(repoDir(app.reposRoot, "bob", "app"), "update-ref", "refs/heads/main", before); |
| 120 | 121 | expect(await behind("bob")).toBe(1); |
test/tui-origins-forge.test.ts +2 −1
| @@ -15,6 +15,7 @@ import type { TuiIdentity } from "../src/tui/session.ts"; | ||
| 15 | 15 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; |
| 16 | 16 | import { quietLogs } from "./helpers/globals.ts"; |
| 17 | 17 | import { settleApp } from "./helpers/tui.ts"; |
| 18 | import { waitFor } from "./helpers/poll.ts"; | |
| 18 | 19 | |
| 19 | 20 | quietLogs(); |
| 20 | 21 | |
| @@ -88,7 +89,7 @@ test("c connects another forge by address and token, and the list says what did | ||
| 88 | 89 | await key("enter"); |
| 89 | 90 | expect(app.screen.toText()).toContain("Read-only?"); |
| 90 | 91 | await key("enter"); |
| 91 | for (let i = 0; i < 50 && listOrigins(mine.db, getUserByName(mine.db, "alice")!.id).length === 0; i++) await Bun.sleep(100); | |
| 92 | await waitFor(() => listOrigins(mine.db, getUserByName(mine.db, "alice")!.id).length !== 0, { every: 100, within: 5_000 }); | |
| 92 | 93 | if (listOrigins(mine.db, getUserByName(mine.db, "alice")!.id).length === 0) throw new Error(`never came: still waiting while listOrigins(mine.db, getUserByName(mine.db, "alice")!.id).length === 0`); |
| 93 | 94 | await settleApp(app); |
| 94 | 95 |
test/tui-origins.test.ts +2 −1
| @@ -14,6 +14,7 @@ import type { TuiIdentity } from "../src/tui/session.ts"; | ||
| 14 | 14 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; |
| 15 | 15 | import { quietLogs } from "./helpers/globals.ts"; |
| 16 | 16 | import { settleApp } from "./helpers/tui.ts"; |
| 17 | import { waitFor } from "./helpers/poll.ts"; | |
| 17 | 18 | |
| 18 | 19 | quietLogs(); |
| 19 | 20 | |
| @@ -120,7 +121,7 @@ describe("connected origins in the terminal", () => { | ||
| 120 | 121 | await h.press("enter"); |
| 121 | 122 | expect(listOrigins(forge.db, made.value.id)[0]!.lastError).toBe(""); |
| 122 | 123 | await h.press("s"); |
| 123 | for (let i = 0; i < 40 && listOrigins(forge.db, made.value.id)[0]!.lastSyncAt === null; i++) await Bun.sleep(100); | |
| 124 | await waitFor(() => listOrigins(forge.db, made.value.id)[0]!.lastSyncAt !== null, { every: 100, within: 4_000 }); | |
| 124 | 125 | if (listOrigins(forge.db, made.value.id)[0]!.lastSyncAt === null) throw new Error(`never came: still waiting while listOrigins(forge.db, made.value.id)[0]!.lastSyncAt === null`); |
| 125 | 126 | await h.press("k"); |
| 126 | 127 | expect(listOrigins(forge.db, made.value.id)[0]!.lastSyncAt).not.toBeNull(); |
test/web-apps.test.ts +2 −1
| @@ -41,6 +41,7 @@ import { getUserByName, type User } from "../src/core/users.ts"; | ||
| 41 | 41 | import { serve, type Servers } from "../src/http/server.ts"; |
| 42 | 42 | import { quietLogs } from "./helpers/globals.ts"; |
| 43 | 43 | import { adminCookie } from "./helpers/admin.ts"; |
| 44 | import { waitFor } from "./helpers/poll.ts"; | |
| 44 | 45 | |
| 45 | 46 | quietLogs(); |
| 46 | 47 | |
| @@ -1015,7 +1016,7 @@ describe("apps, runs and the crew on the web", () => { | ||
| 1015 | 1016 | const noPreviewServed = async (): Promise<void> => { |
| 1016 | 1017 | const open = () => |
| 1017 | 1018 | app.db.query("SELECT 1 FROM run WHERE state IN ('queued','running') LIMIT 1").get(); |
| 1018 | for (let i = 0; i < 200 && open() !== null; i++) await Bun.sleep(25); | |
| 1019 | await waitFor(() => open() === null, { every: 25, within: 5_000 }); | |
| 1019 | 1020 | if (open() !== null) throw new Error(`never came: still waiting while open() !== null`); |
| 1020 | 1021 | app.db.query("UPDATE run SET serving_at = NULL WHERE env LIKE 'pull-%'").run(); |
| 1021 | 1022 | }; |
new above
offline
Shared shape is taken from at most 300 tokens per member, so the report overstates how much one implementation would save for long functions. Say so in the header or lift the cap.