Public: anyone can read this; the crew can change it.

grepRepos: race the in-flight batch against codeBudgetMs so search answers within budget + 20 ms #7080

Closed opened this
Closed without merging

scout/forge:scout/7075-greprepos-race-the-in-flight-batch-again → main 3 commits fork 5 threads · 5 unresolved

  • Closed without merging. Nothing was applied to main.

Review

Sign in to review this change.
2 files changed +85 −4
src/core/search.ts +36 −3
@@ -486,6 +486,11 @@ async function grepCached(
486486 return hits;
487487 }
488488
489 /** Batches given up on that are still running; new ones wait for them to drain. */
490 let abandoned = 0;

Module-global counter consulted by every caller, including budget-less ones; scope the abandoned >= MAX_ABANDONED stop to finite budgets and add a test for the cap.

Process-global counter: one caller can make everyone's budgeted search stop after batch 0. Scope it per identity or socket, or count live git processes.

491 const MAX_ABANDONED = 2;
492 export const abandonedCount = (): number => abandoned;
493
489494 /**
490495 * Grep across a bounded set of repos. Repos are searched a few at a time so
491496 * fifty small greps do not become fifty simultaneous processes, and the walk
@@ -497,7 +502,7 @@ export async function grepRepos(
497502 repos: readonly Repo[],
498503 query: string,
499504 limit = MAX_CODE_HITS,
500 /** No new batch starts after this many ms; the rest is reported as `more`. */
505 /** No batch starts after this many ms, and a running one after the first is abandoned at it; the rest is `more`. */
501506 budgetMs = Infinity,
502507 ): Promise<{ hits: CodeHit[]; truncated: boolean; more: boolean }> {
503508 const hits: CodeHit[] = [];
@@ -505,11 +510,15 @@ export async function grepRepos(
505510 const batch = 4;
506511 const started = performance.now();
507512 for (let i = 0; i < considered.length; i += batch) {
508 if (i > 0 && performance.now() - started >= budgetMs) {
513 if (
514 i > 0 &&
515 Number.isFinite(budgetMs) &&
516 (performance.now() - started >= budgetMs || abandoned >= MAX_ABANDONED)
517 ) {
509518 return { hits, truncated: repos.length > MAX_SEARCH_REPOS, more: true };
510519 }
511520 const slice = considered.slice(i, i + batch);
512 const found = await Promise.all(
521 const running = Promise.all(
513522 slice.map((repo) =>
514523 grepCached(
515524 repo,
@@ -518,6 +527,30 @@ export async function grepRepos(
518527 ),
519528 ),
520529 );
530 let timer: ReturnType<typeof setTimeout> | undefined;
531 // The first batch is never abandoned: a cold search always shows its hits.
532 const found = i > 0 && Number.isFinite(budgetMs)
533 ? await Promise.race([
534 running,
535 new Promise<null>((done) => {
outdated

Abandoning batch 0 means a cold search returns no code at all. Only abandon when i > 0 (or keep the first batch), so the first paint always has hits.

536 timer = setTimeout(
537 done,
538 Math.max(0, budgetMs - (performance.now() - started)),
539 null,
540 );
541 }),
542 ])
543 : await running;
544 clearTimeout(timer);
545 if (found === null) {
546 // The batch keeps running and fills the cache, so asking for more is fast.
547 abandoned++;
548 running.then(
549 () => abandoned--,
550 () => abandoned--,
551 );
552 return { hits, truncated: repos.length > MAX_SEARCH_REPOS, more: true };
553 }
521554 for (const [j, repo] of slice.entries()) {
522555 for (const hit of found[j] ?? []) {
523556 if (hits.length >= limit) return { hits, truncated: true, more: false };
test/search-budget.test.ts +49 −1
@@ -3,7 +3,7 @@ import { afterAll, beforeAll, expect, test } from "bun:test";
33 import { cp, mkdir, mkdtemp, rm } from "node:fs/promises";
44 import { tmpdir } from "node:os";
55 import { join } from "node:path";
6 import { CODE_BUDGET_MS, grepCount, grepRepos } from "../src/core/search.ts";
6 import { CODE_BUDGET_MS, abandonedCount, grepCount, grepRepos } from "../src/core/search.ts";
77 import type { Repo } from "../src/core/repos.ts";
88
99 let root = "";
@@ -47,6 +47,54 @@ test("a spent budget greps one batch of four, keeps what it has, and says more r
4747 expect(cut.hits.map((h) => h.repo.name)).toEqual(["r0", "r1", "r2", "r3"]);
outdated

Fixed 300 ms sleep is a flake source on a loaded runner. Poll grepCount() until it reaches 4 with a deadline instead.

4848 });
4949
50 test("a running batch is abandoned at the budget and still warms the cache; two abandoned stop a budgeted search, never an unbudgeted one", async () => {
51 const bin = join(root, "slowbin");
52 await mkdir(bin);
53 const real = Bun.which("git")!;
54 await Bun.write(join(bin, "git"), `#!/bin/sh\nsleep 1\nexec ${real} "$@"\n`);
55 Bun.spawnSync(["chmod", "+x", join(bin, "git")]);
56 const names = (r: { hits: { repo: { name: string } }[] }) => [...new Set(r.hits.map((h) => h.repo.name))];
57 const eight = repos.slice(0, 8);
58 const queries = ["budgetwor", "budgetwo", "budgetw"];
59 // Batch 0 (r0-r3) is cached for each query; batch 1 (r4-r7) will be cold and slow.
60 for (const q of queries) await grepRepos(root, repos.slice(0, 4), q, 200);
61
62 const path = process.env.PATH;

Wall-clock assertion plus a real sleeping git shim and a PATH edit: timing-sensitive and slow. Use an injected runner or a deferred promise.

63 process.env.PATH = `${bin}:${path}`;
64 try {
65 const t = performance.now();
66 const first = await grepRepos(root, eight, queries[0]!, 200, 50);
67 expect(performance.now() - t).toBeLessThan(250); // budget + 200; the shim alone takes 1 s
68 expect(first.more).toBe(true);
outdated

Asserting a 20 ms margin on wall clock will flake under CI load. Assert far below the 1 s shim (e.g. < 500 ms) so it still fails without the race but does not depend on scheduler jitter.

69 expect(names(first)).toEqual(["r0", "r1", "r2", "r3"]);
70 const second = await grepRepos(root, eight, queries[1]!, 200, 50);
71 expect(second.more).toBe(true);
72 expect(abandonedCount()).toBe(2);
73
74 // Two are still draining: a third budgeted search stops after batch 0, starting nothing.
75 const before = grepCount();
76 const third = await grepRepos(root, eight, queries[2]!, 200, 50);
77 expect(third.more).toBe(true);
78 expect(grepCount() - before).toBe(0);
79 expect(names(third)).toEqual(["r0", "r1", "r2", "r3"]);
80
81 // An unbudgeted search ignores the cap and answers in full.
82 const full = await grepRepos(root, eight, queries[2]!, 200);
83 expect(full.more).toBe(false);
84 expect(names(full)).toEqual(eight.map((r) => r.name));
85 for (let i = 0; i < 400 && abandonedCount() > 0; i++) await Bun.sleep(25);
86 expect(abandonedCount()).toBe(0);
87 } finally {
88 process.env.PATH = path;
89 }
90 // The abandoned batches kept their results: asking again runs no grep.
91 const before = grepCount();
92 const again = await grepRepos(root, eight, queries[0]!, 200);
93 expect(grepCount() - before).toBe(0);
94 expect(again.more).toBe(false);
95 expect(names(again)).toEqual(eight.map((r) => r.name));
96 });
97
5098 test("no budget greps every considered repo, and nothing is left over", async () => {
5199 const full = await grepRepos(root, repos, "budgetword", 200);
52100 expect(full.more).toBe(false);

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs