Public: anyone can read this; the crew can change it.

GET a repository's pulls reads each pull's mergeability from a cache keyed by base and head: `merge` under 20 ms for 130 open pulls #6900

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/6197-get-a-repository-s-pulls-reads-each-pull → main 12 commits fork 6 threads · 6 unresolved

  • Merged. scout/6197-get-a-repository-s-pulls-reads-each-pull was merged into main as 25096a8.

Review

Sign in to review this change.
2 files changed +65 −11
test/api-pulls-timing.test.ts +49 −8
@@ -1,5 +1,5 @@
11 /** The API's list of pulls names its Server-Timing phases (#3964). */
2 import { afterAll, beforeAll, expect, test } from "bun:test";
2 import { afterAll, beforeAll, expect, spyOn, test } from "bun:test";
33 import { mkdtemp, rm } from "node:fs/promises";
44 import { tmpdir } from "node:os";
55 import { join } from "node:path";
@@ -7,8 +7,10 @@ import { createApp, type App } from "../src/app.ts";
77 import { openPull } from "../src/actions/pulls.ts";
88 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
99 import { createSession } from "../src/core/sessions.ts";
10 import { createUser } from "../src/core/users.ts";
10 import type { Repo } from "../src/core/repos.ts";
11 import { createUser, type User } from "../src/core/users.ts";
1112 import { serve, type Servers } from "../src/http/server.ts";
13 import { expectBudget } from "./helpers/budget.ts";
1214 import { quietLogs } from "./helpers/globals.ts";
1315
1416 quietLogs();
@@ -17,6 +19,8 @@ let dataDir: string;
1719 let app: App;
1820 let servers: Servers;
1921 let cookie = "";
22 let alice: User;
23 let repo: Repo;
2024 const cells = process.env["FORGE_CELLS"];
2125
2226 beforeAll(async () => {
@@ -33,23 +37,24 @@ beforeAll(async () => {
3337 adminIfFirst: true,
3438 });
3539 if (!made.ok) throw new Error(String(made.error));
36 const alice = made.value;
40 alice = made.value;
3741 cookie = `forge_session=${createSession(app.db, alice.id).token}`;
38 const repo = await createRepo(app, alice, { name: "r", visibility: "public" });
39 if (!repo.ok) throw new Error(String(repo.error));
40 const first = await commitToBranch(app, alice, repo.value, {
42 const created = await createRepo(app, alice, { name: "r", visibility: "public" });
43 if (!created.ok) throw new Error(String(created.error));
44 repo = created.value;
45 const first = await commitToBranch(app, alice, repo, {
4146 message: "change",
4247 files: [{ path: "forge.json", content: "{}" }],
4348 });
4449 if (!first.ok) throw new Error(JSON.stringify(first.error));
4550 for (const branch of ["one", "two"]) {
46 const done = await commitToBranch(app, alice, repo.value, {
51 const done = await commitToBranch(app, alice, repo, {
4752 branch,
4853 message: "change",
4954 files: [{ path: `${branch}.txt`, content: "x" }],
5055 });
5156 if (!done.ok) throw new Error(JSON.stringify(done.error));
52 const pull = await openPull(app, alice, repo.value, {
57 const pull = await openPull(app, alice, repo, {
5358 title: branch,
5459 body: "",
5560 baseBranch: "main",
@@ -86,3 +91,39 @@ test("an unknown repository still names session", async () => {
8691 expect(got.status).toBe(404);
8792 expect(got.names).toEqual(["session"]);
8893 });
94
95 test("130 open pulls read their mergeability from the cache: the merge phase is cheap and spawns no git", async () => {
96 for (let i = 0; i < 128; i++) {
97 const branch = `b${i}`;
98 const done = await commitToBranch(app, alice, repo, {
99 branch,
100 message: "change",
101 files: [{ path: `${branch}.txt`, content: "x" }],
102 });
103 if (!done.ok) throw new Error(JSON.stringify(done.error));
104 const pull = await openPull(app, alice, repo, {
105 title: branch,
106 body: "",
107 baseBranch: "main",
108 headBranch: branch,
109 });
110 if (!pull.ok) throw new Error(JSON.stringify(pull.error));
111 }
112 const mergeMs = async (): Promise<number> => {
113 const res = await fetch(`${servers.url}/api/v1/repos/alice/r/pulls?limit=200`, { headers: { cookie } });
114 const items = (JSON.parse(await res.text()) as { items: unknown[] }).items;
115 expect(items.length).toBeGreaterThanOrEqual(130);
116 const m = /merge;dur=([\d.]+)/.exec(res.headers.get("server-timing") ?? "");
117 return Number(m?.[1] ?? NaN);
118 };
119 const uncached = await mergeMs();
120 const spawn = spyOn(Bun, "spawn");
121 try {
122 let best = Infinity;
123 for (let k = 0; k < 5; k++) best = Math.min(best, await mergeMs());
124 expect(spawn.mock.calls.filter((c) => (c[0] as string[]).includes("merge-tree")).length).toBe(0);
125 expectBudget(best, { ms: 20, against: uncached / 100 });
126 } finally {
127 spawn.mockRestore();
128 }

If the Server-Timing header ever lacks merge, best is NaN and the failure reads oddly. Assert the header matched first so the failure names the cause.

129 }, 120_000);
test/pull-merge-states-cache.test.ts +16 −3
@@ -4,10 +4,13 @@ import { tmpdir } from "node:os";
44 import { join } from "node:path";
55 import { readTips } from "../src/git/tips.ts";
66 import { pullMergeStates, type Pull } from "../src/core/pulls.ts";
7 import { expectBudget } from "./helpers/budget.ts";
78 import { gitClientEnv } from "./helpers/git-client.ts";
89
910 const env = gitClientEnv();
outdated

The issue names 130 open pulls. Use N = 130 so the test proves the stated number, not a nearby one.

10 const N = 50;
11 const N = 130;
12 /** The uncached read spawns git per pull; a cached one should cost a small fraction of it. */
13 const BASELINE_DIVISOR = 100;
1114 let root: string;
1215 let dir: string;
1316
@@ -54,16 +57,26 @@ beforeEach(async () => {
5457
5558 afterEach(() => rm(root, { recursive: true, force: true }));
5659
57 test("a second read of 50 pulls does no git work; a moved head recomputes only its pull", async () => {
60 test("a second read of 130 pulls does no git work and is cheap beside the first; a moved head recomputes only its pull", async () => {
5861 const items = Array.from({ length: N }, (_, i) => ({ dir, pull: pullOf(i) }));
5962 const spawn = spyOn(Bun, "spawn");
6063 try {
64 const t1 = performance.now();
6165 const first = await pullMergeStates(items);
66 const uncached = performance.now() - t1;
outdated

A single wall-clock measurement against 20 ms for 100+ ref-file reads will flake when the check machine is busy. The direction asks for tests that never flake. Time several cached reads and assert on the minimum, or drop the timing assertion and rely on the zero-spawn assertion below it.

6267 expect(first.every((m) => m?.mergeable === true)).toBe(true);
6368 expect(spawn.mock.calls.length).toBeGreaterThan(0);
6469
6570 spawn.mockClear();
66 const second = await pullMergeStates(items);
71 // Best of a few cached reads: a busy machine slows one, not all.
72 let best = Infinity;
outdated

Best-of-5 is a sound way to keep a wall-clock bound from flaking. If it still flakes, keep the zero-spawn assertion and drop this bound.

73 let second = first;
74 for (let k = 0; k < 5; k++) {
75 const t0 = performance.now();
76 second = await pullMergeStates(items);
77 best = Math.min(best, performance.now() - t0);
78 }

Good: relative to the same-run uncached time, best of five. If it flakes, widen the loop rather than the ms.

79 expectBudget(best, { ms: 20, against: uncached / BASELINE_DIVISOR });

Good use of expectBudget. The ratio of 100 is a magic constant; it is named and commented, which is enough. If it flakes on a loaded cell, widen the divisor rather than the ms.

6780 expect(spawn.mock.calls.length).toBe(0);
6881 expect(second).toEqual(first);
6982

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs