trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns15g rEventsFlows

A pull opens on the origin forge in one action, and its check and review state come back here #8990

Merged masoncrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

mason/forge:mason/8965-a-pull-opens-on-the-origin-forge-in-one → main 4 commits fork 5 threads · 5 unresolved

  • Merged. mason/8965-a-pull-opens-on-the-origin-forge-in-one was merged into main as 2cd7122.

Review

Sign in to review this change.
18 files changed +775 −3
docs/features/origin-pull-state.md added +35
@@ -0,0 +1,35 @@
1 # A pull proposed to an origin, and its state coming back
2
3 On a repository mirrored from another forge, "Propose upstream" from a native
4 fork carries the branch to the origin and opens the pull there
5 (`actions/pulls.ts`, `origins/push-through.ts`), under the connecting account.
6
7 What the origin then says comes back two ways, through one path
8 (`origins/pull-check.ts`):
9
10 - **Webhook.** After the pull opens, a forge origin is asked once to deliver
11 the mirror's events to `POST /api/v1/repos/:owner/:name/origin-hook`
12 (`origins/hook.ts`; needs this forge's `url` to be https and a token with
13 admin scope on the origin). The delivery is signed with a secret derived
14 from the forge's key and the repository, so nothing is stored. It is
15 believed for nothing but "look again": the named pull is re-read from the
16 origin. Every refusal is the same 404.
17 A delivery that names no pull is accepted and ignored, so one delivery costs
18 one pull's re-read.
19 - **Poll.** Each sync sweep re-reads at most 20 open pulls: those whose check
20 is not settled (never read, running, not run, or for an older head), and
21 any settled one not read for an hour, least recently read first. So a
22 delivery that never came, or an origin that was down, is caught up on the
23 next sweep, and a sweep costs a bounded number of origin requests.
24
25 Asking the origin for the hook happens after the pull opens and off the
26 request path (10 s at most); the person has their pull first.
27
28 The pull page, and the pull screen in the terminal (the merge box), show the
29 origin's check ("Check on bob/app: passing, as of …") and import its reviews. An origin that does not answer leaves the last word
30 standing. A forge origin gives its own `check` word; a Gitea or Forgejo origin
31 gives its combined commit status; GitHub's checks are not read yet.
32
33 The signed webhook path is tested by a hand-posted, correctly signed request
34 (`test/origin-pull-state.test.ts`); no real origin has delivered to it yet.
35 The terminal line is tested in `test/tui-pull-origin-check.test.ts`.
src/actions/pulls.ts +5
@@ -80,6 +80,7 @@ import {
8080 } from "./access.ts";
8181 import { canWrite } from "../core/repos.ts";
8282 import { refreshPullHead as refreshHead } from "../core/following.ts";
83 import { watchPullsSoon } from "../origins/hook.ts";
8384 import { carryBranchThrough } from "../origins/push-through.ts";
8485
8586 export interface OpenPullInput {
@@ -179,6 +180,10 @@ export async function openPull(
179180 actorId: user.value.id,
180181 payload: { number: created.value.number, via: "origin" },
181182 });
183 // Ask the origin to say when this pull's check or reviews move; the sync
184 // poll covers an origin that cannot or will not. Not waited for: the
185 // person has their pull already.
sentinelcrew outdated

This awaits two origin round-trips inside the user request. Run it detached with a timeout.
186 void watchPullsSoon(app, baseRepo);
182187 const stored = getIssueById(app.db, imported.issueId);
183188 const pull = stored
184189 ? getPullByNumber(app, baseRepo, created.value.number)
src/contract/openapi/routes/getApiV1ReposOwnerNamePullsNumber.ts +1 −1
@@ -1,3 +1,3 @@
11 import { REPO, type Route } from "../shared.ts";
22
3 export const route: Route = { method: "get", path: `${REPO}/pulls/{number}`, summary: "Read one pull with its diff and review gate (diff=0 leaves the diff out)", auth: false, query: { diff: "string" } };
3 export const route: Route = { method: "get", path: `${REPO}/pulls/{number}`, summary: "Read one pull with its diff and review gate (diff=0 leaves the diff out); `check` is the word for the head's check: green, red, running or none", auth: false, query: { diff: "string" }, reply: { check: "string" } };
src/contract/openapi/routes/index.ts +1
@@ -269,6 +269,7 @@ export { route as postApiV1ReposOwnerNameIssuesNumberPrivate } from "./postApiV1
269269 export { route as postApiV1ReposOwnerNameLabels } from "./postApiV1ReposOwnerNameLabels.ts";
270270 export { route as postApiV1ReposOwnerNameMakeItYours } from "./postApiV1ReposOwnerNameMakeItYours.ts";
271271 export { route as postApiV1ReposOwnerNameMilestonesNumberLimb } from "./postApiV1ReposOwnerNameMilestonesNumberLimb.ts";
272 export { route as postApiV1ReposOwnerNameOriginHook } from "./postApiV1ReposOwnerNameOriginHook.ts";
272273 export { route as postApiV1ReposOwnerNameOriginReconnect } from "./postApiV1ReposOwnerNameOriginReconnect.ts";
273274 export { route as postApiV1ReposOwnerNamePlan } from "./postApiV1ReposOwnerNamePlan.ts";
274275 export { route as postApiV1ReposOwnerNamePrompts } from "./postApiV1ReposOwnerNamePrompts.ts";
src/contract/openapi/routes/postApiV1ReposOwnerNameOriginHook.ts added +9
@@ -0,0 +1,9 @@
1 import { REPO, type Route } from "../shared.ts";
2
3 export const route: Route = {
4 method: "post",
5 path: `${REPO}/origin-hook`,
6 summary:
7 "The origin of a mirror saying a pull moved: a webhook body signed `x-forge-signature-256: sha256=<hmac>` with the secret this forge derived for the mirror; it only names a pull, which is then read again from the origin. Answers 200 { ok }, and 404 for any refusal",
8 auth: false,
9 };
src/core/migration-pins/20261103160000-origin-pull-check.pin added +1
@@ -0,0 +1,1 @@
1 4200e89310aff7209911c83e66a55745361cc9d72673aa6dbc640d437ddbed5f
src/core/migrations/20261103160000-origin-pull-check.ts added +11
@@ -0,0 +1,11 @@
1 // How a mirrored pull's check stands on its origin (#8965): the word the
2 // origin gave for one head, and when it was last asked. One row per pull,
3 // replaced as the origin's webhook or the poll brings news. Additive.
4 export default `
5 CREATE TABLE origin_pull_check (
6 issue_id TEXT PRIMARY KEY REFERENCES issue(id) ON DELETE CASCADE,
7 head_oid TEXT NOT NULL,
8 word TEXT NOT NULL CHECK (word IN ('green','red','running','none')),
9 checked_at TEXT NOT NULL
10 );
11 `;
src/http/api.ts +14
@@ -55,6 +55,7 @@ import { isProbe } from "../kernel/probe.ts";
5555 import { PRIVATE_ADDRESS_REFUSED } from "../kernel/address_words.ts";
5656 import { forgeState, PARENT_HEADER } from "../spawn/clients.ts";
5757 import { familyView } from "../spawn/family.ts";
58 import { HOOK_BODY_MAX, HOOK_SIGNATURE_HEADER, receiveOriginHook } from "../origins/hook.ts";
5859 import { receiveBeat, SIGNATURE_HEADER } from "../spawn/heartbeat.ts";
5960 import { takeOrTell } from "../actions/follow.ts";
6061 import { noteRead, telemetryReader, type Reader } from "../actions/telemetry.ts";
@@ -2466,6 +2467,7 @@ export function apiRoutes(router: Router<Handler>): void {
24662467 const hold = getHold(ctx.app.db, found.pull.id);
24672468 return json({
24682469 ...withLabels(ctx, found.pull),
2470 check: checkWord(checkSummary(ctx.app.db, found.pull.id, status.headOid).run ?? undefined, status.headOid),
24692471 checkStale: checkStaleNow,
24702472 variantSiblings: variantSiblings(ctx.app.db, [found.pull])[0] ?? [],
24712473 ...(live ? { mergeable: status.mergeable, conflicts: status.conflicts } : {}),
@@ -3241,6 +3243,18 @@ export function apiRoutes(router: Router<Handler>): void {
32413243 return kept ? json({ ok: true }) : apiError("not_found", 404, "no such door");
32423244 });
32433245
3246 // A mirror's origin saying a pull moved (`origins/hook.ts`). Signed; every
3247 // refusal is the same 404.
3248 router.add("POST", "/api/v1/repos/:owner/:name/origin-hook", async (ctx) => {
3249 // Not `loadRepo`: the origin is no reader here, the signature is the proof.
3250 const repo = getRepo(ctx.app.db, ctx.params.owner ?? "", ctx.params.name ?? "");
3251 const bytes = await readBounded(ctx.request, HOOK_BODY_MAX);
3252 if (!repo || bytes === "too_large") return notFound();
3253 const raw = new TextDecoder().decode(bytes);
3254 const kept = await receiveOriginHook(ctx.app, repo, raw, ctx.request.headers.get(HOOK_SIGNATURE_HEADER));
3255 return kept ? json({ ok: true }) : notFound();
3256 });
3257
32443258 // The family (#425): the parent for anyone, the children for an admin.
32453259 router.add("GET", "/api/v1/family", async (ctx) => {
32463260 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
src/origins/hook.ts added +111
@@ -0,0 +1,111 @@
1 /**
2 * The origin's webhook, in both directions.
3 *
4 * Out: after a pull is proposed there, a forge origin is asked once to send
5 * this mirror's door its events (`watchPulls`). In: the delivery is believed
6 * for nothing but "look again" — it is signed with a secret only this forge
7 * and the origin hold, and it only names a pull; the pull is then read from
8 * the origin as the sync worker would (`pull-check.ts`). The secret is
9 * derived from the forge's key and the repository, so nothing is stored.
10 */
11
12 import { createHmac } from "node:crypto";
13 import type { App } from "../app.ts";
14 import { verifySignature } from "../core/webhooks.ts";
15 import { originForRepo } from "../core/origins.ts";
16 import type { Repo } from "../core/repos.ts";
17 import { loadKey } from "../kernel/secret.ts";
18 import { bustOrigin } from "../view/cache.ts";
19 import { providerForOrigin } from "./credential.ts";
20 import { refreshOriginPull } from "./pull-check.ts";
21
22 /** The most a delivery may say; it names a pull and little else. */
23 export const HOOK_BODY_MAX = 256 * 1024;
24 export const HOOK_SIGNATURE_HEADER = "x-forge-signature-256";
25
26 /** The secret the origin signs this mirror's deliveries with. */
27 export function hookSecret(app: App, repo: Repo): string {
28 return createHmac("sha256", loadKey(app.config.dataDir))
29 .update(`origin-hook:${repo.id}`)
30 .digest("hex");
31 }
32
33 /** Where the origin delivers: https only, because the origin refuses the rest. */
34 export function hookUrl(app: App, repo: Repo): string | undefined {
35 const base = app.config.url;
36 if (base === undefined || !/^https:\/\//i.test(base)) return undefined;
37 return `${base.replace(/\/+$/, "")}/api/v1/repos/${encodeURIComponent(repo.ownerName)}/${encodeURIComponent(repo.name)}/origin-hook`;
38 }
39
40 /** The pull number a delivery names: ours, Gitea's or GitHub's shape. */
41 export function numberIn(body: unknown): number | undefined {
42 const at = (v: unknown, key: string): unknown =>
43 typeof v === "object" && v !== null ? (v as Record<string, unknown>)[key] : undefined;
44 for (const n of [
45 at(at(body, "payload"), "number"),
46 at(body, "number"),
47 at(at(body, "pull_request"), "number"),
48 ])
49 if (typeof n === "number" && Number.isSafeInteger(n) && n > 0) return n;
50 return undefined;
51 }
52
53 /**
54 * A delivery for a mirror: answers whether it was accepted. Every refusal is
55 * the same answer, so nothing says which mirrors have an origin.
56 */
57 export async function receiveOriginHook(
58 app: App,
59 repo: Repo,
60 raw: string,
61 signature: string | null,
62 ): Promise<boolean> {
63 const link = originForRepo(app.db, repo.id);
64 if (!link || signature === null) return false;
65 if (!(await verifySignature(hookSecret(app, repo), raw, signature))) return false;
66 let body: unknown;
67 try {
68 body = JSON.parse(raw);
69 } catch {
70 return false;
71 }
72 const provider = providerForOrigin(app, link.origin);
73 if (!provider.ok) return false;
74 const ctx = { db: app.db, repoId: repo.id, actorId: link.origin.userId };
75 const number = numberIn(body);
76 // A delivery naming no pull costs nothing; the sync sweep catches up.
sentinelcrew outdated

A signed delivery that names no pull triggers a full re-read of up to 20 pulls. Refuse it, or re-read only what changed, so one delivery costs one pull.
77 if (number === undefined) return true;
78 await refreshOriginPull(ctx, provider.value, link.fullName, number);
79 bustOrigin(link.origin.id, link.fullName);
80 return true;
81 }
82
83 /**
84 * Ask the origin to deliver this mirror's events here, once. Best effort: an
85 * origin that cannot be asked is still polled, so the pull is never lost.
86 */
87 export async function watchPulls(app: App, repo: Repo): Promise<void> {
88 const link = originForRepo(app.db, repo.id);
89 const url = hookUrl(app, repo);
90 if (!link || url === undefined || link.origin.software !== "forge") return;
91 const provider = providerForOrigin(app, link.origin);
92 if (!provider.ok || provider.value.watch === undefined) return;
93 await provider.value.watch(link.fullName, { url, secret: hookSecret(app, repo) });
94 }
95
96 /** The longest the origin is given to accept the hook. */
97 export const WATCH_TIMEOUT_MS = 10_000;
98
99 /**
100 * `watchPulls`, for the request path: the caller does not wait for it, and it
101 * gives up after `WATCH_TIMEOUT_MS`. It never rejects.
102 */
103 export function watchPullsSoon(app: App, repo: Repo, timeoutMs = WATCH_TIMEOUT_MS): Promise<void> {
104 let timer: ReturnType<typeof setTimeout> | undefined;
105 const giveUp = new Promise<void>((resolve) => {
106 timer = setTimeout(resolve, timeoutMs);
107 });
108 return Promise.race([watchPulls(app, repo).catch(() => undefined), giveUp]).finally(() =>
109 clearTimeout(timer),
110 );
111 }
src/origins/providers/forge.ts +58 −1
@@ -12,6 +12,7 @@
1212 import type {
1313 ListIssuesQuery,
1414 Page,
15 RemoteCheck,
1516 RemoteComment,
1617 ProviderResult,
1718 RemoteIdentity,
@@ -19,11 +20,12 @@ import type {
1920 RemoteMilestone,
2021 RemoteIssue,
2122 RemoteRepo,
23 RemoteReviewState,
2224 } from "../types.ts";
2325 import { ForgejoProvider, nextPage, toRepo } from "./forgejo.ts";
2426 import { NO_ACCOUNT } from "../words.ts";
2527 import { forgeCursor, toForgeComment, toForgeWork } from "./forge-work.ts";
26 import { asArray, asString, dig, fail, field, seg, splitFullName, succeed } from "./http.ts";
28 import { asArray, asId, asString, asTime, dig, fail, field, seg, splitFullName, succeed } from "./http.ts";
2729
2830 const MAX_COMMENT_PAGES = 1000;
2931 const NONE = { ok: true, value: [] } as const;
@@ -134,6 +136,61 @@ export class ForgeProvider extends ForgejoProvider {
134136 return succeed(toForgeWork(result.value.data, this.webUrl, fullName));
135137 }
136138
139 /** The pull's own `check` word, as the forge says it of the head it names. */
140 override async getPullCheck(fullName: string, number: number): Promise<ProviderResult<RemoteCheck>> {
141 const base = this.repoPath(fullName);
142 if (base === undefined) return fail("not_found", `not a repository name: ${fullName}`);
143 const result = await this.api.call({ path: `${base}/pulls/${number}`, query: { diff: 0 } });
144 if (!result.ok) return result;
145 const word = asString(field(result.value.data, "check"));
146 return succeed({
147 word: word === "green" || word === "red" || word === "running" ? word : "none",
148 headOid: asString(field(result.value.data, "headOid")),
149 });
150 }
151
152 /** The verdicts, from the forge's own `review-states` (a forge has no Gitea review objects). */
153 override async listReviewStates(
154 fullName: string,
155 number: number,
156 ): Promise<ProviderResult<readonly RemoteReviewState[]>> {
157 const base = this.repoPath(fullName);
158 if (base === undefined) return fail("not_found", `not a repository name: ${fullName}`);
159 const result = await this.api.call({ path: `${base}/pulls/${number}/review-states` });
160 if (!result.ok) return result;
161 const states: RemoteReviewState[] = [];
162 for (const raw of asArray(result.value.data)) {
163 const state = asString(field(raw, "state"));
164 if (state !== "approved" && state !== "changes_requested" && state !== "commented") continue;
165 const created = asTime(field(raw, "createdAt")) ?? new Date().toISOString();
166 states.push({
167 remoteId: asId(field(raw, "id")),
168 author: asString(field(raw, "originAuthor")) || asString(field(raw, "authorName")),
169 state,
170 body: asString(field(raw, "body")),
171 commitOid: asString(field(raw, "commitOid")),
172 createdAt: created,
173 htmlUrl: `${this.webUrl}/${fullName}/pulls/${number}`,
174 });
175 }
176 return succeed(states);
177 }
178
179 async watch(fullName: string, hook: { url: string; secret: string }): Promise<ProviderResult<void>> {
180 const base = this.repoPath(fullName);
181 if (base === undefined) return fail("not_found", `not a repository name: ${fullName}`);
182 const listed = await this.api.call({ path: `${base}/hooks` });
183 if (!listed.ok) return listed;
184 const rows = asArray(field(listed.value.data, "items") ?? listed.value.data);
185 if (rows.some((row) => asString(field(row, "url")) === hook.url)) return succeed(undefined);
186 const made = await this.api.call({
187 path: `${base}/hooks`,
188 method: "POST",
189 body: { url: hook.url, secret: hook.secret, kinds: "*" },
190 });
191 return made.ok ? succeed(undefined) : made;
192 }
193
137194 override async listComments(
138195 fullName: string,
139196 number: number,
src/origins/providers/forgejo.ts +24
@@ -32,6 +32,7 @@ import type {
3232 ProviderResult,
3333 RemoteBlob,
3434 RemoteBranch,
35 RemoteCheck,
3536 RemoteComment,
3637 RemoteCommit,
3738 RemoteIdentity,
@@ -502,6 +503,29 @@ export class ForgejoProvider implements Provider {
502503 return succeed(toPull(result.value.data));
503504 }
504505
506 /** The head's combined commit status: success, failure or error, pending. */
507 async getPullCheck(fullName: string, number: number): Promise<ProviderResult<RemoteCheck>> {
508 const named = splitFullName(fullName);
509 if (!named) return fail("not_found", `not a repository name: ${fullName}`);
510 const base = `/repos/${seg(named.owner)}/${seg(named.name)}`;
511 const pull = await this.api.call({ path: `${base}/pulls/${number}` });
512 if (!pull.ok) return pull;
513 const headOid = asString(dig(pull.value.data, "head", "sha"));
514 if (!/^[0-9a-f]{40,64}$/.test(headOid)) return succeed({ word: "none", headOid: "" });
515 const status = await this.api.call({ path: `${base}/commits/${headOid}/status` });
516 if (!status.ok) return status;
517 const state = asString(field(status.value.data, "state"));
518 const word =
519 state === "success"
520 ? "green"
521 : state === "failure" || state === "error"
522 ? "red"
523 : state === "pending" && asNumber(field(status.value.data, "total_count")) > 0
524 ? "running"
525 : "none";
526 return succeed({ word, headOid });
527 }
528
505529 async listComments(
506530 fullName: string,
507531 number: number,
src/origins/pull-check.ts added +114
@@ -0,0 +1,114 @@
1 /**
2 * A mirrored pull as its origin currently shows it: the pull itself, its
3 * reviewers' verdicts, and how its check stands.
4 *
5 * Two things bring it up to date and they share this one path: the origin's
6 * webhook (`origins/hook.ts`) names a pull and it is re-read at once, and
sentinelcrew outdated

Wrong path: the hook is origins/hook.ts.
7 * the sync worker walks the open pulls each sweep, so a delivery that never
8 * came is caught on the next. Nothing the webhook says is believed; it only
9 * says where to look, and the origin is asked.
10 */
11
12 import type { Database } from "bun:sqlite";
13 import { importIssue, importReviewStates } from "./import.ts";
14 import type { Provider } from "./types.ts";
15
16 /** The most pulls re-read per sweep. */
17 export const POLLED_PULLS = 20;
18
19 export type CheckWord = "green" | "red" | "running" | "none";
20
21 export interface OriginCheck {
22 readonly word: CheckWord;
23 readonly headOid: string;
24 /** When the origin last answered. */
25 readonly checkedAt: string;
26 }
27
28 /** How the origin said a pull's check stands, when it ever has. */
29 export function originCheckOf(db: Database, issueId: string): OriginCheck | undefined {
30 const row = db
31 .query<{ word: CheckWord; head_oid: string; checked_at: string }, [string]>(
32 "SELECT word, head_oid, checked_at FROM origin_pull_check WHERE issue_id = ?1",
33 )
34 .get(issueId);
35 return row ? { word: row.word, headOid: row.head_oid, checkedAt: row.checked_at } : undefined;
36 }
37
38 /**
39 * Ask the origin about one pull and keep its answer. An origin that does not
40 * answer leaves what was kept exactly as it was. Answers whether it spoke.
41 */
42 export async function refreshOriginPull(
43 ctx: { db: Database; repoId: string; actorId: string },
44 provider: Provider,
45 fullName: string,
46 number: number,
47 now: Date = new Date(),
48 ): Promise<boolean> {
49 const remote = await provider.getIssue(fullName, number);
50 if (!remote.ok || remote.value.pull === undefined) return false;
51 const imported = importIssue(ctx, remote.value);
52 const states = await provider.listReviewStates(fullName, number);
53 if (states.ok) importReviewStates(ctx, imported.issueId, states.value);
54 if (provider.getPullCheck === undefined) return true;
55 const check = await provider.getPullCheck(fullName, number);
56 if (!check.ok) return false;
57 ctx.db
58 .query(
59 `INSERT INTO origin_pull_check (issue_id, head_oid, word, checked_at)
60 VALUES (?1, ?2, ?3, ?4)
61 ON CONFLICT (issue_id) DO UPDATE SET head_oid = ?2, word = ?3, checked_at = ?4`,
62 )
63 .run(imported.issueId, check.value.headOid, check.value.word, now.toISOString());
64 return true;
65 }
66
67 /** A settled pull is still re-read this often, to catch a review that moved nothing else. */
68 export const SETTLED_REREAD_MS = 60 * 60 * 1000;
69
70 /**
71 * The first half of the line both doors show: "Check on bob/app: passing".
72 * The caller adds when the origin said so, in its own voice.
73 */
74 export function originCheckLead(fullName: string, kept: OriginCheck, pullHead: string): string {
75 const stale = kept.headOid !== "" && kept.headOid !== pullHead;
76 return `Check on ${fullName === "" ? "the origin" : fullName}: ${
77 stale ? "waiting for the new head" : CHECK_WORDS[kept.word]
78 }`;
79 }
80
81 const CHECK_WORDS: Record<CheckWord, string> = {
82 green: "passing",
83 red: "failing",
84 running: "running",
85 none: "has not run",
86 };
87
88 /**
89 * The poll: the open pulls of a mirror whose check is not settled (never
90 * read, running, not yet run, or for an older head), plus any settled one
91 * not read for an hour, least recently read first and at most
92 * `POLLED_PULLS` — so a sweep costs a bounded number of origin requests.
93 */
94 export async function refreshOpenPulls(
95 ctx: { db: Database; repoId: string; actorId: string },
96 provider: Provider,
97 fullName: string,
98 now: Date = new Date(),
99 ): Promise<number> {
100 const open = ctx.db
101 .query<{ number: number }, [string, number, string]>(
102 `SELECT i.number FROM issue i JOIN pull p ON p.issue_id = i.id
103 LEFT JOIN origin_pull_check c ON c.issue_id = i.id
104 WHERE i.repo_id = ?1 AND i.state = 'open' AND p.merged_at IS NULL
105 AND (c.issue_id IS NULL OR c.word IN ('running', 'none')
106 OR c.head_oid <> p.head_oid OR c.checked_at < ?3)
107 ORDER BY COALESCE(c.checked_at, '') ASC, i.number DESC LIMIT ?2`,
108 )
109 .all(ctx.repoId, POLLED_PULLS, new Date(now.getTime() - SETTLED_REREAD_MS).toISOString());
110 let spoke = 0;
111 for (const { number } of open)
112 if (await refreshOriginPull(ctx, provider, fullName, number)) spoke++;
113 return spoke;
114 }
src/origins/sync.ts +6
@@ -54,6 +54,7 @@ import {
5454 importReviews,
5555 importReviewStates,
5656 } from "./import.ts";
57 import { refreshOpenPulls } from "./pull-check.ts";
5758 import type { Provider, ProviderFailure, RemoteIssue } from "./types.ts";
5859
5960 const log = createLog("origins");
@@ -487,6 +488,11 @@ async function syncMirrorUnguarded(
487488 setCursor(app.db, origin.id, key, runStarted);
488489 }
489490
491 // A check finishing or a review landing does not move a pull's updated
492 // time on every origin, so the open pulls are read again each sweep: the
493 // catch-up for a webhook that never came.
494 await refreshOpenPulls(ctx, provider, mirror.fullName);
sentinelcrew

Every sweep makes up to 20×(3–4) sequential origin requests per mirror. Limit this to pulls whose check is unsettled, or cap the work per sweep.
495
490496 // Whatever the browse cache holds for this remote was true before this
491497 // sweep and is a guess after it. Dropping it costs one request; keeping it
492498 // means a page that shows a stale answer with a fresh timestamp.
src/origins/types.ts +24
@@ -167,6 +167,12 @@ export interface RemoteReviewState {
167167 readonly htmlUrl: string;
168168 }
169169
170 /** How a pull's check stands on its origin, for the head named. */
171 export interface RemoteCheck {
172 readonly word: "green" | "red" | "running" | "none";
173 readonly headOid: string;
174 }
175
170176 export interface RemoteReview {
171177 readonly remoteId: string;
172178 readonly author: string;
@@ -374,6 +380,24 @@ export interface Provider {
374380 number: number,
375381 ): Promise<ProviderResult<readonly RemoteReviewState[]>>;
376382
383 /**
384 * Where the origin's check of a pull stands. Left out by an origin whose
385 * checks this adapter cannot read; the pull page then shows none.
386 */
387 getPullCheck?(
388 fullName: string,
389 number: number,
390 ): Promise<ProviderResult<RemoteCheck>>;
391
392 /**
393 * Ask the origin to deliver this repository's events to `url`, signed with
394 * `secret`, unless it already does. Left out by an origin that cannot.
395 */
396 watch?(
397 fullName: string,
398 hook: { url: string; secret: string },
399 ): Promise<ProviderResult<void>>;
400
377401 createIssue(
378402 fullName: string,
379403 input: { title: string; body: string; labels?: readonly string[] },
src/tui/screens/pull.ts +17
@@ -59,6 +59,7 @@ import { teamVerdicts, type TeamVerdict } from "../../core/issue_teams.ts";
5959 import { getIssue, type Issue } from "../../core/issues.ts";
6060 import { isProposed, pullMayRun } from "../../core/pull_trust.ts";
6161 import { runningProgress, stepsLine } from "../../checks/progress.ts";
62 import { originCheckLead, originCheckOf } from "../../origins/pull-check.ts";
6263 import { askCheck } from "../../checks/again.ts";
6364 import { checkStale, checksHold } from "../../checks/stale.ts";
6465 import { checkWords } from "../../checks/summary.ts";
@@ -265,6 +266,8 @@ export function pullScreen(
265266 /** The pull's base branch, to name what the check was merged with. */
266267 let checkBase = "";
267268 let checkProgress = "";
269 /** The origin's own check on a mirrored pull, worded; "" when it never said. */
270 let originCheckText = "";
268271 /** The run page's three step rows as a line; empty when the log is hidden. */
269272 let checkSteps = "";
270273 /** `findingsWords` of the preview's website-spec findings; empty when none were recorded. */
@@ -338,6 +341,12 @@ export function pullScreen(
338341 ? NO_CHECK
339342 : checkSummary(f.db, row.id, row.headOid || (facts?.headOid ?? ""));
340343 checkBase = row?.baseBranch ?? facts?.baseBranch ?? "";
344 const originSaid =
345 row === undefined || repo?.origin == null ? undefined : originCheckOf(f.db, row.id);
346 originCheckText =
347 originSaid === undefined || repo?.origin == null || row === undefined
348 ? ""
349 : `${originCheckLead(repo.origin.fullName, originSaid, row.headOid)}, as of ${originSaid.checkedAt.slice(0, 16).replace("T", " ")} UTC`;
341350 next =
342351 row === undefined
343352 ? null
@@ -1014,6 +1023,7 @@ export function pullScreen(
10141023 3 +
10151024 (standingText(standing(), requireReview) === "" ? 0 : 1) +
10161025 (checksLine === "" ? 0 : 1) +
1026 (originCheckText === "" ? 0 : 1) +
10171027 teams.length;
10181028 const [strip, head, previewRow, boxRow, tabRow, body, comp] = split(
10191029 area,
@@ -1069,6 +1079,7 @@ export function pullScreen(
10691079 checkOld,
10701080 check,
10711081 checksLine,
1082 originLine: originCheckText,
10721083 standing: standing(),
10731084 required: requireReview,
10741085 teams,
@@ -1695,6 +1706,8 @@ function drawMergeBox(
16951706 check: CheckSummary;
16961707 /** The checks row, already worded; "" draws none. */
16971708 checksLine: string;
1709 /** The origin's check on a mirrored pull, already worded; "" draws none. */
1710 originLine: string;
16981711 standing: Standing;
16991712 required: boolean;
17001713 teams: readonly TeamVerdict[];
@@ -1799,6 +1812,10 @@ function drawMergeBox(
17991812 );
18001813 y++;
18011814 }
1815 if (o.originLine !== "" && y < inner.y + inner.h) {
1816 s.text(inner.x, y, truncate(o.originLine, inner.w), style(c.mutedFg, c.card), inner.x + inner.w);
1817 y++;
1818 }
18021819 // The teams the issue names, one row each: who still owes a review.
18031820 for (const v of o.teams) {
18041821 if (y >= inner.y + inner.h) break;
src/web/pages/pull.ts +13 −1
@@ -9,6 +9,7 @@
99 */
1010
1111 import { pullRoster } from "../studio/pull_roster.ts";
12 import { originCheckLead, originCheckOf } from "../../origins/pull-check.ts";
1213 import { askCheck } from "../../checks/again.ts";
1314 import { addCommentThrough } from "../../actions/issues.ts";
1415 import { COMMENT_EMPTY, COMMENT_NOT_POSTED } from "../../actions/comment_words.ts";
@@ -410,7 +411,18 @@ function pullChecksLine(
410411 );
411412 const found = !isApp ? null : pullPreviewFindings(ctx.app.db, repo.id, previewEnv(pull.number));
412413 const red = redAloneLine(redMark(repo.id, pull.id, status.headOid));
413 return html`${line}${red}${found === null ? "" : specFindings(found)}`;
414 return html`${originCheckLine(ctx, repo, pull)}${line}${red}${found === null ? "" : specFindings(found)}`;
415 }
416
417 /**
418 * How the origin's own check stands on a mirrored pull, as it last said and
419 * when. Empty for a pull the origin has never reported on.
420 */
421 function originCheckLine(ctx: Ctx, repo: Repo, pull: Pull): Html | "" {
422 if (repo.origin === null) return "";
423 const kept = originCheckOf(ctx.app.db, pull.id);
424 if (kept === undefined) return "";
425 return html`<p class="muted">${originCheckLead(repo.origin.fullName, kept, pull.headOid)}, as of ${time(kept.checkedAt)}</p>`;
414426 }
415427
416428 /** One §5 row in a toned card; `.setting-list` wraps the sentence and drops the action under it on a phone. */
test/origin-pull-state.test.ts added +231
@@ -0,0 +1,231 @@
1 import { afterAll, beforeAll, describe, expect, setDefaultTimeout, test } from "bun:test";
2 import { mkdtemp, rm } from "node:fs/promises";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { commitToBranch } from "../src/actions/repos.ts";
6 import { forkRepo } from "../src/actions/repos.ts";
7 import { proposeUpstream } from "../src/actions/upstream.ts";
8 import { createApp, type App } from "../src/app.ts";
9 import { getOriginById } from "../src/core/origins.ts";
10 import { getPull } from "../src/core/pulls.ts";
11 import { getRepo } from "../src/core/repos.ts";
12 import { createToken } from "../src/core/tokens.ts";
13 import { getUserByName } from "../src/core/users.ts";
14 import { serve, type Servers } from "../src/http/server.ts";
15 import { hookSecret, HOOK_SIGNATURE_HEADER } from "../src/origins/hook.ts";
16 import { originCheckOf } from "../src/origins/pull-check.ts";
17 import { syncOrigin, syncsSettled } from "../src/origins/sync.ts";
18 import { mirrorAndWait } from "./fake-origin.ts";
19 import { seedAdmin } from "./helpers/admin.ts";
20 import { proveAll } from "./helpers/verified.ts";
21
22 /**
23 * A pull proposed from a fork of a mirror opens on the origin (#8965), and
24 * how its check and reviews stand there comes back: on the origin's signed
25 * webhook at once, by the poll when the webhook never arrived, and again
26 * after the origin has been down.
27 */
28
29 setDefaultTimeout(60_000);
30
31 const ALICE = "alice-login-secret";
32 const BOB = "bob-login-secret";
33 const basic = (user: string, secret: string): string =>
34 `Basic ${Buffer.from(`${user}:${secret}`).toString("base64")}`;
35
36 describe("a pull proposed to an origin forge", () => {
37 const dirs: string[] = [];
38 let mine: App;
39 let friend: App;
40 let mineServers: Servers;
41 let friendServers: Servers;
42 let originId = "";
43 let number = 0;
44
45 const mineApi = (path: string, init: RequestInit = {}) =>
46 fetch(`${mineServers.url}${path}`, {
47 ...init,
48 headers: {
49 authorization: basic("alice", ALICE),
50 "content-type": "application/json",
51 ...(init.headers as Record<string, string> | undefined),
52 },
53 });
54 const bobApi = (path: string, init: RequestInit = {}) =>
55 fetch(`${friendServers.url}${path}`, {
56 ...init,
57 headers: { authorization: basic("bob", BOB), "content-type": "application/json" },
58 });
59 const mirror = () => getRepo(mine.db, "alice", "app")!;
60 const sync = () => syncOrigin(mine, getOriginById(mine.db, originId)!);
61 const ours = () => getPull(mine.db, mirror().id, number)!;
62
63 /** The origin runs a check of the pull's head, and it ends as `state`. */
64 const checkOnOrigin = (state: "ok" | "failed") => {
65 const bob = getRepo(friend.db, "bob", "app")!;
66 const issue = friend.db
67 .query<{ id: string; head_oid: string }, [string, number]>(
68 "SELECT i.id, p.head_oid FROM issue i JOIN pull p ON p.issue_id = i.id WHERE i.repo_id = ?1 AND i.number = ?2",
69 )
70 .get(bob.id, number)!;
71 friend.db.query("DELETE FROM run WHERE issue_id = ?1").run(issue.id);
72 friend.db
73 .query("INSERT INTO run (id, repo_id, kind, issue_id, commit_oid, state) VALUES (?1, ?2, 'check', ?3, ?4, ?5)")
74 .run(`run-${state}`, bob.id, issue.id, issue.head_oid, state);
75 };
76
77 const deliver = async (body: unknown, secret = hookSecret(mine, mirror())) => {
78 const raw = JSON.stringify(body);
79 const mac = new Bun.CryptoHasher("sha256", secret).update(raw).digest("hex");
80 return fetch(`${mineServers.url}/api/v1/repos/alice/app/origin-hook`, {
81 method: "POST",
82 headers: { "content-type": "application/json", [HOOK_SIGNATURE_HEADER]: `sha256=${mac}` },
83 body: raw,
84 });
85 };
86
87 beforeAll(async () => {
88 const make = async () => {
89 const dir = await mkdtemp(join(tmpdir(), "forge-origin-pull-"));
90 dirs.push(dir);
91 return dir;
92 };
93 const probe = Bun.serve({ port: 0, fetch: () => new Response("") });
94 const friendPort = probe.port as number;
95 await probe.stop(true);
96 friend = createApp({ dataDir: await make(), port: friendPort });
97 friendServers = await serve(friend);
98 await seedAdmin(friend, { username: "bob", email: "[email protected]", password: BOB });
99 proveAll(friend);
100 const token = createToken(friend.db, getUserByName(friend.db, "bob")!.id, "alice", "admin").token;
101 await bobApi("/api/v1/repos", { method: "POST", body: JSON.stringify({ name: "app", visibility: "public" }) });
102 await bobApi("/api/v1/repos/bob/app/contents", {
103 method: "POST",
104 body: JSON.stringify({ branch: "main", message: "first", files: [{ path: "README.md", content: "app\n" }] }),
105 });
106
107 mine = createApp({ dataDir: await make(), port: 0, url: "https://mine.test" });
108 mineServers = await serve(mine);
109 await seedAdmin(mine, { username: "alice", email: "[email protected]", password: ALICE });
110 proveAll(mine);
111 const connected = await mineApi("/api/v1/user/origins", {
112 method: "POST",
113 body: JSON.stringify({ host: friendServers.url, token }),
114 });
115 originId = ((await connected.json()) as { id: string }).id;
116 const job = await mirrorAndWait(
117 await mineApi(`/api/v1/user/origins/${originId}/mirror`, {
118 method: "POST",
119 body: JSON.stringify({ fullName: "bob/app" }),
120 }),
121 );
122 expect(job?.stage).toBe("done");
123 }, 120_000);
124
125 afterAll(async () => {
126 await syncsSettled();
127 mineServers.stop();
128 friendServers.stop();
129 mine.close();
130 friend.close();
131 for (const dir of dirs) await rm(dir, { recursive: true, force: true, maxRetries: 5 });
132 }, 120_000);
133
134 test("Propose upstream from a fork opens the pull on the origin with the author kept, and asks for its events", async () => {
135 const alice = getUserByName(mine.db, "alice")!;
136 const fork = await forkRepo(mine, alice, mirror(), "app-mine");
137 if (!fork.ok) throw new Error(JSON.stringify(fork.error));
138 const edited = await commitToBranch(mine, alice, fork.value, {
139 branch: "main",
140 base: "main",
141 message: "my change",
142 files: [{ path: "NOTE.md", content: "mine\n" }],
143 });
144 if (!edited.ok) throw new Error(JSON.stringify(edited.error));
145 const proposed = await proposeUpstream(mine, alice, fork.value);
146 if (!proposed.ok) throw new Error(JSON.stringify(proposed.error));
147 expect(proposed.value.opened).toBe(true);
148 number = proposed.value.pull.number;
149
150 const there = await (await bobApi(`/api/v1/repos/bob/app/pulls/${number}`)).json();
151 expect(there).toMatchObject({ headBranch: "alice/main", baseBranch: "main", state: "open" });
152 expect(getPull(mine.db, mirror().id, number)?.originUrl).toContain(`/bob/app/pulls/${number}`);
153
154 // The commit that arrived keeps its author.
155 const commits = (await (await bobApi("/api/v1/repos/bob/app/commits?ref=alice/main&limit=1")).json()) as { author: string; subject: string }[];
156 expect(commits[0]).toMatchObject({ author: "alice", subject: "my change" });
157
158 // Asking for the hook is off the request path: it lands shortly after.
159 let hooks = "";
160 for (let i = 0; i < 50 && !hooks.includes("/alice/app/origin-hook"); i++) {
161 hooks = JSON.stringify(await (await bobApi("/api/v1/repos/bob/app/hooks")).json());
162 if (!hooks.includes("/alice/app/origin-hook")) await Bun.sleep(100);
163 }
164 expect(hooks).toContain("https://mine.test/api/v1/repos/alice/app/origin-hook");
165 });
166
167 test("a webhook from the origin brings the check and a review at once", async () => {
168 checkOnOrigin("ok");
169 const review = await fetch(`${friendServers.url}/api/v1/repos/bob/app/pulls/${number}/review-states`, {
170 method: "POST",
171 headers: { authorization: basic("bob", BOB), "content-type": "application/json" },
172 body: JSON.stringify({ state: "changes_requested", body: "not yet" }),
173 });
174 expect(review.status).toBe(201);
175 expect((await deliver({ kind: "run.finished", payload: { number } }, "wrong")).status).toBe(404);
176 expect(originCheckOf(mine.db, ours().id)).toBeUndefined();
177
178 expect((await deliver({ kind: "run.finished", payload: { number } })).status).toBe(200);
179 expect(originCheckOf(mine.db, ours().id)?.word).toBe("green");
180 const page = await (await mineApi(`/alice/app/pulls/${number}`)).text();
181 expect(page.replace(/\s+/g, " ")).toContain("passing");
182 const reviews = mine.db
183 .query<{ n: number }, [string]>("SELECT COUNT(*) AS n FROM review WHERE issue_id = ?1 AND state = 'changes_requested'")
184 .get(ours().id)!;
185 expect(reviews.n).toBe(1);
186 });
187
188 test("the poll catches a change whose webhook never came", async () => {
189 checkOnOrigin("failed");
190 expect(originCheckOf(mine.db, ours().id)?.word).toBe("green");
191 // A settled pull is read again by the poll once it has not been for an hour.
192 await sync();
193 expect(originCheckOf(mine.db, ours().id)?.word).toBe("green");
194 mine.db.query("UPDATE origin_pull_check SET checked_at = '2020-01-01T00:00:00.000Z'").run();
195 await sync();
196 expect(originCheckOf(mine.db, ours().id)?.word).toBe("red");
197 });
198
199 test("the origin down for a while leaves the last word standing, and it catches up when back", async () => {
200 friendServers.stop();
201 const report = await sync();
202 expect(report.error).not.toBe("");
203 expect(originCheckOf(mine.db, ours().id)?.word).toBe("red");
204 expect((await mineApi(`/alice/app/pulls/${number}`)).status).toBe(200);
205
206 friendServers = await serve(friend);
207 checkOnOrigin("ok");
208 mine.db.query("UPDATE origin_pull_check SET checked_at = '2020-01-01T00:00:00.000Z'").run();
209 expect((await sync()).error).toBe("");
210 expect(originCheckOf(mine.db, ours().id)?.word).toBe("green");
211 });
212
213 test("a signed delivery that names no pull re-reads nothing", async () => {
214 checkOnOrigin("failed");
215 expect((await deliver({ kind: "push" })).status).toBe(200);
216 expect(originCheckOf(mine.db, ours().id)?.word).toBe("green");
217 expect((await deliver({ kind: "run.finished", payload: { number } })).status).toBe(200);
218 expect(originCheckOf(mine.db, ours().id)?.word).toBe("red");
219 // A settled pull is not read by the sweep either.
220 checkOnOrigin("ok");
221 expect((await sync()).error).toBe("");
222 expect(originCheckOf(mine.db, ours().id)?.word).toBe("red");
223 });
224
225 test("a delivery that is unsigned, or for a repository with no origin, is the same 404", async () => {
226 const unsigned = await fetch(`${mineServers.url}/api/v1/repos/alice/app/origin-hook`, { method: "POST", body: "{}" });
227 expect(unsigned.status).toBe(404);
228 const none = await fetch(`${mineServers.url}/api/v1/repos/alice/app-mine/origin-hook`, { method: "POST", body: "{}" });
229 expect(none.status).toBe(404);
230 });
231 });
test/tui-pull-origin-check.test.ts added +100
@@ -0,0 +1,100 @@
1 /**
2 * The terminal's pull screen says how the origin's own check stands on a
3 * mirrored pull, in the words the web page uses (#8965, both doors).
4 */
5 import { afterAll, beforeAll, expect, test } from "bun:test";
6 import { mkdtemp, rm } from "node:fs/promises";
7 import { tmpdir } from "node:os";
8 import { join } from "node:path";
9 import { openPull } from "../src/actions/pulls.ts";
10 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
11 import { createApp, type App } from "../src/app.ts";
12 import { createOrigin, linkRepoToOrigin } from "../src/core/origins.ts";
13 import { getRepoById } from "../src/core/repos.ts";
14 import { getPull } from "../src/core/pulls.ts";
15 import { createUser } from "../src/core/users.ts";
16 import { createForgeApp } from "../src/tui/forge.ts";
17 import { pullScreen } from "../src/tui/screens/pull.ts";
18 import { local } from "../src/tui/screens/where.ts";
19 import { headlessTerminal } from "../src/tui/term/terminal.ts";
20 import { quietLogs } from "./helpers/globals.ts";
21 import { settleApp } from "./helpers/tui.ts";
22
23 quietLogs();
24
25 let dataDir: string;
26 let app: App;
27
28 beforeAll(async () => {
29 dataDir = await mkdtemp(join(tmpdir(), "forge-tui-origin-check-"));
30 app = createApp({ dataDir, port: 0, host: "forge.test" });
31 });
32
33 afterAll(async () => {
34 app.close();
35 await rm(dataDir, { recursive: true, force: true });
36 });
37
38 test("a mirrored pull shows the origin's check and when it said so; a pull it never reported on shows none", async () => {
39 const made = await createUser(app.db, {
40 origin: "seed",
41 trusted: true,
42 username: "alice",
43 email: "[email protected]",
44 password: "a-good-passphrase",
45 adminIfFirst: true,
46 });
47 if (!made.ok) throw new Error(String(made.error));
48 const alice = made.value;
49 const r = await createRepo(app, alice, { name: "lib", visibility: "public" });
50 if (!r.ok) throw new Error(String(r.error));
51 for (const branch of ["main", "feat"]) {
52 const c = await commitToBranch(app, alice, r.value, {
53 branch,
54 message: branch,
55 files: [{ path: `${branch}.md`, content: "x\n" }],
56 });
57 if (!c.ok) throw new Error(JSON.stringify(c.error));
58 }
59 const o = await openPull(app, alice, r.value, { title: "Pull feat", body: "", baseBranch: "main", headBranch: "feat" });
60 if (!o.ok) throw new Error(String(o.error));
61 const origin = createOrigin(app.db, dataDir, {
62 userId: alice.id,
63 kind: "forgejo",
64 apiUrl: "https://bob.test/api/v1",
65 webUrl: "https://bob.test",
66 login: "bob",
67 token: "t",
68 software: "forge",
69 });
70 if (!origin.ok) throw new Error(origin.error);
71 linkRepoToOrigin(app.db, r.value.id, {
72 originId: origin.value.id,
73 fullName: "bob/lib",
74 remoteId: "1",
75 htmlUrl: "https://bob.test/bob/lib",
76 });
77 const pull = getPull(app.db, r.value.id, o.value.number)!;
78
79 const screenText = async () => {
80 const { app: tui, session } = createForgeApp(
81 app,
82 { sshUsername: "alice", remoteAddress: "127.0.0.1", fingerprint: "SHA256:t", user: alice },
83 { terminal: headlessTerminal(160, 44), exitHooks: false },
84 );
85 tui.start();
86 const repo = getRepoById(app.db, r.value.id)!;
87 tui.ctx.push(pullScreen(session, tui.ctx, local(session, repo), pull.number));
88 await settleApp(tui);
89 const text = tui.screen.toText();
90 tui.stop();
91 return text;
92 };
93
94 expect(await screenText()).not.toContain("Check on");
95
96 app.db
97 .query("INSERT INTO origin_pull_check (issue_id, head_oid, word, checked_at) VALUES (?1, ?2, 'green', '2026-11-03T10:15:00.000Z')")
98 .run(pull.id, pull.headOid);
99 expect(await screenText()).toContain("Check on bob/lib: passing, as of 2026-11-03 10:15 UTC");
100 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs