trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns14g rEventsFlows

Trust does something: Until shows its date, and a repo can follow several trusted forks at once #9004

Merged torquecrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/forge:torque/9003-trust-does-something-until-shows-its-dat → main 4 commits fork 3 threads · 3 unresolved

  • Merged. torque/9003-trust-does-something-until-shows-its-dat was merged into main as c9616bb.

Review

Sign in to review this change.
23 files changed +1327 −47
docs/features/trusted-sources.md +16
@@ -12,3 +12,19 @@ Work whose `Landed-via` already names this forge is not taken again, so forges f
1212 **Visible.** A landing carries `Landed-from` / `Landed-via` trailers, the `pull.merged` event names `source`, and the changelog (which swaps live when a pull merges) credits the row "folded in from account@forge".
1313
1414 **Both doors.** Settings → Trusted sources on the web, and **Trusted sources** in the terminal's command palette (a forge admin's): `a` trusts, `x` removes, `s` is the kill switch (stop or resume), `v` reverts what the one under the cursor landed. Both ask before reverting, and both go through `src/actions/trusted_sources.ts`.
15
16 ## Following several sources (#9003)
17
18 Settings → Trusted sources shows each entry's end as a date and the days left (`Until 2026-12-02 (29 days left)`); a date is read as the end of that UTC day, and a day that does not exist (`2026-02-31`) or any other shape is refused.
19
20 A repository's settings (**Followed sources**) follow any number of sources next to its one upstream: paste one per line, `address [branch] [as account]`, in one action. An address is an https or http git address (a forge, or a Forgejo fork); it is read anonymously, or with the actor's connected origin when its https origin is the address's. A migrated repository links back by adding where it came from. The account a source's work is trusted as is the address's owner (`jmoran` for `…/jmoran/forge-jay.git`); only a forge admin who is a person may name another with `as` (for `https://forge-apps.benjivers.com/apps/foundry.git`, `as benjivers`). It is never taken from anything the source says.
21
22 Every 30 minutes (and on "Check sources now") each source is checked like the upstream: a new head that merges cleanly opens one pull, `upstream/<source>-<tip7>`, stamped with the source; the next head moves that pull rather than opening another. A trusted account's pull lands on its own; any other account's waits as a guest. A conflict opens one issue for the crew, and no second while it is open; the agent's pull that closes it carries the source. A fold of 50 commits or more, and the crew's resolution of a conflict, are reviewed by the crew before they land; all of them land by true merge.
23
24 ## Followed sources at the terminal
25
26 On a repository's About card an admin presses `s` for the list of followed sources: `a` follows an address (`<url> [branch] [as account]`), `x` stops following the one under the cursor, `u` checks them all now. These go through the same actions as Settings → Followed sources, so the gate (admin of the repository), the audit and the event are the same, and an open settings page updates as the terminal writes.
27
28 A source's address is fetched like the repository's own upstream: `vettedEnv` vets the host (private and loopback addresses are refused) and pins the address that passed, https and http only. The clock checks sources four at a time and starts no new check once a sweep has run ten minutes; what is left is due at the next tick.
29
30 A pull that an agent opens or edits to say it closes a source's conflict issue is stamped with that source when it is opened or edited (`adoptResolution`), never by a predicate that only reads.
src/actions/follow_sources.ts added +129
@@ -0,0 +1,129 @@
1 /**
2 * Following several sources (#9003): an admin of a repository adds sources
3 * by address, one line each, and removes them. Each line is
4 * `<url> [branch] [as <account>]`; the account a source's work is trusted
5 * as is the address's owner unless a forge admin who is a person names
6 * another (a forge whose repositories live under a shared name, such as
7 * `/apps/foundry.git`, belongs to the account that runs it). A connected
8 * origin of the actor's whose https origin is the address's is used to read
9 * it; otherwise it is read anonymously. Checking is `checkSource` in
10 * `upstream_follow.ts`.
11 */
12 import type { App } from "../app.ts";
13 import {
14 addSource,
15 getSource,
16 listSources,
17 removeSource,
18 type FollowSource,
19 } from "../core/follow_sources.ts";
20 import { listOrigins } from "../core/origins.ts";
21 import { canAdmin, type Repo } from "../core/repos.ts";
22 import { cleanAccount } from "../core/trusted_sources.ts";
23 import { upstreamUrl } from "../core/upstream_follow.ts";
24 import type { User } from "../core/users.ts";
25 import { isSafeRef } from "../git/guard.ts";
26 import { err, ok, type Result } from "../kernel/result.ts";
27 import { migrateUrl } from "./migrate.ts";
28 import { checkSource, mayCarryCredential, type CheckError, type Checked } from "./upstream_follow.ts";
29
30 export interface SourceLine {
31 readonly url: string;
32 readonly branch: string;
33 readonly account: string;
34 }
35
36 export type SourceError = "invalid_url" | "invalid_branch" | "invalid_account" | "exists" | "too_many" | "not_found";
37
38 export const SOURCE_REFUSALS: Record<SourceError, string> = {
39 invalid_url: "That is not an https or http git address, without a password.",
40 invalid_branch: "That is not a branch name git accepts.",
41 invalid_account: "That is not an account name, or only a forge admin may name one.",
42 exists: "Already following that address and branch.",
43 too_many: "This repository follows as many sources as it may.",
44 not_found: "No such repository or source.",
45 };
46
47 /** One line of the add form, or the refusal it earns. */
48 export function parseSourceLine(raw: string): Result<SourceLine, SourceError> {
49 const words = raw.trim().split(/\s+/);
50 const as = words.indexOf("as");
51 const account = as === -1 ? "" : (words[as + 1] ?? "");
52 if (as !== -1 && cleanAccount(account) === undefined) return err("invalid_account");
53 const head = as === -1 ? words : words.slice(0, as);
54 if (head.length > 2 || as !== -1 && words.length !== as + 2) return err("invalid_url");
55 const normal = migrateUrl(head[0] ?? "");
56 const url = normal === undefined ? undefined : upstreamUrl(normal);
57 if (url === undefined) return err("invalid_url");
58 const branch = head[1] ?? "main";
59 if (!isSafeRef(branch) || branch.length > 200) return err("invalid_branch");
60 return ok({ url, branch, account: as === -1 ? "" : cleanAccount(account)! });
61 }
62
63 const mayNameAccount = (actor: User): boolean => actor.isAdmin && actor.kind !== "agent";
64
65 /** Add the sources named in `text`, one per line; what each line came to, in order. */
66 export function addFollowSources(
67 app: App,
68 actor: User | undefined,
69 repo: Repo,
70 text: string,
71 ): Result<{ line: string; result: Result<FollowSource, SourceError> }[], "not_found"> {
72 if (actor === undefined || !canAdmin(app.db, repo, actor)) return err("not_found");
73 const origins = listOrigins(app.db, actor.id);
74 const lines = text.split("\n").map((l) => l.trim()).filter((l) => l !== "").slice(0, 20);
75 const done = lines.map((line) => {
76 const parsed = parseSourceLine(line);
77 if (!parsed.ok) return { line, result: parsed };
78 if (parsed.value.account !== "" && !mayNameAccount(actor)) return { line, result: err<SourceError>("invalid_account") };
79 const origin = origins.find((o) => mayCarryCredential(parsed.value.url, o.webUrl));
80 const added = addSource(app.db, {
81 repoId: repo.id,
82 ...parsed.value,
83 originId: origin?.id ?? "",
84 byId: actor.id,
85 });
86 if (typeof added === "string") return { line, result: err<SourceError>(added) };
87 app.bus.append({
88 kind: "repo.source_added",
89 repoId: repo.id,
90 actorId: actor.id,
91 payload: { branch: added.branch },
92 });
93 return { line, result: ok(added) };
94 });
95 return ok(done);
96 }
97
98 export function removeFollowSource(
99 app: App,
100 actor: User | undefined,
101 repo: Repo,
102 sourceId: string,
103 ): Result<FollowSource, "not_found"> {
104 if (actor === undefined || !canAdmin(app.db, repo, actor)) return err("not_found");
105 const source = getSource(app.db, sourceId);
106 if (source === undefined || source.repoId !== repo.id) return err("not_found");
107 removeSource(app.db, sourceId);
108 app.bus.append({ kind: "repo.source_removed", repoId: repo.id, actorId: actor.id, payload: {} });
109 return ok(source);
110 }
111
112 /** "Check now" for every source of a repository, an admin's. */
113 export async function checkSourcesNow(
114 app: App,
115 actor: User | undefined,
116 repo: Repo,
117 ): Promise<Result<{ source: FollowSource; result: Result<Checked, CheckError> }[], "not_found">> {
118 if (actor === undefined || !canAdmin(app.db, repo, actor)) return err("not_found");
119 const out = [];
120 for (const source of listSources(app.db, repo.id)) out.push({ source, result: await checkSource(app, repo, source) });
121 return ok(out);
122 }
123
124 /** Where a source's last check stands, in words, at every door. */
125 export function sourceStanding(s: FollowSource): string {
126 if (s.error !== "") return s.error === "conflict" ? "conflicts, with the crew" : "last check failed";
127 if (s.checkedAt === null) return "not checked yet";
128 return s.behind === 0 ? "up to date" : `${s.behind} behind`;
129 }
src/actions/issues.ts +2
@@ -22,6 +22,7 @@ import { mayView, setSealed } from "../core/sealed.ts";
2222 import { requireVerified } from "./verify.ts";
2323 import { requireAbility } from "./roles.ts";
2424 import type { App } from "../app.ts";
25 import { adoptResolution } from "../core/trusted_sources.ts";
2526 import { crewRole, isCrew } from "../core/agents.ts";
2627 import { moveBlockersToChildren } from "../core/dependencies.ts";
2728 import {
@@ -250,6 +251,7 @@ export function editIssue(
250251 if (changed.length === 0) return ok(issue);
251252 const updated = updateIssueText(app.db, issue.id, input);
252253 if (!updated.ok) return updated;
254 if (issue.kind === "pull" && changed.includes("body")) adoptResolution(app.db, issue.id);
253255 app.bus.append({
254256 kind: `${issue.kind}.edited`,
255257 repoId: repo.id,
src/actions/pulls.ts +2 −1
@@ -83,7 +83,7 @@ import { canWrite } from "../core/repos.ts";
8383 import { refreshPullHead as refreshHead } from "../core/following.ts";
8484 import { getOriginRepo } from "../core/origins.ts";
8585 import { watchPullsSoon } from "../origins/hook.ts";
86 import { ownHost, pullSourceOf, sourceTrailers } from "../core/trusted_sources.ts";
86 import { adoptResolution, ownHost, pullSourceOf, sourceTrailers } from "../core/trusted_sources.ts";
8787 import { carryBranchThrough } from "../origins/push-through.ts";
8888 import { ensureFork, forkHead, pushToFork } from "../origins/fork-route.ts";
8989 import { denied } from "../origins/operator.ts";
@@ -300,6 +300,7 @@ export async function openPull(
300300 },
301301 );
302302 if (!created.ok) return created;
303 adoptResolution(app.db, created.value.id);
303304 // A pull the crew opens is the crew's to see through: it starts assigned
304305 // to the agent, so "who is on it" is legible without a second step (an
305306 // agent cannot assign, so this is the one place it happens).
src/actions/upstream_follow.ts +148 −22
@@ -44,7 +44,27 @@ import { isSafeOid, isSafeRef } from "../git/guard.ts";
4444 import { aheadCount, mergeBase, mergeable, resolveOid, writeRef } from "../git/merge.ts";
4545 import { REMOTE_PROTOCOLS } from "../git/remote-env.ts";
4646 import { repoDir } from "../git/repos.ts";
47 import { ownHost, recordPullSource, sourceOfUrl, takeable, viaOf } from "../core/trusted_sources.ts";
47 import {
48 BIG_FOLD_COMMITS,
49 ownHost,
50 recordPullSource,
51 sourceOfUrl,
52 takeable,
53 viaOf,
54 } from "../core/trusted_sources.ts";
55 import {
56 conflictOf,
57 getSource,
58 listSources,
59 listActiveSources,
60 noteSource,
61 openConflictOf,
62 recordConflict,
63 sourceProposalBranch,
64 sourceRef,
65 stampOf,
66 type FollowSource,
67 } from "../core/follow_sources.ts";
4868 import { err, ok, type Result } from "../kernel/result.ts";
4969 import { providerForOrigin } from "../origins/credential.ts";
5070 import { redact, secretsIn } from "../origins/credentials.ts";
@@ -57,6 +77,9 @@ import { openPull } from "./pulls.ts";
5777 export const FOLLOW_EVERY_MS = 30 * 60_000;
5878 /** One check's whole budget: ls-remote and fetch together. */
5979 const CHECK_TIMEOUT_MS = 120_000;
80 /** Source checks the clock runs at once, and how long a sweep may keep starting new ones. */
81 const SWEEP_LANES = 4;
82 const SWEEP_BUDGET_MS = 10 * 60_000;
6083
6184 /** Where the upstream tip is staged: not a branch, so a check moves nothing of ours. */
6285 export const stagedRef = (branch: string): string => `refs/upstream-follow/${branch}`;
@@ -139,6 +162,26 @@ export async function checkDue(app: App, now = Date.now()): Promise<number> {
139162 app.log.error("upstream check failed", { repo: repo.id, error: String(error) });
sentinelcrew outdated

Sources are checked one after another on the clock, each a network fetch; up to 20 per repo. Bound the time so one slow remote cannot stall the sweep.
140163 }
141164 }
165 // Sources are fetched a few at a time, and none is started once the sweep
166 // has run its budget: each check is bounded (`CHECK_TIMEOUT_MS`), but 20
167 // slow remotes in a row would hold the clock for forty minutes. What is
168 // left is still due at the next tick.
169 const due = listActiveSources(app.db).filter(
170 (s) => s.checkedAt === null || now - Date.parse(s.checkedAt) >= FOLLOW_EVERY_MS,
171 );
172 const stopAt = Date.now() + SWEEP_BUDGET_MS;
173 const lanes = Array.from({ length: Math.min(SWEEP_LANES, due.length) }, async () => {
174 for (let source = due.shift(); source !== undefined && Date.now() < stopAt; source = due.shift()) {
175 const repo = getRepoById(app.db, source.repoId);
176 if (repo === undefined) continue;
177 try {
178 if ((await checkSource(app, repo, source)).ok) checked++;
179 } catch (error) {
180 app.log.error("source check failed", { repo: repo.id, source: source.id, error: String(error) });
181 }
182 }
183 });
184 await Promise.all(lanes);
142185 return checked;
143186 }
144187
@@ -156,13 +199,16 @@ function convergeConflict(
156199 app: App,
157200 checked: Repo,
158201 by: User,
159 follow: UpstreamFollow,
202 follow: { readonly url: string; readonly branch: string },
160203 tip: string,
161204 files: readonly string[],
205 source?: { readonly id: string; readonly host: string; readonly account: string; readonly via: readonly string[] },
162206 ): void {
163207 // The setting as it is now, not as the caller read it before the fetch.
164208 const repo = getRepoById(app.db, checked.id);
165209 if (repo === undefined || repo.autonomy === "off") return;
210 // A source with a conflict still open waits for the crew: a moving head is not asked again.
211 if (source !== undefined && openConflictOf(app.db, source.id)) return;
166212 const title = conflictTitle(tip);
167213 const seen = app.db
168214 .query<{ n: number }, [string, string]>(
@@ -181,6 +227,7 @@ function convergeConflict(
181227 });
182228 if (!opened.ok) return;
183229 let issue = opened.value;
230 if (source !== undefined) recordConflict(app.db, issue.id, { sourceId: source.id, ...source });
184231 const inFlight = new Map<string, number>();
185232 for (const r of listRuns(app.db, repo.id, { kind: "agent", limit: 50 }))
186233 if ((r.state === "running" || r.state === "queued") && r.actorId !== null)
@@ -199,6 +246,18 @@ function convergeConflict(
199246 });
200247 }
201248
249 /** What the crew's brief says about the source a conflict issue was raised for (#9003). */
250 export function sourceFollowing(
251 app: App,
252 issueId: string,
253 ): { parent: string; branch: string; ref: string; upstream: true } | undefined {
254 const conflict = conflictOf(app.db, issueId);
255 const source = conflict === undefined ? undefined : getSource(app.db, conflict.sourceId);
256 return source === undefined
257 ? undefined
258 : { parent: source.url, branch: source.branch, ref: sourceRef(source.id), upstream: true };
259 }
260
202261 /** What the crew's brief says about a following repository's converge run. */
203262 export function upstreamFollowing(
204263 app: App,
@@ -221,23 +280,77 @@ export async function carriesUpstream(
221280 headOid: string,
222281 baseOid: string,
223282 ): Promise<boolean> {
224 const follow = getFollow(app.db, repo.id);
225 if (follow === undefined) return false;
226 const staged = await resolveOid(dir, stagedRef(follow.branch));
227 if (staged === undefined || !isSafeOid(headOid) || !isSafeOid(baseOid)) return false;
228 const shared = await mergeBase(dir, staged, headOid);
229 if (shared === undefined) return false;
230 const inBase = await runGit(["merge-base", "--is-ancestor", shared, baseOid], { cwd: dir });
231 return !inBase.ok;
283 const refs = [
284 ...(getFollow(app.db, repo.id) === undefined ? [] : [stagedRef(getFollow(app.db, repo.id)!.branch)]),
285 ...listSources(app.db, repo.id).map((s) => sourceRef(s.id)),
286 ];
287 if (!isSafeOid(headOid) || !isSafeOid(baseOid)) return false;
288 for (const ref of refs) {
289 const staged = await resolveOid(dir, ref);
290 if (staged === undefined) continue;
291 const shared = await mergeBase(dir, staged, headOid);
292 if (shared === undefined) continue;
293 const inBase = await runGit(["merge-base", "--is-ancestor", shared, baseOid], { cwd: dir });
294 if (!inBase.ok) return true;
295 }
296 return false;
297 }
298
299 /** What one check reads and writes of a followed address: the repository's upstream, or one of its sources. */
300 interface Target {
301 readonly url: string;
302 readonly branch: string;
303 readonly originId: string;
304 readonly byId: string;
305 readonly pendingId: string | null;
306 /** Where the fetched tip is staged. */
307 readonly staged: string;
308 readonly proposal: (tip: string) => string;
309 readonly note: (reading: Parameters<typeof noteCheck>[2]) => void;
310 /** The forge and account the work is stamped as. */
311 readonly stamp: { host: string; account: string } | undefined;
312 readonly sourceId?: string;
232313 }
233314
234315 async function checkOnce(app: App, repo: Repo): Promise<Result<Checked, CheckError>> {
235316 const follow = getFollow(app.db, repo.id);
236317 if (follow === undefined || !follow.enabled || repo.archivedAt !== null || repo.origin !== null)
237318 return err("off");
319 return checkTarget(app, repo, {
320 ...follow,
321 staged: stagedRef(follow.branch),
322 proposal: proposalBranch,
323 note: (r) => noteCheck(app.db, repo.id, r),
324 stamp: sourceOfUrl(follow.url),
325 });
326 }
327
328 /** One check of one followed source (#9003), through the same claim, vet and proposal as the upstream's. */
329 export async function checkSource(app: App, repo: Repo, source: FollowSource): Promise<Result<Checked, CheckError>> {
330 const key = `${repo.id}:${source.id}`;
331 if (checking.has(key)) return err("busy");
332 checking.add(key);
333 try {
334 if (!source.enabled || repo.archivedAt !== null || repo.origin !== null) return err("off");
335 const result = await checkTarget(app, repo, {
336 ...source,
337 sourceId: source.id,
338 staged: sourceRef(source.id),
339 proposal: (tip) => sourceProposalBranch(source.id, tip),
340 note: (r) => noteSource(app.db, source.id, r),
341 stamp: stampOf(source),
342 });
343 app.bus.append({ kind: "repo.upstream_checked", repoId: repo.id, payload: {} });
344 return result;
345 } finally {
346 checking.delete(key);
347 }
348 }
349
350 async function checkTarget(app: App, repo: Repo, follow: Target): Promise<Result<Checked, CheckError>> {
238351 const at = new Date().toISOString();
239352 const failed = (error: CheckError, why = ""): Result<Checked, CheckError> => {
240 noteCheck(app.db, repo.id, { at, error: why || CHECK_REFUSALS[error] });
353 follow.note({ at, error: why || CHECK_REFUSALS[error] });
241354 return err(error);
242355 };
243356 const by = getUserById(app.db, follow.byId);
@@ -273,7 +386,7 @@ async function checkOnce(app: App, repo: Repo): Promise<Result<Checked, CheckErr
273386 if (!isSafeOid(tip)) return failed("no_branch");
274387
275388 // Fetch only when the tip moved: the staged ref is what the last fetch brought.
276 const ref = stagedRef(follow.branch);
389 const ref = follow.staged;
277390 if ((await resolveOid(dir, ref)) !== tip) {
278391 const fetched = await runGitCapture(
279392 [
@@ -298,33 +411,45 @@ async function checkOnce(app: App, repo: Repo): Promise<Result<Checked, CheckErr
298411 if (main === undefined) return failed("git", "The default branch has no commits yet.");
299412 const behind = await aheadCount(dir, main, staged);
300413 if (behind === 0) {
301 noteCheck(app.db, repo.id, { at, tip: staged, behind, error: "" });
414 follow.note({ at, tip: staged, behind, error: "" });
302415 return ok({ tip: staged, behind, outcome: "none" });
303416 }
304417
305418 // Work that already landed here and came back round a ring of forges is not taken again (#8937).
306419 const seenVia = viaOf((await runGitCapture(["log", "-1", "--format=%B", staged], { cwd: dir })).stdout);
307420 if (!takeable(seenVia, ownHost(app.config))) {
308 noteCheck(app.db, repo.id, { at, tip: staged, behind: 0, error: "" });
421 follow.note({ at, tip: staged, behind: 0, error: "" });
309422 return ok({ tip: staged, behind: 0, outcome: "none" });
310423 }
311424 const merged = await mergeable(dir, main, staged);
312425 if (!merged.ok) {
313426 if (merged.error.kind !== "conflict") return failed("git");
314 noteCheck(app.db, repo.id, { at, tip: staged, behind, error: "conflict", files: merged.error.files.length });
315 convergeConflict(app, repo, by, follow, staged, merged.error.files);
427 follow.note({ at, tip: staged, behind, error: "conflict", files: merged.error.files.length });
428 convergeConflict(
429 app,
430 repo,
431 by,
432 follow,
433 staged,
434 merged.error.files,
435 follow.sourceId === undefined || follow.stamp === undefined
436 ? undefined
437 : { id: follow.sourceId, ...follow.stamp, via: seenVia },
438 );
316439 return ok({ tip: staged, behind, outcome: "conflict" });
317440 }
318441
319442 // The branch is ours only if it is at the fetched head, or not there yet.
320 const name = proposalBranch(staged);
443 const name = follow.proposal(staged);
321444 const planted = await resolveOid(dir, `refs/heads/${name}`);
322445 if (planted !== undefined && planted !== staged) return failed("squatted");
323446 if (planted === undefined && !(await writeRef(dir, `refs/heads/${name}`, staged)))
324447 return failed("git", "The proposal's branch could not be written.");
325448
326449 const text = {
327 title: `Upstream: ${behind} new commit${behind === 1 ? "" : "s"} on ${follow.branch}`,
450 title: `Upstream: ${behind} new commit${behind === 1 ? "" : "s"} on ${
451 follow.sourceId === undefined ? follow.branch : `${upstreamShort(follow.url)} ${follow.branch}`
452 }`,
328453 body: `New commits from ${follow.url} (${follow.branch}), up to ${staged.slice(0, 7)}. They merge cleanly into ${repo.defaultBranch}.`,
329454 };
330455 const known = pullForBranch(app.db, repo.id, name);
@@ -332,9 +457,10 @@ async function checkOnce(app: App, repo: Repo): Promise<Result<Checked, CheckErr
332457 const open = pending !== undefined && pending.kind === "pull" && pending.state === "open";
333458 // Where the work came from, so a landing can say, and a trusted source's folds in by itself.
334459 // Only a pull this follow opened is stamped (a person's or a guest's under the same branch name never is).
335 const from = sourceOfUrl(follow.url);
460 const from = follow.stamp;
461 const big = behind >= BIG_FOLD_COMMITS;
336462 const came = (pullId: string): void => {
337 if (from !== undefined) recordPullSource(app.db, { pullId, ...from, via: seenVia });
463 if (from !== undefined) recordPullSource(app.db, { pullId, ...from, via: seenVia, big });
338464 };
339465 let number: number;
340466 if (known !== undefined) {
@@ -351,10 +477,10 @@ async function checkOnce(app: App, repo: Repo): Promise<Result<Checked, CheckErr
351477 const opened = await openPull(app, by, repo, { ...text, baseBranch: repo.defaultBranch, headBranch: name });
352478 if (!opened.ok) return failed("git", `The proposal could not be opened: ${opened.error}`);
353479 came(opened.value.id);
354 noteCheck(app.db, repo.id, { at, tip: staged, behind, error: "", pendingId: opened.value.id, opened: true });
480 follow.note({ at, tip: staged, behind, error: "", pendingId: opened.value.id, opened: true });
355481 return ok({ tip: staged, behind, outcome: "proposed", pull: opened.value.number });
356482 }
357 noteCheck(app.db, repo.id, { at, tip: staged, behind, error: "" });
483 follow.note({ at, tip: staged, behind, error: "" });
358484 return ok({ tip: staged, behind, outcome: "proposed", pull: number });
359485 }
360486
src/core/event_audience.ts +2
@@ -194,6 +194,8 @@ export function classifiedAudience(kind: string): EventAudience | undefined {
194194 // address is often a private repository's.
195195 kind === "repo.upstream_follow" ||
196196 kind === "repo.upstream_checked" ||
197 kind === "repo.source_added" ||
198 kind === "repo.source_removed" ||
197199 kind === "repo.crew_outsiders" ||
198200 // A valve turn names a repository and the owner's budget: for those who administer it (`actions/valves.ts`).
199201 kind === "valve.turned" ||
src/core/follow_sources.ts added +205
@@ -0,0 +1,205 @@
1 /**
2 * The sources a repository follows (#9003): any number, each an https or
3 * http git address (a forge, or a fork on Forgejo) with its branch, read
4 * anonymously or with a connected origin's credential. The commits are
5 * git's (`refs/upstream-sources/<id>`); the row is the setting and the last
6 * reading, rebuilt by the next check. The repository's one `upstream`
7 * (`core/upstream_follow.ts`) stays as it was; these are in addition.
8 *
9 * Pure data, no authz, no events.
10 */
11 import type { Database } from "bun:sqlite";
12 import { id } from "../kernel/id.ts";
13 import { sourceOfUrl } from "./trusted_sources.ts";
14 import { upstreamUrl } from "./upstream_follow.ts";
15
16 export interface FollowSource {
17 readonly id: string;
18 readonly repoId: string;
19 readonly url: string;
20 readonly branch: string;
21 readonly originId: string;
22 /** The trusted account this source's work is stamped as; "" for the address's owner. */
23 readonly account: string;
24 readonly byId: string;
25 readonly enabled: boolean;
26 readonly createdAt: string;
27 readonly checkedAt: string | null;
28 readonly tip: string;
29 readonly behind: number;
30 readonly error: string;
31 readonly pendingId: string | null;
32 readonly opened: number;
33 }
34
35 interface Row {
36 id: string;
37 repo_id: string;
38 url: string;
39 branch: string;
40 origin_id: string;
41 account: string;
42 by_id: string;
43 enabled: number;
44 created_at: string;
45 checked_at: string | null;
46 tip: string;
47 behind: number;
48 error: string;
49 pending_id: string | null;
50 opened: number;
51 }
52
53 const fromRow = (r: Row): FollowSource => ({
54 id: r.id,
55 repoId: r.repo_id,
56 url: r.url,
57 branch: r.branch,
58 originId: r.origin_id,
59 account: r.account,
60 byId: r.by_id,
61 enabled: r.enabled === 1,
62 createdAt: r.created_at,
63 checkedAt: r.checked_at,
64 tip: r.tip,
65 behind: r.behind,
66 error: r.error,
67 pendingId: r.pending_id,
68 opened: r.opened,
69 });
70
71 /** Sources one repository may follow: a bound on the clock's work. */
72 export const SOURCES_PER_REPO = 20;
73
74 /** Where a staged source tip lives: not a branch, so a check moves nothing of ours. */
75 export const sourceRef = (sourceId: string): string => `refs/upstream-sources/${sourceId}`;
76
77 /** The branch a proposal for this source's tip is pushed to. */
78 export const sourceProposalBranch = (sourceId: string, tip: string): string =>
79 `upstream/${sourceId.slice(-6).toLowerCase()}-${tip.slice(0, 7)}`;
80
81 export function addSource(
82 db: Database,
83 input: {
84 readonly repoId: string;
85 readonly url: string;
86 readonly branch: string;
87 readonly originId: string;
88 readonly account: string;
89 readonly byId: string;
90 },
91 now: Date = new Date(),
92 ): FollowSource | "exists" | "too_many" {
93 if (upstreamUrl(input.url) === undefined) throw new Error("a source is an https or http address with no credentials");
94 const have = listSources(db, input.repoId);
95 if (have.some((s) => s.url === input.url && s.branch === input.branch)) return "exists";
96 if (have.length >= SOURCES_PER_REPO) return "too_many";
97 const sourceId = id();
98 db.query(
99 `INSERT INTO repo_source (id, repo_id, url, branch, origin_id, account, by_id, created_at)
100 VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)`,
101 ).run(sourceId, input.repoId, input.url, input.branch, input.originId, input.account, input.byId, now.toISOString());
102 return getSource(db, sourceId)!;
103 }
104
105 export function getSource(db: Database, sourceId: string): FollowSource | undefined {
106 const row = db.query<Row, [string]>("SELECT * FROM repo_source WHERE id = ?1").get(sourceId);
107 return row === null ? undefined : fromRow(row);
108 }
109
110 /** A repository's sources, oldest first. */
111 export function listSources(db: Database, repoId: string): FollowSource[] {
112 return db
113 .query<Row, [string]>("SELECT * FROM repo_source WHERE repo_id = ?1 ORDER BY created_at, id")
114 .all(repoId)
115 .map(fromRow);
116 }
117
118 /** Every source whose following is on. */
119 export function listActiveSources(db: Database): FollowSource[] {
120 return db
121 .query<Row, []>("SELECT * FROM repo_source WHERE enabled = 1 ORDER BY repo_id, created_at, id")
122 .all()
123 .map(fromRow);
124 }
125
126 export function removeSource(db: Database, sourceId: string): FollowSource | undefined {
127 const was = getSource(db, sourceId);
128 if (was === undefined) return undefined;
129 db.query("DELETE FROM repo_source WHERE id = ?1").run(sourceId);
130 return was;
131 }
132
133 /** What one check of a source read; `pendingId` is left alone when absent. */
134 export function noteSource(
135 db: Database,
136 sourceId: string,
137 reading: {
138 readonly at: string;
139 readonly tip?: string;
140 readonly behind?: number;
141 readonly error: string;
142 readonly pendingId?: string | null;
143 readonly opened?: boolean;
144 },
145 ): void {
146 db.query(
147 `UPDATE repo_source SET checked_at = ?2, tip = COALESCE(?3, tip), behind = COALESCE(?4, behind),
148 error = ?5, pending_id = CASE WHEN ?6 = 1 THEN ?7 ELSE pending_id END, opened = opened + ?8
149 WHERE id = ?1`,
150 ).run(
151 sourceId,
152 reading.at,
153 reading.tip ?? null,
154 reading.behind ?? null,
155 reading.error,
156 reading.pendingId === undefined ? 0 : 1,
157 reading.pendingId ?? null,
158 reading.opened === true ? 1 : 0,
159 );
160 }
161
162 /** The forge and account a source's work is stamped as: the one a forge admin named, else the address's owner. */
163 export function stampOf(s: Pick<FollowSource, "url" | "account">): { host: string; account: string } | undefined {
164 const named = sourceOfUrl(s.url);
165 if (named === undefined) return undefined;
166 return s.account === "" ? named : { host: named.host, account: s.account };
167 }
168
169 // --- a conflict issue remembers its source ---------------------------------
170
171 export interface ConflictOf {
172 readonly sourceId: string;
173 readonly host: string;
174 readonly account: string;
175 readonly via: readonly string[];
176 }
177
178 export function recordConflict(db: Database, issueId: string, c: ConflictOf): void {
179 db.query(
180 "INSERT OR IGNORE INTO source_conflict (issue_id, source_id, host, account, via) VALUES (?1, ?2, ?3, ?4, ?5)",
181 ).run(issueId, c.sourceId, c.host, c.account, c.via.join(","));
182 }
183
184 export function conflictOf(db: Database, issueId: string): ConflictOf | undefined {
185 const r = db
186 .query<{ source_id: string; host: string; account: string; via: string }, [string]>(
187 "SELECT source_id, host, account, via FROM source_conflict WHERE issue_id = ?1",
188 )
189 .get(issueId);
190 return r === null
191 ? undefined
192 : { sourceId: r.source_id, host: r.host, account: r.account, via: r.via === "" ? [] : r.via.split(",") };
193 }
194
195 /** Whether a source has a conflict issue still open: it waits for the crew rather than opening a second. */
196 export function openConflictOf(db: Database, sourceId: string): boolean {
197 return (
198 db
199 .query<{ n: number }, [string]>(
200 `SELECT COUNT(*) AS n FROM source_conflict c JOIN issue i ON i.id = c.issue_id
201 WHERE c.source_id = ?1 AND i.state = 'open'`,
202 )
203 .get(sourceId)!.n > 0
204 );
205 }
src/core/migration-pins/20261104100000-follow-sources.pin added +1
@@ -0,0 +1,1 @@
1 f3342f9c9c78b5833d5e2c91443d04c6179e1d127d6e333f5169fe92fa079df1
src/core/migrations/20261104100000-follow-sources.ts added +36
@@ -0,0 +1,36 @@
1 // A repository follows any number of sources (#9003): each its own address and
2 // branch, with its own reading. `account` is the trusted account the source's
3 // work is stamped as, when a forge admin named it (else the address's owner).
4 // `source_conflict` ties a conflict issue to the source whose head it is
5 // about, so the crew's resolving pull carries that source. `pull_source.big`
6 // marks a first fold too large to land on a check alone: the crew reviews it.
7 // Additive.
8 export default `
9 CREATE TABLE repo_source (
10 id TEXT PRIMARY KEY,
11 repo_id TEXT NOT NULL REFERENCES repo(id) ON DELETE CASCADE,
12 url TEXT NOT NULL,
13 branch TEXT NOT NULL,
14 origin_id TEXT NOT NULL DEFAULT '',
15 account TEXT NOT NULL DEFAULT '',
16 by_id TEXT NOT NULL,
17 enabled INTEGER NOT NULL DEFAULT 1,
18 created_at TEXT NOT NULL,
19 checked_at TEXT,
20 tip TEXT NOT NULL DEFAULT '',
21 behind INTEGER NOT NULL DEFAULT 0,
22 error TEXT NOT NULL DEFAULT '',
23 pending_id TEXT,
24 opened INTEGER NOT NULL DEFAULT 0,
25 UNIQUE (repo_id, url, branch)
26 );
27 CREATE TABLE source_conflict (
28 issue_id TEXT PRIMARY KEY REFERENCES issue(id) ON DELETE CASCADE,
29 source_id TEXT NOT NULL,
30 host TEXT NOT NULL,
31 account TEXT NOT NULL,
32 via TEXT NOT NULL DEFAULT ''
33 );
34 CREATE INDEX source_conflict_by_source ON source_conflict (source_id);
35 ALTER TABLE pull_source ADD COLUMN big INTEGER NOT NULL DEFAULT 0;
36 `;
src/core/trusted_sources.ts +74 −7
@@ -8,6 +8,7 @@
88 */
99 import type { Database } from "bun:sqlite";
1010 import { id } from "../kernel/id.ts";
11 import { closesFromBody } from "./pulls.ts";
1112
1213 export interface TrustedSource {
1314 readonly id: string;
@@ -70,9 +71,11 @@ export function trustSource(
7071 if (host === undefined) return "bad_host";
7172 const account = cleanAccount(input.account);
7273 if (account === undefined) return "bad_account";
73 if (!/^\d{4}-\d{2}-\d{2}$/.test(input.until)) return "bad_date";
74 const end = new Date(`${input.until}T23:59:59.999Z`);
75 if (Number.isNaN(end.getTime())) return "bad_date";
74 const day = input.until.trim();
75 if (!/^\d{4}-\d{2}-\d{2}$/.test(day)) return "bad_date";
76 const end = new Date(`${day}T23:59:59.999Z`);
77 // A day that does not exist (02-31) rolls into the next month: refuse it rather than guess.
78 if (Number.isNaN(end.getTime()) || end.toISOString().slice(0, 10) !== day) return "bad_date";
7679 if (end.getTime() <= now.getTime()) return "in_the_past";
7780 const until = end.toISOString();
7881 db.query(
@@ -124,6 +127,15 @@ export function markWarned(db: Database, sourceId: string, now: Date = new Date(
124127 db.query("UPDATE trusted_source SET warned_at = ?2 WHERE id = ?1").run(sourceId, now.toISOString());
125128 }
126129
130 /** How long a trust has left, in words: "29 days left", "less than a day left", or "lapsed". */
131 export function leftWords(until: string, now: Date = new Date()): string {
132 const ms = Date.parse(until) - now.getTime();
133 if (Number.isNaN(ms) || ms <= 0) return "lapsed";
134 const days = Math.floor(ms / DAY_MS);
135 if (days < 1) return "less than a day left";
136 return `${days} day${days === 1 ? "" : "s"} left`;
137 }
138
127139 /** Whether the entry ends within the warning window (or already has). */
128140 export const endsSoon = (s: TrustedSource, now: Date = new Date()): boolean =>
129141 Date.parse(s.until) - now.getTime() <= WARN_DAYS * DAY_MS;
@@ -156,13 +168,68 @@ export interface PullSource {
156168 /** Where a pull came from; the first word stands, a later call never rewrites it. */
157169 export function recordPullSource(
158170 db: Database,
159 input: { pullId: string; host: string; account: string; via: readonly string[] },
171 input: { pullId: string; host: string; account: string; via: readonly string[]; big?: boolean },
160172 now: Date = new Date(),
161173 ): void {
162174 db.query(
163 `INSERT OR IGNORE INTO pull_source (pull_id, host, account, via, received_at)
164 VALUES (?1, ?2, ?3, ?4, ?5)`,
165 ).run(input.pullId, input.host.toLowerCase(), input.account.toLowerCase(), input.via.join(","), now.toISOString());
175 `INSERT OR IGNORE INTO pull_source (pull_id, host, account, via, received_at, big)
176 VALUES (?1, ?2, ?3, ?4, ?5, ?6)`,
177 ).run(
178 input.pullId,
179 input.host.toLowerCase(),
180 input.account.toLowerCase(),
181 input.via.join(","),
182 now.toISOString(),
183 input.big === true ? 1 : 0,
184 );
185 if (input.big === true) db.query("UPDATE pull_source SET big = 1 WHERE pull_id = ?1").run(input.pullId);
186 }
187
188 /** A fold this many commits long is not landed on a check alone: the crew reviews it first (#9003). */
189 export const BIG_FOLD_COMMITS = 50;
190
191 /** Whether the crew reviews this pull before it lands, trusted or not: a big fold, or the crew's resolution of a conflict. */
192 export function needsCrewReview(db: Database, pullId: string): boolean {
193 const row = db
194 .query<{ big: number }, [string]>("SELECT big FROM pull_source WHERE pull_id = ?1")
195 .get(pullId);
196 return row !== null && row.big === 1;
197 }
198
199 /**
200 * The crew's resolution of a source's conflict carries that source: a pull
201 * written by an agent whose description closes the conflict issue the source
202 * raised. Only an agent's pull is adopted, so a guest writing `Closes #N`
203 * gains nothing. Stamped once, reviewed by the crew before it lands. A write,
204 * so it is called where a pull is opened or its description edited, never
205 * from the predicates that read.
206 */
207 export function adoptResolution(db: Database, pullId: string): PullSource | undefined {
208 const pull = db
209 .query<{ repo_id: string; body: string; is_agent: number }, [string]>(
210 `SELECT i.repo_id, i.body, u.is_agent FROM issue i JOIN user u ON u.id = i.author_id
211 WHERE i.id = ?1 AND i.kind = 'pull'`,
212 )
213 .get(pullId);
214 if (pull === null || pull.is_agent !== 1) return undefined;
215 for (const number of closesFromBody(pull.body)) {
216 const c = db
217 .query<{ host: string; account: string; via: string }, [string, number]>(
218 `SELECT c.host, c.account, c.via FROM source_conflict c JOIN issue i ON i.id = c.issue_id
219 WHERE i.repo_id = ?1 AND i.number = ?2`,
220 )
221 .get(pull.repo_id, number);
222 if (c === null) continue;
223 recordPullSource(db, {
224 pullId,
225 host: c.host,
226 account: c.account,
227 via: c.via === "" ? [] : c.via.split(","),
228 big: true,
229 });
230 return pullSourceOf(db, pullId);
231 }
232 return undefined;
166233 }
167234
168235 export function pullSourceOf(db: Database, pullId: string): PullSource | undefined {
src/crew/autonomy.ts +5 −3
@@ -134,7 +134,7 @@ import {
134134 landsOnItsOwn,
135135 markReaders,
136136 } from "./direct.ts";
137 import { foldsIn } from "../core/trusted_sources.ts";
137 import { foldsIn, needsCrewReview } from "../core/trusted_sources.ts";
138138 import { foldWall } from "./fold_walls.ts";
139139 import { warnExpiring } from "./fold_in.ts";
140140 import { crewOffInPreview } from "./preview.ts";
@@ -1960,7 +1960,7 @@ export function bringUpToDate(
19601960 * crew, and is `lapsed` when the reviewer was asked twice and never
19611961 * answered: it lands without it, and says so (`crew/review.ts`).
19621962 */
1963 async function reviewed(
1963 export async function reviewed(
19641964 app: App,
19651965 repo: Repo,
19661966 pull: Pull,
@@ -1971,7 +1971,9 @@ async function reviewed(
19711971 if (seen.approvals > 0) return "pass";
19721972 // A trusted source's pull needs no person's review (#8937); its check, the
19731973 // boot proof and the walls above still stood before this point.
1974 if (foldsIn(app.db, pull.id, now)) return "pass";
1974 // A big first fold, and the crew's resolution of a conflict, are reviewed
1975 // by the crew first (#9003): they take the path below.
1976 if (foldsIn(app.db, pull.id, now) && !needsCrewReview(app.db, pull.id)) return "pass";
19751977 // An agent's pull ships only on an approving review — the crew's reviewer
19761978 // or a person's. With no reviewer, or a review that lapsed, it waits for
19771979 // a person (`waitsForPerson` puts it where a person looks); never by lapse.
src/crew/runner.ts +2 −2
@@ -45,7 +45,7 @@ import {
4545 type Issue,
4646 } from "../core/issues.ts";
4747 import { refreshPullHead } from "../core/following.ts";
48 import { UPSTREAM_TITLE, upstreamFollowing } from "../actions/upstream_follow.ts";
48 import { UPSTREAM_TITLE, sourceFollowing, upstreamFollowing } from "../actions/upstream_follow.ts";
4949 import { getPull, listPulls, type Pull } from "../core/pulls.ts";
5050 import {
5151 canWrite,
@@ -848,7 +848,7 @@ export function convergeFollowing(
848848 user: User,
849849 ): TaskInput["following"] {
850850 if (issue.title.startsWith(UPSTREAM_TITLE)) {
851 const upstream = upstreamFollowing(app, repo.id);
851 const upstream = sourceFollowing(app, issue.id) ?? upstreamFollowing(app, repo.id);
852852 if (upstream !== undefined) return upstream;
853853 }
854854 // Only a parent the agent may read is put in its brief.
src/design/activity.ts +2
@@ -145,6 +145,8 @@ const WORDS: Record<string, string> = {
145145 "repo.detached": "detached a fork from its parent",
146146 "repo.settings_updated": "changed repository settings",
147147 "repo.upstream_follow": "changed which upstream the repository follows",
148 "repo.source_added": "followed another source",
149 "repo.source_removed": "stopped following a source",
148150 "repo.following": "turned following an upstream on or off",
149151 "repo.crew_outsiders": "changed who may trigger the crew",
150152 "repo.mirrored": "mirrored a repository",
src/http/server.ts +10 −1
@@ -6,7 +6,13 @@
66 */
77
88 import { isTokenScheme, tokenSecret } from "./token_header.ts";
9 import { followUpstreamCheck, followUpstreamUpdate } from "../web/pages/upstream-follow.ts";
9 import {
10 followSourceRemove,
11 followSourcesAdd,
12 followSourcesCheck,
13 followUpstreamCheck,
14 followUpstreamUpdate,
15 } from "../web/pages/upstream-follow.ts";
1016 import { secureFor } from "./cookie_secure.ts";
1117 import { refusedByDatabase, shownRefusal } from "./refusal.ts";
1218 import { NOT_A_MEMBER } from "../core/roles.ts";
@@ -1368,6 +1374,9 @@ export function buildRouter(): Router<Handler> {
13681374 router.add("POST", "/:owner/:name/settings/holds", fileHoldUpdate);
13691375 router.add("POST", "/:owner/:name/settings/upstream", followUpstreamUpdate);
13701376 router.add("POST", "/:owner/:name/settings/upstream/check", followUpstreamCheck);
1377 router.add("POST", "/:owner/:name/settings/sources", followSourcesAdd);
1378 router.add("POST", "/:owner/:name/settings/sources/check", followSourcesCheck);
1379 router.add("POST", "/:owner/:name/settings/sources/:id/remove", followSourceRemove);
13711380 router.add("GET", "/:owner/:name/settings/labels", labelsPage);
13721381 router.add("POST", "/:owner/:name/settings/labels", labelCreate);
13731382 router.add("POST", "/:owner/:name/settings/labels/:id", labelUpdate);
src/tui/screens/about.ts +6
@@ -15,6 +15,7 @@
1515 * card is an OSC 8 hyperlink besides.
1616 */
1717
18 import { sourcesScreen } from "./sources.ts";
1819 import { audienceFor } from "../../core/audience.ts";
1920 import { isPinned } from "../../core/pins.ts";
2021 import { parentReleases, releasesSentence } from "../../core/parent_releases.ts";
@@ -227,6 +228,7 @@ export function aboutScreen(
227228 binding(["f"], checking() ? "stop following upstream" : "follow upstream"),
228229 ]
229230 : []),
231 ...(may() && repo?.origin === null ? [binding(["s"], "followed sources")] : []),
230232 // Who can read it is a person's, an admin's call (#8008).
231233 ...(mayPerson()
232234 ? [binding(["V"], repo?.visibility === "public" ? "make private" : "make public")]
@@ -588,6 +590,10 @@ export function aboutScreen(
588590 .finally(() => ctx.invalidate());
589591 return true;
590592 }
593 if (e.key === "s" && repo !== undefined && may()) {
594 ctx.push(sourcesScreen(f, ctx, repo));
595 return true;
596 }
591597 if (e.key === "f" && repo !== undefined && mayFollow()) {
592598 const on = !checking();
593599 const done = setFollowUpstream(f.app, f.user, repo, {
src/tui/screens/sources.ts added +115
@@ -0,0 +1,115 @@
1 /**
2 * The sources a repository follows (#9003) in the terminal, a repository
3 * admin's: the list Settings → Followed sources has, with the same doors —
4 * `a` adds sources (one per line in the web's box; here one at a time,
5 * `<url> [branch] [as account]`), `x` stops following the one under the
6 * cursor, `u` checks them all now. Each goes through
7 * `actions/follow_sources.ts`, so it is the same gate, the same audit and
8 * the same event as the page's, and an open settings page updates as this
9 * one writes. Anyone who is not an admin of the repository is shown nothing.
10 */
11
12 import {
13 addFollowSources,
14 checkSourcesNow,
15 removeFollowSource,
16 SOURCE_REFUSALS,
17 sourceStanding,
18 } from "../../actions/follow_sources.ts";
19 import { upstreamShort } from "../../actions/upstream_follow.ts";
20 import { listSources, stampOf, type FollowSource } from "../../core/follow_sources.ts";
21 import { canAdmin, getRepoById, type Repo } from "../../core/repos.ts";
22 import type { Ctx, Screen } from "../app/app.ts";
23 import { binding } from "../app/keys.ts";
24 import { style } from "../term/style.ts";
25 import { truncate } from "../term/width.ts";
26 import type { Session } from "../session.ts";
27 import { fill, len, split } from "../ui/layout.ts";
28 import { ListState, confirmScreen, drawList, live, promptScreen, runAsync, type ListRow } from "./common.ts";
29
30 export function sourcesScreen(f: Session, ctx: Ctx, repo: Repo): Screen {
31 const state = new ListState();
32 // The row as it is now: the gate is read at use, never remembered.
33 const may = (): boolean => f.user !== undefined && canAdmin(f.db, getRepoById(f.db, repo.id) ?? repo, f.user);
34 const sources = (): FollowSource[] => (may() ? listSources(f.db, repo.id) : []);
35
36 const rows = (): ListRow[] =>
37 sources().map((s) => {
38 const stamp = stampOf(s);
39 return {
40 title: `${upstreamShort(s.url)} ${s.branch}`,
41 meta: `${stamp === undefined ? "" : `${stamp.account}@${stamp.host} · `}${sourceStanding(s)}`,
42 };
43 });
44
45 const screen: Screen = {
46 title: "Followed sources",
47 keys: [binding(["j", "k"], "move cursor"), binding(["a"], "follow"), binding(["x"], "stop following"), binding(["u"], "check now"), binding(["esc"], "close")],
48 load(signal) {
49 live(f, signal, { repoId: repo.id, kinds: ["repo.source_added", "repo.source_removed", "repo.upstream_checked"] }, () => ctx.invalidate());
50 return Promise.resolve();
51 },
52 render(area, s) {
53 const c = ctx.theme.c;
54 const [head, body] = split(area, "v", [len(1), fill(1)]);
55 const text = may()
56 ? "A trusted account's commits fold in as one pull; anyone else's wait as a guest."
57 : "Only an admin of this repository changes what it follows.";
58 s.text(head!.x, head!.y, truncate(text, head!.w), style(c.mutedFg, c.bg), head!.x + head!.w);
59 drawList(s, body!, c, state, rows());
60 },
61 key(e) {
62 if (!may() || e.ctrl || e.alt) return false;
63 if (state.key(e, sources().length)) return true;
64 const user = f.user;
65 const at = sources()[state.index];
66 if (e.key === "a") {
67 ctx.push(
68 promptScreen(ctx, screen, {
69 title: "Follow a source",
70 label: "address [branch] [as account], e.g. https://git.open-platform.sh/jmoran/forge.git main",
71 hint: "address, branch, trusted account",
72 onSubmit: (raw) => {
73 const done = addFollowSources(f.app, user, repo, raw.split("\n")[0] ?? "");
74 if (!done.ok) return SOURCE_REFUSALS.not_found;
75 const one = done.value[0]?.result;
76 if (one === undefined || !one.ok) return SOURCE_REFUSALS[one?.error ?? "invalid_url"];
77 ctx.toast(`following ${upstreamShort(one.value.url)} ${one.value.branch}`, "success");
78 ctx.invalidate();
79 return undefined;
80 },
81 }),
82 );
83 return true;
84 }
85 if (e.key === "u") {
86 ctx.toast("checking the sources…", "default");
87 runAsync(ctx, async () => {
88 const done = await checkSourcesNow(f.app, user, repo);
89 ctx.toast(done.ok ? `checked ${done.value.length} source${done.value.length === 1 ? "" : "s"}` : SOURCE_REFUSALS.not_found, done.ok ? "success" : "danger");
90 ctx.invalidate();
91 });
92 return true;
93 }
94 if (at === undefined) return false;
95 if (e.key === "x") {
96 ctx.push(
97 confirmScreen(ctx, screen, {
98 title: "Stop following",
99 body: [`${upstreamShort(at.url)} ${at.branch} will no longer be checked.`],
100 actions: [{ label: "Keep", variant: "ghost" }, { label: "Stop following", variant: "default" }],
101 onChoose: (i) => {
102 if (i !== 1) return;
103 const gone = removeFollowSource(f.app, user, repo, at.id);
104 ctx.toast(gone.ok ? "no longer following it" : SOURCE_REFUSALS.not_found, gone.ok ? "success" : "danger");
105 ctx.invalidate();
106 },
107 }),
108 );
109 return true;
110 }
111 return false;
112 },
113 };
114 return screen;
115 }
src/tui/screens/trusted.ts +2 −2
@@ -9,7 +9,7 @@
99 */
1010
1111 import { addTrust, removeTrust, revertBySource, setFoldInStopped } from "../../actions/trusted_sources.ts";
12 import { endsSoon, foldInStopped, listTrusted, type TrustError } from "../../core/trusted_sources.ts";
12 import { endsSoon, foldInStopped, leftWords, listTrusted, type TrustError } from "../../core/trusted_sources.ts";
1313 import type { Ctx, Screen } from "../app/app.ts";
1414 import { binding } from "../app/keys.ts";
1515 import { style } from "../term/style.ts";
@@ -40,7 +40,7 @@ export function trustedScreen(f: Session, ctx: Ctx): Screen {
4040 meta:
4141 s.until <= now.toISOString()
4242 ? "lapsed, waits for review"
43 : `until ${s.until.slice(0, 10)}${endsSoon(s, now) ? " (ends soon)" : ""}`,
43 : `until ${s.until.slice(0, 10)} (${leftWords(s.until, now)})${endsSoon(s, now) ? ", ends soon" : ""}`,
4444 }));
4545 };
4646
src/web/live.ts +1 −1
@@ -304,7 +304,7 @@ export const liveClientJs = /* js */ `
304304 'label.created', 'label.updated', 'label.deleted',
305305 'run.queued', 'run.started', 'run.finished', 'app.serving',
306306 'app.created', 'app.updated', 'app.deleted', 'app.stopped',
307 'repo.upstream_checked', 'repo.upstream_follow', 'repo.following', 'limb.moved', 'limb.replan', 'limb.replan_taken_back', 'issue.assigned', 'pull.assigned', 'crew.added', 'crew.removed',
307 'repo.upstream_checked', 'repo.source_added', 'repo.source_removed', 'repo.upstream_follow', 'repo.following', 'limb.moved', 'limb.replan', 'limb.replan_taken_back', 'issue.assigned', 'pull.assigned', 'crew.added', 'crew.removed',
308308 ];
309309 var extra = (cfg.getAttribute('data-repos') || '').split(' ').filter(Boolean);
310310 var watching = [repo].concat(extra).filter(function (r, i, all) {
src/web/pages/settings.ts +3 −2
@@ -108,7 +108,7 @@ import { cutWaveFor } from "../../actions/cut_wave.ts";
108108 import { proposalTokensSection } from "./proposal-tokens.ts";
109109 import { secretsSection } from "./secrets.ts";
110110 import { checkMemoryAt } from "../../apps/check-memory.ts";
111 import { followUpstreamCard } from "./upstream-follow.ts";
111 import { followSourcesCard, followUpstreamCard } from "./upstream-follow.ts";
112112
113113 type Section =
114114 | "general"
@@ -370,6 +370,7 @@ export async function settingsPage(ctx: Ctx): Promise<Response> {
370370 );
371371 const secrets = secretsSection(ctx, "repo", `${repo.ownerName}/${repo.name}`);
372372 const upstream = followUpstreamCard(ctx, repo);
373 const sources = followSourcesCard(ctx, repo);
373374 const wave = card(
374375 html`<form method="post" action="${base}/settings/wave">
375376 <p class="muted">
@@ -426,7 +427,7 @@ export async function settingsPage(ctx: Ctx): Promise<Response> {
426427 "Settings",
427428 html`${lead}${mirror ? html`<div class="mt">${mirror}</div>` : null}<div class="mt">${body}</div>${trail
428429 ? html`<div class="mt">${trail}</div>`
429 : null}${upstream ? html`<div class="mt">${upstream}</div>` : null}<div class="mt">${wave}</div><div class="mt">${archive}</div>${proposals
430 : null}${upstream ? html`<div class="mt">${upstream}</div>` : null}${sources ? html`<div class="mt">${sources}</div>` : null}<div class="mt">${wave}</div><div class="mt">${archive}</div>${proposals
430431 ? html`<div class="mt">${proposals}</div>`
431432 : null}${secrets
432433 ? html`<div class="mt">${secrets}</div>`
src/web/pages/trusted.ts +5 −4
@@ -7,6 +7,7 @@ import { addTrust, removeTrust, revertBySource, setFoldInStopped } from "../../a
77 import {
88 endsSoon,
99 foldInStopped,
10 leftWords,
1011 listTrusted,
1112 type TrustError,
1213 } from "../../core/trusted_sources.ts";
@@ -53,7 +54,7 @@ export function trustedPage(ctx: Ctx): Response {
5354 <input type="hidden" name="stop" value="${stopped ? "0" : "1"}" />
5455 ${stopped
5556 ? html`${badge("Folding in is stopped", { variant: "outline" })}
56 <span class="muted small">by ${stopped.by}, <time datetime="${stopped.at}">${stopped.at.slice(0, 10)}</time></span>
57 <span class="muted small">by ${stopped.by}, ${time(stopped.at, { date: true })}</span>
5758 ${btn("Resume folding in", { size: "sm" })}`
5859 : btn("Stop all folding in", { size: "sm", variant: "outline" })}
5960 </form></div>`,
@@ -68,9 +69,9 @@ export function trustedPage(ctx: Ctx): Response {
6869 return row({
6970 lead: icon("user"),
7071 title: html`${s.account} <span class="muted small mono">${s.host}</span>`,
71 meta: html`${lapsed ? "Lapsed, waits for review" : "Until"} ${time(s.until)}${
72 !lapsed && endsSoon(s, now) ? " (ends soon)" : ""
73 } · set by ${s.setBy}`,
72 meta: html`${lapsed ? "Lapsed, waits for review, since" : "Until"}
sentinelcrew

Hand-rolled <time> markup replaces the time() component. DESIGN.md wants the named component; extend it rather than inlining.
73 ${time(s.until, { date: true })}
74 (${leftWords(s.until, now)})${!lapsed && endsSoon(s, now) ? ", ends soon" : ""} · set by ${s.setBy}`,
7475 trail: html`<form class="row-fields" method="post" action="/settings/trusted/${s.id}/remove">
7576 ${btn("Remove", { size: "sm", variant: "outline", label: `Remove ${s.account} on ${s.host}` })}
7677 </form>
src/web/pages/upstream-follow.ts +118
@@ -16,8 +16,18 @@ import {
1616 setFollowUpstream,
1717 standing,
1818 upstreamLabel,
19 upstreamShort,
1920 } from "../../actions/upstream_follow.ts";
21 import {
22 addFollowSources,
23 checkSourcesNow,
24 removeFollowSource,
25 SOURCE_REFUSALS,
26 sourceStanding,
27 } from "../../actions/follow_sources.ts";
28 import { listSources, stampOf, type FollowSource } from "../../core/follow_sources.ts";
2029 import { canAdmin, type Repo } from "../../core/repos.ts";
30 import { getProvenance } from "../../core/provenance.ts";
2131 import { getFollow, listChecks } from "../../core/upstream_follow.ts";
2232 import type { Ctx } from "../../http/context.ts";
2333 import { loadRepo, repoBase, repoNotFound } from "../chrome.ts";
@@ -174,3 +184,111 @@ export async function followUpstreamCheck(ctx: Ctx): Promise<Response> {
174184 : { tone: "danger", message: FOLLOW_REFUSALS[checked.error] },
175185 );
176186 }
187
188 /** One source on one line: where, which branch, who it is trusted as, where its last check stands. */
189 function sourceLine(base: string, s: FollowSource): Html {
190 const stamp = stampOf(s);
191 const stand = sourceStanding(s);
192 return row({
193 lead: icon("sync"),
194 title: html`<a href="${s.url}" rel="noreferrer noopener">${upstreamShort(s.url)}</a> <span class="mono">${s.branch}</span>`,
195 meta: html`${stamp === undefined ? "" : `${stamp.account}@${stamp.host}`} · ${stand}${
196 s.checkedAt === null ? "" : html` · checked ${clock(s.checkedAt)}`
197 }`,
198 trail: html`<form method="post" action="${base}/settings/sources/${s.id}/remove" class="row-fields">
199 ${btn("Remove", { size: "sm", variant: "outline", label: `Stop following ${upstreamShort(s.url)}` })}
200 </form>`,
201 });
202 }
203
204 /** The settings card: every source on a line, and one box to add as many as you like. */
205 export function followSourcesCard(ctx: Ctx, repo: Repo): Html | null {
206 if (!canAdmin(ctx.app.db, repo, ctx.auth.user)) return null;
207 const sources = listSources(ctx.app.db, repo.id);
208 const base = repoBase(repo);
209 // A migrated repository's link back to where it came from is offered as the first line (#3293).
210 const from = sources.length === 0 ? getProvenance(ctx.app.db, repo.id)?.sourceUrl : undefined;
211 return card(
212 html`<ul class="list" data-follow-sources>
213 ${sources.length === 0
214 ? html`<li class="muted small">Following no other sources.</li>`
215 : sources.map((s) => sourceLine(base, s))}
216 </ul>
217 <form method="post" action="${base}/settings/sources" class="stack mt">
218 <p class="muted small">
219 A new commit on a source whose account is <a href="/settings/trusted">trusted</a> opens or updates
220 one pull, stamped with where it came from; a conflict goes to the crew. Anyone else's waits as a guest.
221 </p>
222 ${formField({
223 label: "Sources, one per line",
224 name: "sources",
225 type: "textarea",
226 rows: 3,
227 ...(from === undefined ? {} : { value: `${from} main` }),
228 mono: true,
229 spellcheck: false,
230 placeholder: "https://git.open-platform.sh/jmoran/forge-jay.git main",
231 hint: "Address, then the branch (main when left out). A forge admin may add “as account” to name the trusted account, e.g. “… main as benjivers”.",
232 })}
233 <div class="form-actions">${btn("Follow")}</div>
234 </form>
235 ${when(
236 sources.length > 0,
237 () => html`<form method="post" action="${base}/settings/sources/check" class="mt">
238 ${btn("Check sources now", { size: "sm", icon: "sync" })}
239 </form>`,
240 )}`,
241 { head: html`${icon("sync")}<h2 id="sources">Followed sources</h2>`, pad: sources.length === 0 },
242 );
243 }
244
245 // POST /:owner/:name/settings/sources
246 export async function followSourcesAdd(ctx: Ctx): Promise<Response> {
247 const repo = loadRepo(ctx);
248 if (!repo || !canAdmin(ctx.app.db, repo, ctx.auth.user)) return repoNotFound(ctx);
249 const form = await readForm(ctx, "admin");
250 if (form instanceof Response) return form;
251 const done = addFollowSources(ctx.app, ctx.auth.user, repo, field(form, "sources"));
252 if (!done.ok) return repoNotFound(ctx);
253 const bad = done.value.find((d) => !d.result.ok);
254 const added = done.value.filter((d) => d.result.ok).length;
255 return flashTo(
256 ctx,
257 `${repoBase(repo)}/settings#sources`,
258 done.value.length === 0
259 ? { tone: "danger", message: "Give at least one address." }
260 : bad !== undefined && !bad.result.ok
261 ? { tone: "danger", message: `${added} added. “${bad.line.slice(0, 80)}”: ${SOURCE_REFUSALS[bad.result.error]}` }
262 : { message: `Following ${added} source${added === 1 ? "" : "s"}. The first check runs within a minute.` },
263 );
264 }
265
266 // POST /:owner/:name/settings/sources/:id/remove
267 export async function followSourceRemove(ctx: Ctx): Promise<Response> {
268 const repo = loadRepo(ctx);
269 if (!repo || !canAdmin(ctx.app.db, repo, ctx.auth.user)) return repoNotFound(ctx);
270 const form = await readForm(ctx, "admin");
271 if (form instanceof Response) return form;
272 const gone = removeFollowSource(ctx.app, ctx.auth.user, repo, ctx.params.id ?? "");
273 return flashTo(
274 ctx,
275 `${repoBase(repo)}/settings#sources`,
276 gone.ok
277 ? { message: `No longer following ${upstreamShort(gone.value.url)}.` }
278 : { tone: "danger", message: SOURCE_REFUSALS.not_found },
279 );
280 }
281
282 // POST /:owner/:name/settings/sources/check
283 export async function followSourcesCheck(ctx: Ctx): Promise<Response> {
284 const repo = loadRepo(ctx);
285 if (!repo || !canAdmin(ctx.app.db, repo, ctx.auth.user)) return repoNotFound(ctx);
286 const form = await readForm(ctx, "admin");
287 if (form instanceof Response) return form;
288 const checked = await checkSourcesNow(ctx.app, ctx.auth.user, repo);
289 if (!checked.ok) return repoNotFound(ctx);
290 const lines = checked.value.map((c) =>
291 c.result.ok ? `${upstreamShort(c.source.url)}: ${checkedWords(c.result.value)}` : `${upstreamShort(c.source.url)}: ${FOLLOW_REFUSALS[c.result.error]}`,
292 );
293 return flashTo(ctx, `${repoBase(repo)}/settings#sources`, { message: lines.join(" ") || "No sources to check." });
294 }
src/web/ui.ts +4 −2
@@ -50,14 +50,16 @@ export function timeAgo(iso: string): string {
5050 /**
5151 * A timestamp a machine and a screen reader can both read: relative text for
5252 * the eye, the exact instant in `datetime` and `title` for everyone else.
53 * `date` words it as the calendar day (`2026-12-02`), for an instant ahead
54 * of now, which "ago" cannot name.
5355 */
54 export function time(iso: string): Html {
56 export function time(iso: string, opts: { date?: boolean } = {}): Html {
5557 const parsed = Date.parse(iso);
5658 if (Number.isNaN(parsed)) return html``;
5759 const at = new Date(parsed);
5860 const exact = `${at.toISOString().slice(0, 16).replace("T", " ")} UTC`;
5961 return html`<time datetime="${at.toISOString()}" title="${exact}"
60 >${timeAgo(iso)}</time
62 >${opts.date === true ? at.toISOString().slice(0, 10) : timeAgo(iso)}</time
6163 >`;
6264 }
6365
test/follow-sources.test.ts added +439
@@ -0,0 +1,439 @@
1 /**
2 * A repository follows several sources (#9003): the trust list shows its date
3 * and how long is left; a source of a trusted account opens one stamped pull
4 * and moves it as the source moves; a conflict goes to the crew once; a big
5 * first fold waits for the crew's review. Sources are read from a fake
6 * Forgejo (a dumb-http `info/refs`), with the tip staged as a fetch leaves it.
7 */
8 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
9 import { mkdtemp, rm } from "node:fs/promises";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12 import { addToCrew, createAgent } from "../src/actions/crew.ts";
13 import { addFollowSources, parseSourceLine, removeFollowSource } from "../src/actions/follow_sources.ts";
14 import { openPull } from "../src/actions/pulls.ts";
15 import { createRepo } from "../src/actions/repos.ts";
16 import { addTrust } from "../src/actions/trusted_sources.ts";
17 import { checkDue, checkSource } from "../src/actions/upstream_follow.ts";
18 import { reviewed } from "../src/crew/autonomy.ts";
19 import { createApp, type App } from "../src/app.ts";
20 import { sourcesScreen } from "../src/tui/screens/sources.ts";
21 import type { Ctx, Screen } from "../src/tui/app/app.ts";
22 import type { Session } from "../src/tui/session.ts";
23 import { listSources, openConflictOf, sourceRef } from "../src/core/follow_sources.ts";
24 import { listIssues } from "../src/core/issues.ts";
25 import { getPull } from "../src/core/pulls.ts";
26 import { recordProvenance } from "../src/core/provenance.ts";
27 import { getRepoById, updateRepoSettings } from "../src/core/repos.ts";
28 import { createSession } from "../src/core/sessions.ts";
29 import {
30 adoptResolution,
31 BIG_FOLD_COMMITS,
32 foldsIn,
33 isTrusted,
34 leftWords,
35 needsCrewReview,
36 pullSourceOf,
37 trustSource,
38 } from "../src/core/trusted_sources.ts";
39 import { createUser, type User } from "../src/core/users.ts";
40 import { runGit } from "../src/git/env.ts";
41 import { repoDir } from "../src/git/repos.ts";
42 import { serve } from "../src/http/server.ts";
43 import { mirrorOutbound, quietLogs } from "./helpers/globals.ts";
44
45 quietLogs();
46 mirrorOutbound({ allow: ["127.0.0.1"] });
47
48 const ok = <T>(r: { ok: true; value: T } | { ok: false; error: unknown }): T => {
49 if (!r.ok) throw new Error(JSON.stringify(r.error));
50 return r.value;
51 };
52
53 let dataDir = "";
54 let app: App;
55 let alice: User;
56 let guest: User;
57
58 beforeAll(async () => {
59 dataDir = await mkdtemp(join(tmpdir(), "forge-sources-"));
60 app = createApp({ dataDir, port: 0, host: "forge.test" });
61 const user = (username: string, extra: object = {}) =>
62 createUser(app.db, {
63 origin: "seed",
64 trusted: true,
65 username,
66 email: `${username}@forge.local`,
67 password: "a-good-passphrase",
68 ...extra,
69 });
70 alice = ok(await user("alice", { adminIfFirst: true }));
71 guest = ok(await user("guest"));
72 });
73
74 afterAll(async () => {
75 app?.close();
76 await rm(dataDir, { recursive: true, force: true });
77 });
78
79 describe("a trust shows its date and what is left", () => {
80 const now = new Date("2026-11-03T01:26:00Z");
81
82 test("a date-only entry is active, not already lapsed", () => {
83 const made = trustSource(app.db, { host: "git.open-platform.sh", account: "jmoran", until: "2026-12-02", setBy: "alice" }, now);
84 if (typeof made === "string") throw new Error(made);
85 expect(made.until).toBe("2026-12-02T23:59:59.999Z");
86 expect(isTrusted(app.db, "git.open-platform.sh", "jmoran", now)).toBe(true);
87 expect(leftWords(made.until, now)).toBe("29 days left");
88 });
89
90 test("the words for the last day, and for an end that has passed", () => {
91 expect(leftWords("2026-11-03T20:00:00.000Z", now)).toBe("less than a day left");
92 expect(leftWords("2026-11-04T23:59:59.999Z", now)).toBe("1 day left");
93 expect(leftWords("2026-11-02T23:59:59.999Z", now)).toBe("lapsed");
94 expect(leftWords("nonsense", now)).toBe("lapsed");
95 });
96
97 for (const bad of ["", "tomorrow", "12/02/2026", "2026-12-02T10:00:00Z", "2026-02-31", "2026-13-01", "2026-00-10"])
98 test(`refuses ${JSON.stringify(bad)}`, () => {
99 expect(trustSource(app.db, { host: "h.test", account: "x", until: bad, setBy: "t" }, now)).toBe("bad_date");
100 });
101
102 test("the settings page says the date and the days left, never 'just now'", async () => {
103 ok(addTrust(app, alice, { host: "forge-apps.benjivers.com", account: "benjivers", until: "2099-12-02" }));
104 const servers = await serve(app);
105 try {
106 const res = await fetch(`${servers.url}/settings/trusted`, {
107 headers: { cookie: `forge_session=${createSession(app.db, alice.id).token}` },
108 });
109 const text = await res.text();
110 expect(res.status).toBe(200);
111 expect(text).toContain("2099-12-02");
112 expect(text).toMatch(/\(\d+ days left\)/);
113 expect(text).not.toContain("just now");
114 } finally {
115 servers.stop();
116 }
117 });
118 });
119
120 describe("the add form's lines", () => {
121 test("an address, a branch, an account", () => {
122 expect(ok(parseSourceLine("https://git.open-platform.sh/jmoran/forge-jay.git"))).toEqual({
123 url: "https://git.open-platform.sh/jmoran/forge-jay.git",
124 branch: "main",
125 account: "",
126 });
127 expect(ok(parseSourceLine("forge-apps.benjivers.com/apps/foundry.git dev as Benjivers"))).toEqual({
128 url: "https://forge-apps.benjivers.com/apps/foundry.git",
129 branch: "dev",
130 account: "benjivers",
131 });
132 });
133 for (const bad of ["", "git://x.test/a/b", "ssh://[email protected]/a/b", "https://u:[email protected]/a/b", "https://x.test/a/b main extra", "https://x.test/a/b as"])
134 test(`refuses ${JSON.stringify(bad)}`, () => {
135 expect(parseSourceLine(bad).ok).toBe(false);
136 });
137 test("a branch git would take for a flag is refused", () => {
138 expect(parseSourceLine("https://x.test/a/b --upload-pack=x")).toEqual({ ok: false, error: "invalid_branch" });
139 });
140 });
141
142 describe("following several sources", () => {
143 const ident = { GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@x", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@x" };
144
145 /** A fake Forgejo: answers `info/refs` for each path with the tip it is given. */
146 function forgejo(tips: Map<string, string>) {
147 const server = Bun.serve({
148 port: 0,
149 hostname: "127.0.0.1",
150 fetch(req) {
151 const url = new URL(req.url);
152 const path = url.pathname.replace(/\/info\/refs$/, "");
153 const tip = tips.get(path);
154 return url.pathname.endsWith("/info/refs") && tip !== undefined
155 ? new Response(`${tip}\trefs/heads/main\n`, { headers: { "content-type": "text/plain" } })
156 : new Response("no", { status: 404 });
157 },
158 });
159 return { server, host: `127.0.0.1:${server.port}`, url: (path: string) => `http://127.0.0.1:${server.port}${path}` };
160 }
161
162 /** A repository whose main holds f.txt = `mainText`, and a commit tree builder. */
163 async function repoWith(name: string, mainText: string) {
164 const r = ok(await createRepo(app, alice, { name, visibility: "public" }));
165 const d = repoDir(app.reposRoot, r.ownerName, r.name);
166 const git = async (args: string[], input?: string) =>
167 ok(await runGit(args, { cwd: d, env: ident, ...(input === undefined ? {} : { stdin: input }) })).trim();
168 const commit = async (text: string, parent?: string) => {
169 const blob = await git(["hash-object", "-w", "--stdin"], text);
170 const tree = await git(["mktree"], `100644 blob ${blob}\tf.txt\n`);
171 return git(["commit-tree", tree, ...(parent ? ["-p", parent] : []), "-m", text]);
172 };
173 const base = await commit("base\n");
174 const main = mainText === "base\n" ? base : await commit(mainText, base);
175 await git(["update-ref", `refs/heads/${r.defaultBranch}`, main]);
176 return { r, d, git, commit, base };
177 }
178
179 /** Follow `path` on the fake forge and stage `tip` as a fetch of it would have. */
180 async function follow(t: Awaited<ReturnType<typeof repoWith>>, f: ReturnType<typeof forgejo>, path: string, tip: string) {
181 const line = `${f.url(path)} main`;
182 const added = ok(addFollowSources(app, alice, getRepoById(app.db, t.r.id)!, line));
183 const source = ok(added[0]!.result);
184 await t.git(["update-ref", sourceRef(source.id), tip]);
185 return source;
186 }
187
188 test("several sources are added in one action, one line each; a bad line is named and the rest still follow", async () => {
189 const t = await repoWith("many", "base\n");
190 const done = ok(
191 addFollowSources(
192 app,
193 alice,
194 t.r,
195 [
196 "https://git.open-platform.sh/jmoran/forge-jay.git",
197 "https://git.open-platform.sh/tdobbertin-espo/forge.git main",
198 "git://nope.test/a/b",
199 "https://git.open-platform.sh/jmoran/forge-jay.git",
200 "https://forge-apps.benjivers.com/apps/foundry.git main as benjivers",
201 ].join("\n"),
202 ),
203 );
204 expect(done.map((d) => (d.result.ok ? "ok" : d.result.error))).toEqual(["ok", "ok", "invalid_url", "exists", "ok"]);
205 const have = listSources(app.db, t.r.id);
206 expect(have.length).toBe(3);
207 expect(have[2]!.account).toBe("benjivers");
208 });
209
210 test("only a repository admin adds or removes; only a forge admin names the account", () => {
211 const r = getRepoById(app.db, listSources(app.db, getRepoByName("many")).at(0)!.repoId)!;
212 expect(addFollowSources(app, guest, r, "https://x.test/a/b")).toEqual({ ok: false, error: "not_found" });
213 expect(addFollowSources(app, undefined, r, "https://x.test/a/b")).toEqual({ ok: false, error: "not_found" });
214 const first = listSources(app.db, r.id)[0]!;
215 expect(removeFollowSource(app, guest, r, first.id)).toEqual({ ok: false, error: "not_found" });
216 expect(removeFollowSource(app, alice, r, first.id).ok).toBe(true);
217 expect(listSources(app.db, r.id).length).toBe(2);
218 });
219
220 const pullCount = (repoId: string): number =>
221 app.db.query<{ n: number }, [string]>("SELECT COUNT(*) AS n FROM issue WHERE repo_id = ?1 AND kind = 'pull'").get(repoId)!.n;
222
223 function getRepoByName(name: string): string {
224 return app.db.query<{ id: string }, [string]>("SELECT id FROM repo WHERE name = ?1").get(name)!.id;
225 }
226
227 test("a trusted account's new commit opens one stamped pull; the next moves it; a guest's pull is stamped and waits", async () => {
228 const t = await repoWith("fold", "base\n");
229 const up = await t.commit("new\n", t.base);
230 const tips = new Map([["/jmoran/forge-jay.git", up]]);
231 const f = forgejo(tips);
232 try {
233 ok(addTrust(app, alice, { host: f.host, account: "jmoran", until: "2099-01-01" }));
234 const source = await follow(t, f, "/jmoran/forge-jay.git", up);
235 const first = ok(await checkSource(app, getRepoById(app.db, t.r.id)!, source));
236 expect(first.outcome).toBe("proposed");
237 const pull = getPull(app.db, t.r.id, first.pull!)!;
238 expect(pull.title).toContain("1 new commit");
239 expect(pullSourceOf(app.db, pull.id)).toMatchObject({ host: f.host, account: "jmoran" });
240 expect(foldsIn(app.db, pull.id)).toBe(true);
241 expect(needsCrewReview(app.db, pull.id)).toBe(false);
242
243 // The source moves: the same pull follows it.
244 const up2 = await t.commit("newer\n", up);
245 tips.set("/jmoran/forge-jay.git", up2);
246 await t.git(["update-ref", sourceRef(source.id), up2]);
247 const second = ok(await checkSource(app, getRepoById(app.db, t.r.id)!, listSources(app.db, t.r.id)[0]!));
248 expect(second.pull).toBe(first.pull);
249 expect(getPull(app.db, t.r.id, first.pull!)!.headOid).toBe(up2);
250 expect(listSources(app.db, t.r.id)[0]!.opened).toBe(1);
251 expect(await t.git(["rev-parse", `refs/pull/${first.pull}/head`])).toBe(up2);
252
253 // Another account on the same forge is a guest: a pull, stamped, never folded in.
254 const other = await repoWith("guestfold", "base\n");
255 const gup = await other.commit("theirs\n", other.base);
256 tips.set("/stranger/x.git", gup);
257 const gs = await follow(other, f, "/stranger/x.git", gup);
258 const g = ok(await checkSource(app, getRepoById(app.db, other.r.id)!, gs));
259 const gpull = getPull(app.db, other.r.id, g.pull!)!;
260 expect(pullSourceOf(app.db, gpull.id)?.account).toBe("stranger");
261 expect(foldsIn(app.db, gpull.id)).toBe(false);
262 } finally {
263 f.server.stop(true);
264 }
265 });
266
267 test("two sources of one repository open a pull each, and the clock checks both", async () => {
268 const t = await repoWith("two", "base\n");
269 const a = await t.commit("a\n", t.base);
270 const b = await t.commit("b\n", t.base);
271 const f = forgejo(new Map([["/jmoran/a.git", a], ["/benji/b.git", b]]));
272 try {
273 const sa = await follow(t, f, "/jmoran/a.git", a);
274 const sb = await follow(t, f, "/benji/b.git", b);
275 expect(sa.id).not.toBe(sb.id);
276 await checkDue(app);
277 const after = listSources(app.db, t.r.id);
278 expect(after.map((s) => s.opened)).toEqual([1, 1]);
279 expect(new Set(after.map((s) => s.pendingId)).size).toBe(2);
280 } finally {
281 f.server.stop(true);
282 }
283 });
284
285 test("a first fold of many commits is one pull that the crew reviews, not a flood", async () => {
286 const t = await repoWith("big", "base\n");
287 let tip = t.base;
288 for (let n = 0; n < BIG_FOLD_COMMITS; n++) tip = await t.commit(`c${n}\n`, tip);
289 const f = forgejo(new Map([["/jmoran/big.git", tip]]));
290 try {
291 ok(addTrust(app, alice, { host: f.host, account: "jmoran", until: "2099-01-01" }));
292 const s = await follow(t, f, "/jmoran/big.git", tip);
293 const r = ok(await checkSource(app, getRepoById(app.db, t.r.id)!, s));
294 expect(r.behind).toBe(BIG_FOLD_COMMITS);
295 const pull = getPull(app.db, t.r.id, r.pull!)!;
296 expect(needsCrewReview(app.db, pull.id)).toBe(true);
297 // Trusted, but big: the crew's review stands before it lands; a small fold of the same source passes.
298 const crewPull = { ...pull, authorIsAgent: true };
299 expect(foldsIn(app.db, pull.id)).toBe(true);
300 expect(await reviewed(app, getRepoById(app.db, t.r.id)!, crewPull)).toBe("hold");
301 app.db.query("UPDATE pull_source SET big = 0 WHERE pull_id = ?1").run(pull.id);
302 expect(await reviewed(app, getRepoById(app.db, t.r.id)!, crewPull)).toBe("pass");
303 app.db.query("UPDATE pull_source SET big = 1 WHERE pull_id = ?1").run(pull.id);
304 expect(pullCount(t.r.id)).toBe(1);
305 ok(await checkSource(app, getRepoById(app.db, t.r.id)!, listSources(app.db, t.r.id)[0]!));
306 expect(pullCount(t.r.id)).toBe(1);
307 } finally {
308 f.server.stop(true);
309 }
310 });
311
312 test("a conflict goes to the crew once; the crew's resolving pull carries the source, and only an agent's", async () => {
313 const t = await repoWith("clash", "ours\n");
314 const up = await t.commit("theirs\n", t.base);
315 const f = forgejo(new Map([["/jmoran/clash.git", up]]));
316 try {
317 ok(addTrust(app, alice, { host: f.host, account: "jmoran", until: "2099-01-01" }));
318 const bot = ok(await createAgent(app, alice, { username: "bot-clash", brief: "Be brief." }));
319 expect(addToCrew(app, alice, getRepoById(app.db, t.r.id)!, "bot-clash").ok).toBe(true);
320 updateRepoSettings(app.db, t.r, { autonomy: "assisted", autonomyBy: alice.id });
321 const s = await follow(t, f, "/jmoran/clash.git", up);
322 const r = getRepoById(app.db, t.r.id)!;
323 expect(ok(await checkSource(app, r, s)).outcome).toBe("conflict");
324 const issues = listIssues(app.db, t.r.id, { state: "all" }).filter((i) => i.title.startsWith("Upstream: conflict at "));
325 expect(issues.length).toBe(1);
326 expect(openConflictOf(app.db, s.id)).toBe(true);
327
328 // The source moves on while the conflict is open: no second issue.
329 const up2 = await t.commit("theirs again\n", up);
330 await t.git(["update-ref", sourceRef(s.id), up2]);
331 await checkSource(app, r, listSources(app.db, t.r.id)[0]!);
332 expect(listIssues(app.db, t.r.id, { state: "all" }).filter((i) => i.title.startsWith("Upstream: conflict at ")).length).toBe(1);
333
334 // A guest's pull that says it closes the issue gains nothing; the agent's does.
335 const resolved = await t.commit("resolved\n", up);
336 await t.git(["update-ref", "refs/heads/bot/resolve", resolved]);
337 await t.git(["update-ref", "refs/heads/guest/resolve", resolved]);
338 const mine = ok(await openPull(app, bot.user, r, { title: "Resolve", body: `Closes #${issues[0]!.number}`, baseBranch: r.defaultBranch, headBranch: "bot/resolve" }));
339 expect(foldsIn(app.db, mine.id)).toBe(true);
340 expect(pullSourceOf(app.db, mine.id)).toMatchObject({ host: f.host, account: "jmoran" });
341 expect(needsCrewReview(app.db, mine.id)).toBe(true);
342 const theirs = await openPull(app, guest, r, { title: "Sneak", body: `Closes #${issues[0]!.number}`, baseBranch: r.defaultBranch, headBranch: "guest/resolve" });
343 if (!theirs.ok) throw new Error(theirs.error);
344 expect(foldsIn(app.db, theirs.value.id)).toBe(false);
345 expect(pullSourceOf(app.db, theirs.value.id)).toBeUndefined();
346 // A read changes nothing: an unstamped agent pull is not adopted by asking; adoption is explicit.
347 app.db.query("DELETE FROM pull_source WHERE pull_id = ?1").run(mine.id);
348 expect(foldsIn(app.db, mine.id)).toBe(false);
349 expect(pullSourceOf(app.db, mine.id)).toBeUndefined();
350 expect(adoptResolution(app.db, mine.id)).toMatchObject({ host: f.host, account: "jmoran" });
351 expect(adoptResolution(app.db, theirs.value.id)).toBeUndefined();
352 expect(foldsIn(app.db, mine.id)).toBe(true);
353 } finally {
354 f.server.stop(true);
355 }
356 });
357
358 test("the settings page lists each source on one line and adds through the form", async () => {
359 const t = await repoWith("pagey", "base\n");
360 const servers = await serve(app);
361 const cookie = `forge_session=${createSession(app.db, alice.id).token}`;
362 try {
363 const add = await fetch(`${servers.url}/${t.r.ownerName}/${t.r.name}/settings/sources`, {
364 method: "POST",
365 redirect: "manual",
366 headers: { cookie, origin: servers.url, "content-type": "application/x-www-form-urlencoded" },
367 body: new URLSearchParams({ sources: "https://git.open-platform.sh/jmoran/forge-jay.git main\nhttps://forge-apps.benjivers.com/apps/foundry.git main as benjivers" }),
368 });
369 await add.arrayBuffer();
370 expect(add.status).toBe(303);
371 expect(listSources(app.db, t.r.id).length).toBe(2);
372 const page = await (await fetch(`${servers.url}/${t.r.ownerName}/${t.r.name}/settings`, { headers: { cookie } })).text();
373 expect(page).toContain("jmoran/forge-jay");
374 expect(page).toContain("[email protected]");
375 expect(page).toContain("/settings/sources/");
376 const stranger = await fetch(`${servers.url}/${t.r.ownerName}/${t.r.name}/settings/sources`, {
377 method: "POST",
378 redirect: "manual",
379 headers: { cookie: `forge_session=${createSession(app.db, guest.id).token}`, origin: servers.url, "content-type": "application/x-www-form-urlencoded" },
380 body: new URLSearchParams({ sources: "https://x.test/a/b" }),
381 });
382 await stranger.arrayBuffer();
383 expect(stranger.status).toBe(404);
384 expect(listSources(app.db, t.r.id).length).toBe(2);
385 } finally {
386 servers.stop();
387 }
388 });
389
390 test("a migrated repository's settings offer its origin as the first line to follow", async () => {
391 const t = await repoWith("moved", "base\n");
392 expect(recordProvenance(app.db, t.r.id, { url: "https://git.open-platform.sh/jmoran/forge-jay", parents: [] })).toBe(true);
393 const servers = await serve(app);
394 try {
395 const page = await (
396 await fetch(`${servers.url}/${t.r.ownerName}/${t.r.name}/settings`, { headers: { cookie: `forge_session=${createSession(app.db, alice.id).token}` } })
397 ).text();
398 expect(page).toContain("https://git.open-platform.sh/jmoran/forge-jay main</textarea");
399 } finally {
400 servers.stop();
401 }
402 });
403
404 test("the terminal adds, lists and removes sources for a repository admin only, and a web viewer hears it", async () => {
405 const t = await repoWith("tui", "base\n");
406 const heard: string[] = [];
407 const off = app.bus.subscribe((e) => {
408 if (e.repoId === t.r.id) heard.push(e.kind);
409 });
410 const pushed: Screen[] = [];
411 const ctx = { invalidate() {}, toast() {}, push: (x: Screen) => pushed.push(x), pop() {}, copy() {} } as unknown as Ctx;
412 const screenFor = (user: User) => sourcesScreen({ app, db: app.db, user } as unknown as Session, ctx, getRepoById(app.db, t.r.id)!);
413 try {
414 const mine = screenFor(alice);
415 expect(mine.key!({ key: "a" } as never)).toBe(true);
416 const prompt = pushed.pop()!;
417 expect(prompt.title).toBe("Follow a source");
418 // Submit through the prompt as typed text.
419 for (const ch of "https://git.open-platform.sh/jmoran/forge-jay.git main") prompt.key!({ key: ch } as never);
420 prompt.key!({ key: "enter" } as never);
421 expect(listSources(app.db, t.r.id).length).toBe(1);
422 expect(heard).toContain("repo.source_added");
423 expect(mine.key!({ key: "x" } as never)).toBe(true);
424 const confirm = pushed.pop()!;
425 expect(confirm.title).toBe("Stop following");
426 confirm.key!({ key: "tab" } as never);
427 confirm.key!({ key: "enter" } as never);
428 expect(listSources(app.db, t.r.id).length).toBe(0);
429 expect(heard).toContain("repo.source_removed");
430
431 // Someone who is not an admin of the repository is offered nothing and changes nothing.
432 const theirs = screenFor(guest);
433 expect(theirs.key!({ key: "a" } as never)).toBe(false);
434 expect(pushed.length).toBe(0);
435 } finally {
436 off();
437 }
438 });
439 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs