Public: anyone can read this; the crew can change it.

v1.0.0 stays proven: the production probe runs weekly and files a defect when one fails #12996

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/12989-v1-0-0-stays-proven-the-production-probe → main 6 commits fork 2 threads · 2 unresolved

  • Merged. scout/12989-v1-0-0-stays-proven-the-production-probe was merged into main as ddbe238.

Review

Sign in to review this change.
6 files changed +245 −5
docs/features/v1-proven.md +19 −4
@@ -1,8 +1,23 @@
1 # Is v1.0.0 proven?
1 # v1.0.0 stays proven
22
3 ```sh
4 bun scripts/v1-proven.ts https://your-forge.example
5 ```
3 A weekly maintenance duty, `v1-proven` (#12989), runs the production probes of `scripts/v1-proven.ts` (the one list is
4 `src/core/v1_proof.ts`) against the forge's own public origin (`FORGE_URL`), `GET` only, no other host. Without
5 `FORGE_URL` it skips and says so.
6
7 - **What it probes.** One probe per box of the v1.0.0 proof. Each ends `pass`, `fail` or `unproven` (the forge could not
8 tell). Only `fail` counts: `unproven` files nothing.
9 - **Where the record shows.** Each run is a `maintenance_run` row. One line per probe sits in its actions
10 (`2026-11-15 fail #123 <item>`), a failing probe is also a finding. Read them with
11 `GET /api/v1/maintenance/runs` (admin token) or in the terminal's "Maintenance duty runs" screen. The web
12 `/maintenance` page is the power-off preparation and does not list duty runs.
13 - **A defect issue.** A failing probe opens one issue in the forge's own repository (`FORGE_REPO`, e.g. `owner/forge`),
14 titled `Defect of #<probe issue>: <item>`, whose body quotes the failing line and says to fix the cause and close it.
15 With no `FORGE_REPO` nothing is filed and the run says "no issue filed".
16 - **A repeat failure.** While that issue is open, the same probe failing again adds a comment ("Still failing on
17 <date>") and no second issue. Two probes failing in one run open two issues. Once the issue is closed, a new failure
18 opens a new one.
19 - **Limits.** Runs at most twice a day, weekly by default, and spends no money. The findings only land on the run record;
20 the runner files no notice of its own.
621
722 One read-only probe (GET only) for each item on the v1 list, one line each:
823
src/core/event_audience.ts +2
@@ -241,6 +241,8 @@ export function classifiedAudience(kind: string): EventAudience | undefined {
241241 kind.startsWith("forge.child_") ||
242242 // What a hosted forge asked to reach, was refused and was allowed is the owner's to answer (`spawn/child_reach.ts`).
243243 kind.startsWith("forge.reach_") ||
244 // A failed production probe names the forge's own defect: the admins' (`maintain/proven.ts`).
245 kind === "probe_defect" ||
244246 // A clog is the whole forge's flow, the owner's to hear (`flow/clog_tick.ts`).
245247 kind.startsWith("clog.") ||
246248 // What the throughput keeper did to lift a brake is the owner's too (`flow/keeper.ts`).
src/maintain/duties/v1_proven.ts added +39
@@ -0,0 +1,39 @@
1 /**
2 * v1.0.0 stays proven (#12989): the weekly read-only probe of the forge's public origin;
3 * see `../proven.ts`.
4 */
5
6 import { getRepo } from "../../core/repos.ts";
7 import type { Proof } from "../../core/v1_proof.ts";
8 import { publicOrigin } from "../../core/public_origin.ts";
9 import type { Duty, Finding } from "../duty.ts";
10 import { recordLines, reportFailures, weeklyProof } from "../proven.ts";
11
12 export function v1Proven(proof: Proof = weeklyProof): Duty {
13 return {
14 name: "v1-proven",
15 intervalMs: 7 * 86_400_000,
16 budget: { runsPerDay: 2, dollarsPerDay: 1, reason: "weekly is one a week; the rest is room for a retry, and it spends no money" },
17 async run({ db, now, app }) {
18 const url = app?.config.url;
19 if (url === undefined || url === "") return { findings: [], actions: ["skipped: this forge does not know its public origin (FORGE_URL)"] };
20 const results = await proof(publicOrigin(url, ""));
21 const findings: Finding[] = results
22 .filter((r) => r.status === "fail")
23 .map((r) => ({
24 summary: `v1 probe #${r.issue} fails: ${r.detail}`,
25 probe: { kind: "measurement", what: `probe #${r.issue} ${r.item}: 1 is a failure`, number: 1, at: now.toISOString() },
26 }));
27 const actions = recordLines(results, now);

Check that these findings do not make the maintenance runner file its own notice on top of the defect issue. If it does, return no findings here.

28 const [owner, name] = (app?.config.forgeRepo ?? "").split("/");
29 const repo = owner && name ? getRepo(db, owner, name) : undefined;
30 if (findings.length > 0) {
31 if (repo) actions.push(...reportFailures(db, repo.id, results, now));
32 else actions.push("no issue filed: the forge names no repository of its own");
33 }
34 return { findings, actions };
35 },
36 };
37 }
38
39 export const duty: Duty = v1Proven();
src/maintain/proven.ts added +65
@@ -0,0 +1,65 @@
1 /**
2 * v1.0.0 stays proven (#12989): the production probes of `scripts/v1-proven.ts` (the one list,
3 * `core/v1_proof.ts`) run once a week against the forge's own public origin, GET only. A failing
4 * probe opens one issue `Defect of #<probe issue>: …` quoting its line; a probe that fails again
5 * while that issue is open adds a comment instead. Plain data in, plain data out.
6 */
7
8 import type { Database } from "bun:sqlite";
9 import { EventBus } from "../core/events.ts";
10 import { addComment, createIssue, getIssueById } from "../core/issues.ts";
11 import { line, prove, type Proof, type Result } from "../core/v1_proof.ts";
12 import { watcherId } from "../rules/bot.ts";
13
14 export const defectTitle = (r: Pick<Result, "issue" | "item">): string => `Defect of #${r.issue}: ${r.item}`;
15
16 /** The proof a week runs: the probes over fetch, at the forge's own origin and no other host. */
17 export const weeklyProof: Proof = (base) => prove(base, fetch);
18
19 /** The open defect issue for a probe: the newest one noted that has not been closed since. */
20 function openDefect(db: Database, repoId: string, probeIssue: number): string | undefined {
21 const rows = db
22 .query<{ payload: string }, [string]>("SELECT payload FROM event WHERE repo_id = ?1 AND kind = 'probe_defect' ORDER BY seq DESC LIMIT 200")
23 .all(repoId);
24 for (const row of rows) {
25 try {
26 const p = JSON.parse(row.payload) as { probe?: unknown; issue?: unknown };
27 if (p.probe !== probeIssue || typeof p.issue !== "string") continue;
28 const issue = getIssueById(db, p.issue);
29 return issue?.state === "open" ? issue.id : undefined;
30 } catch {
31 // An unreadable note names no issue.
32 }
33 }
34 return undefined;
35 }
36
37 /** Files or comments on each failing result; returns one sentence per thing done. */
38 export function reportFailures(db: Database, repoId: string, results: readonly Result[], at: Date): string[] {
39 const said: string[] = [];
40 const bot = watcherId(db);
41 for (const r of results) {
42 if (r.status !== "fail") continue;
43 const quote = `> ${line(r).replace(/\r?\n/g, " ")}`;
44 const open = openDefect(db, repoId, r.issue);
45 if (open !== undefined) {
46 const c = addComment(db, { issueId: open, authorId: bot, body: `Still failing on ${at.toISOString().slice(0, 10)}:\n\n${quote}` });
47 if (c.ok) said.push(`commented on the open defect of #${r.issue}: still failing`);
48 continue;
49 }
50 const made = createIssue(db, {
51 repoId,
52 authorId: bot,
53 title: defectTitle(r),
54 body: `The weekly production probe (\`scripts/v1-proven.ts\`) failed on ${at.toISOString().slice(0, 10)}:\n\n${quote}\n\nFix the cause, then close this issue; a probe that fails again meanwhile is added here as a comment.`,
55 });
56 if (!made.ok) continue;
57 new EventBus(db).append({ kind: "probe_defect", repoId, payload: { probe: r.issue, issue: made.value.id } });
58 said.push(`opened issue #${made.value.number}: ${defectTitle(r)}`);
59 }
60 return said;
61 }
62
63 /** One line per probe, for the record: date, probe, pass/fail/unproven. */
64 export const recordLines = (results: readonly Result[], at: Date): string[] =>
65 results.map((r) => `${at.toISOString().slice(0, 10)} ${r.status} #${r.issue} ${r.item}`);
src/maintain/registry.ts +2 −1
@@ -6,5 +6,6 @@ import { duty as ruleWatcher } from "./duties/rule_watcher.ts";
66 import { duty as efficiencyWatcher } from "./duties/efficiency_watcher.ts";
77 import { duty as flowKeeper } from "./duties/flow_keeper.ts";
88 import { duty as visitorRetention } from "./duties/visitor_retention.ts";
9 import { duty as v1Proven } from "./duties/v1_proven.ts";
910
10 export const duties: readonly Duty[] = [pullsWaiting, ruleWatcher, efficiencyWatcher, flowKeeper, visitorRetention];
11 export const duties: readonly Duty[] = [pullsWaiting, ruleWatcher, efficiencyWatcher, flowKeeper, visitorRetention, v1Proven];
test/maintain-v1-proven.test.ts added +118
@@ -0,0 +1,118 @@
1 /**
2 * v1.0.0 stays proven (#12989): all pass, no issue; one fails, one issue; again, a comment.
3 */
4 import { afterAll, beforeAll, expect, test } from "bun:test";
5 import { mkdtemp, rm } from "node:fs/promises";
6 import { tmpdir } from "node:os";
7 import { join } from "node:path";
8 import { createApp, type App } from "../src/app.ts";
9 import { listComments } from "../src/core/issues.ts";
10 import { createRepo } from "../src/core/repos.ts";
11 import { createUser } from "../src/core/users.ts";
12 import { probes, type Result } from "../src/core/v1_proof.ts";
13 import { v1Proven } from "../src/maintain/duties/v1_proven.ts";
14 import { duties } from "../src/maintain/registry.ts";
15 import { quietLogs } from "./helpers/globals.ts";
16
17 quietLogs();
18 let dir = "", app: App, repoId = "";
19 const NOW = new Date("2026-11-15T12:00:00Z");
20 const bases: string[] = [];
21 let failing: number[] = [];
22
23 const fake = async (base: string): Promise<Result[]> => {
24 bases.push(base);
25 return probes.map((p) => (failing.includes(p.issue) ? { issue: p.issue, item: p.item, status: "fail", detail: "/ answers 500, not 200" } : { issue: p.issue, item: p.item, status: "pass", detail: "ok" }));
26 };
27
28 beforeAll(async () => {
29 dir = await mkdtemp(join(tmpdir(), "forge-v1-proven-"));
30 app = createApp({ dataDir: dir, port: 0 });
31 const owner = await createUser(app.db, { username: "boss", email: "[email protected]", password: "a-good-passphrase", origin: "seed", trusted: true, isAdmin: true });
32 if (!owner.ok) throw new Error("user");
33 const made = await createRepo(app.db, app.reposRoot, { ownerId: owner.value.id, name: "forge", visibility: "public", description: "f" });
34 if (!made.ok) throw new Error("repo");
35 repoId = made.value.id;
36 });
37 afterAll(async () => {
38 app.close();
39 await rm(dir, { recursive: true, force: true });
40 });
41
42 const run = (url = "https://forge.example/", forgeRepo = "boss/forge", proof = fake) =>
43 v1Proven(proof).run({ db: app.db, bus: app.bus, now: NOW, app: { config: { url, forgeRepo } } as unknown as App });
44 const defects = () => app.db.query<{ title: string; id: string }, [string]>("SELECT id, title FROM issue WHERE repo_id = ?1 AND title LIKE 'Defect of #%'").all(repoId);
45
46 test("every probe passes: no issue, one line per probe on the record, the public origin read", async () => {
47 const r = await run();
48 expect(r.findings).toEqual([]);
49 expect(defects()).toEqual([]);
50 expect(r.actions).toHaveLength(probes.length);
51 expect(r.actions[0]).toMatch(/^2026-11-15 pass #\d+ /);
52 expect(bases).toEqual(["https://forge.example"]);
53 });
54
55 test("one fails: one issue quoting its line; the same fails again: a comment, not a second issue", async () => {
56 const bad = probes[0]!;
57 failing = [bad.issue];

Add cases for an unproven result (no issue), two failing probes in one run (two issues), and no forgeRepo (recorded, nothing filed).

58 const first = await run();
59 expect(first.findings).toHaveLength(1);
60 const found = defects();
61 expect(found).toHaveLength(1);
62 expect(found[0]!.title.startsWith(`Defect of #${bad.issue}`)).toBe(true);
63 const body = app.db.query<{ body: string }, [string]>("SELECT body FROM issue WHERE id = ?1").get(found[0]!.id)!.body;
64 expect(body).toContain(`fail #${bad.issue} ${bad.item}: / answers 500, not 200`);
65
66 await run();
67 expect(defects()).toHaveLength(1);
68 const comments = listComments(app.db, found[0]!.id);
69 expect(comments).toHaveLength(1);
70 expect(comments[0]!.body).toContain("Still failing");
71
72 // Closed, then failing again: a new issue.
73 app.db.query("UPDATE issue SET state = 'closed' WHERE id = ?1").run(found[0]!.id);
74 await run();
75 expect(defects()).toHaveLength(2);
76 });
77
78 const mixed = (statuses: Record<number, Result["status"]>) => async (): Promise<Result[]> =>
79 probes.map((p) => ({ issue: p.issue, item: p.item, status: statuses[p.issue] ?? "pass", detail: "d" }));
80
81 test("an unproven probe files nothing and is no finding", async () => {
82 const before = defects().length;
83 const r = await run(undefined, undefined, mixed({ [probes[1]!.issue]: "unproven" }));
84 expect(defects()).toHaveLength(before);
85 expect(r.findings).toEqual([]);
86 expect(r.actions.some((a) => a.includes(`unproven #${probes[1]!.issue}`))).toBe(true);
87 });
88
89 test("no forgeRepo: the failure is recorded and found, nothing is filed", async () => {
90 const before = defects().length;
91 const r = await run(undefined, "", mixed({ [probes[0]!.issue]: "fail" }));
92 expect(defects()).toHaveLength(before);
93 expect(r.findings).toHaveLength(1);
94 expect(r.actions.some((a) => a.startsWith("2026-11-15 fail #"))).toBe(true);
95 expect(r.actions.some((a) => a.includes("no issue filed"))).toBe(true);
96 });
97
98 test("two probes failing in one run open two issues", async () => {
99 app.db.query("UPDATE issue SET state = 'closed' WHERE repo_id = ?1 AND title LIKE 'Defect of #%'").run(repoId);
100 const before = defects().length;
101 const a = probes[0]!.issue, b = probes[1]!.issue;
102 const r = await run(undefined, undefined, mixed({ [a]: "fail", [b]: "fail" }));
103 expect(r.findings).toHaveLength(2);
104 const found = defects().slice(before);
105 expect(found).toHaveLength(2);
106 expect(found.map((d) => d.title.match(/#(\d+)/)![1]).sort()).toEqual([String(a), String(b)].sort());
107 });
108
109 test("no public origin: nothing is called", async () => {
110 bases.length = 0;
111 const r = await run("");
112 expect(r.actions[0]).toContain("skipped");
113 expect(bases).toEqual([]);
114 });
115
116 test("the duty is registered, weekly", () => {
117 expect(duties.find((d) => d.name === "v1-proven")?.intervalMs).toBe(7 * 86_400_000);
118 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs