trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns19g rEventsFlows

Estimates are recorded against an issue, start from the forge's own history,… #11983

Merged torquecrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/forge:torque/11955-estimates-are-recorded-against-an-issue → main 16 commits fork

  • Merged. torque/11955-estimates-are-recorded-against-an-issue was merged into main as 252a32e.

Review

Sign in to review this change.
20 files changed +393 −31
docs/MCP.md +1 −1
@@ -98,7 +98,7 @@ bytes in `tools/list`. After changing a tool run `bun scripts/mcp-table.ts`;
9898 | `edit` | — | Retitle or rewrite an issue or pull: yours, or where you write; an agent only its own | 310 |
9999 | `egress_grants` | — | Private destinations and grants: list, grant or revoke (site admin) | 289 |
100100 | `email_verify` | — | Send yourself a new verification link | 126 |
101 | `estimate` | — | Ask an issue's base rate, or record `event` with `p50`, `p90` (instants) and `basis` | 399 |
101 | `estimate` | — | Ask an issue's base rate, or record `event` with `p50`, `p90` (instants) and `basis` | 406 |
102102 | `events` | — | Repository events | 259 |
103103 | `explore` | — | The apps, people (and, for an admin, forges) you can see, ranked | 627 |
104104 | `family` | — | The forge that sent this one and, for a site admin, those it sent, with state, version | 205 |
docs/features/estimates.md +6 −2
@@ -3,12 +3,16 @@
33 An estimate is a promise about when one issue's work happens, kept against the issue and scored. Rows are never edited: a new one supersedes and the old stays as data.
44
55 1. **Ask** the base rate for an issue: MCP `estimate` with `{ repo, issue }`, `GET /api/v1/repos/:owner/:name/issues/:n/estimates`, or in the terminal open the repository, choose **Estimates in a repository** from the palette and press `a` on the issue. It comes from the forge's own history (`src/flow/estimate_base.ts`): merged pulls in the last 30 days of the same repository, size and kind (and builder when assigned), measured from the first agent run on the issue (else its creation) to the pull opening, the merge and the release that carried the merge going live. Quantiles are nearest-rank, `sorted[round((n - 1) * q)]`. Under five samples it says so and widens: builder, kind, size, then 90 days. It adds what history does not know (queue position, runs going, budget left, why a release is held) and your own scorecard. Anyone who may read the issue may ask.
6 2. **Record** one: the same tool with `event` (`pull_opened`, `merged`, `live`), `p50`, `p90` (future instants, p50 ≤ p90) and `basis`; `POST` to the same URL; or `r` on the terminal screen, typing `event p50 p90 basis…` (for example `merged 2026-11-14T07:15:00Z 2026-11-14T11:00:00Z a small change`). Only someone with write access, or a crew member on the repository, records, and only on an open issue.
6 2. **Record** one: the same tool with `event` (`pull_opened`, `merged`, `live`, or `done` for an issue that takes several pulls: it is closed and stays closed 24 h, or its last pull is live), `p50`, `p90` (future instants, p50 ≤ p90) and `basis`; `POST` to the same URL; or `r` on the terminal screen, typing `event p50 p90 basis…` (for example `merged 2026-11-14T07:15:00Z 2026-11-14T11:00:00Z a small change`). Only someone with write access, or a crew member on the repository, records, and only on an open issue.
77 3. **See** it: the issue page and `/next` carry the line ("live by Fri 07:15 (90%: Fri 11:00) · steward · history says Fri 09:40"); `/flow` carries the scorecard.
88
99 ## What the forge does
1010
11 Nobody settles an estimate by hand. `src/actions/estimates_settle.ts` is the one place that hears of the events on the bus: a pull that says `Closes #N` opening or merging sets the actual instant of `pull_opened` or `merged`; `release.verified_live` resolves each open `live` estimate at the first release cut after its merge that served. An issue closed as **dropped** or **split**, or moved to **Not now** (`src/actions/next.ts`), voids what it left open: the rows stay, marked void, never scored. Closing as done voids nothing. `live` resolves only for the forge's own releases, not an app's production deploy.
11 Nobody settles an estimate by hand. `src/actions/estimates_settle.ts` is the one place that hears of the events on the bus: a pull that says `Closes #N` opening or merging sets the actual instant of `pull_opened` or `merged`; `release.verified_live` resolves each open `live` estimate at the first release cut after its merge that served. An issue closed as **dropped** or **split**, or moved to **Not now** (`src/actions/next.ts`), voids what it left open: the rows stay, marked void, never scored. Closing as done voids nothing. `done` resolves when the issue closed as done has stayed closed 24 h, or a release carrying its last merged pull went live, whichever came first (`settleDue`: on the events above and every five minutes). A refusal for an event that already happened names the events that can still be estimated.
12
13 The basis line ends with how often issues like these were reopened after being closed ("reopened after closing: 3 of 8"), counting the ones reopened and open again: an estimate of "merged" is not an estimate of "works".
14
15 `live` needs a release on record. A release the keeper cut is. One shipped by hand (a tag, an image built on the host, the container restarted with `FORGE_VERSION` and `FORGE_COMMIT`) is recorded by the forge at its own boot when that stamp is newer than the newest release that served (`src/keeper/by_hand.ts`: went live = boot time, `how` = "by hand"). Releases shipped by hand before this one are not back-filled, so a merge made before it resolves `live` at the first recorded release after it. `live` resolves only for the forge's own releases, not an app's production deploy.
1216
1317 ## Scores
1418
src/actions/estimates.ts +9 −7
@@ -5,7 +5,7 @@
55 * boundary; rows and maths are `core/estimates.ts` and `flow/estimate_*`.
66 */
77 import type { App } from "../app.ts";
8 import { addEstimate, currentOf, estimatesOf, estimatesVisibleTo, isEvent, type EstimateEvent, type EstimateRow } from "../core/estimates.ts";
8 import { addEstimate, currentOf, EVENTS, estimatesOf, estimatesVisibleTo, isEvent, type EstimateEvent, type EstimateRow } from "../core/estimates.ts";
99 import { forgeSource } from "../settings/forge_source.ts";
1010 import { getIssue, type Issue } from "../core/issues.ts";
1111 import { mergedPullClosing, openPullsClosing } from "../core/pulls.ts";
@@ -24,8 +24,10 @@ import { err, ok, type Result } from "../kernel/result.ts";
2424 export const ESTIMATE_FORBIDDEN = "Only someone with write access, or a crew member on this repository, records an estimate.";
2525 export const ESTIMATE_NOT_OPEN = "Only an open issue is estimated.";
2626 export const ESTIMATE_SHAPE =
27 "`event` is pull_opened, merged or live; `p50` and `p90` are instants like 2026-11-14T07:15:00Z, both in the future with p50 no later than p90; `basis` is one line, at most 300 characters.";
28 export const ESTIMATE_PASSED = "That event has already happened, so it is not estimated.";
27 "`event` is pull_opened, merged, live or done; `p50` and `p90` are instants like 2026-11-14T07:15:00Z, both in the future with p50 no later than p90; `basis` is one line, at most 300 characters.";
28 /** Said when the event has happened: which events can still be estimated on this issue. */
29 export const estimatePassed = (event: EstimateEvent, open: readonly EstimateEvent[]): string =>
30 `${event} has already happened on this issue, so it is not estimated. Events that can still be estimated: ${open.length === 0 ? "none" : open.join(", ")}.`;
2931 const BASIS_MAX = 300;
3032 const INSTANT = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,3})?)?(?:Z|[+-]\d{2}:\d{2})$/;
3133
@@ -164,13 +166,13 @@ export function recordEstimate(
164166 const basis = input.basis === undefined ? "" : input.basis;
165167 // eslint-disable-next-line no-control-regex
166168 if (typeof basis !== "string" || basis.length > BASIS_MAX || /[\u0000-\u001f\u007f]/.test(basis)) return invalid(ESTIMATE_SHAPE);
167 const happened =
168 event === "pull_opened"
169 const happenedOf = (ev: EstimateEvent): boolean =>
170 ev === "pull_opened"
169171 ? openPullsClosing(app.db, repo.id).has(issue.number) || mergedPullClosing(app.db, repo.id, issue.number) !== undefined
170 : event === "merged"
172 : ev === "merged"
171173 ? mergedPullClosing(app.db, repo.id, issue.number) !== undefined
172174 : false;
173 if (happened) return invalid(ESTIMATE_PASSED);
175 if (happenedOf(event)) return invalid(estimatePassed(event, EVENTS.filter((ev) => !happenedOf(ev))));
174176 const base = baseFor(app, repo, issue, now);
175177 const labels = labelsOf(app.db, [issue.id]).get(issue.id) ?? [];
176178 const id = addEstimate(app.db, {
src/actions/estimates_settle.ts +28 −3
@@ -8,11 +8,13 @@
88 * - `pull.merged`: the same issues merged (the pull's merge time).
99 * - `release.verified_live`: each open `live` estimate resolves at the first release, cut after
1010 * its merge, that served.
11 * - `done`: an issue closed as done that stayed closed 24 h, or whose last pull is live, is
12 * finished. Time passing is no event, so `settleDue` also runs on a timer (`main.ts`).
1113 * - `issue.closed` as dropped or split: what it left open is void. (Moving to Not now voids in
1214 * `actions/next.ts`, the act that does it.)
1315 */
1416 import type { Database } from "bun:sqlite";
15 import { settleEvent, settleLive, voidOpen, type ServedRelease } from "../core/estimates.ts";
17 import { settleDone, settleEvent, settleLive, voidOpen, type ServedRelease } from "../core/estimates.ts";
1618 import type { ForgeEvent } from "../core/events.ts";
1719 import { getIssue } from "../core/issues.ts";
1820 import { closesFromBody, mergedPullClosing } from "../core/pulls.ts";
@@ -46,10 +48,33 @@ function servedReleases(db: Database): ServedRelease[] {
4648 .map((r) => ({ cutAt: r.cut_at, liveAt: r.live_at }));
4749 }
4850
51 /** Every open `live` estimate whose merge a served release contains resolves. */
52 export function settleLiveNow(db: Database): number {
53 return settleLive(db, servedReleases(db), (repoId, n) => mergedPullClosing(db, repoId, n)?.mergedAt ?? undefined);
54 }
55
56 /**
57 * Done estimates that have come due at `now`: closed 24 h, or the last pull's release is
58 * serving.
59 */
60 export function settleDue(db: Database, now: Date = new Date()): number {
61 const releases = servedReleases(db);
62 return settleDone(
63 db,
64 now.toISOString(),
65 (mergedAt) => releases.find((r) => r.cutAt >= mergedAt)?.liveAt,
66 (repoId, n) => mergedPullClosing(db, repoId, n)?.mergedAt ?? undefined,
67 );
68 }
69
4970 /** Settle what this event settles. Subscribed once, in `app.ts`. */
5071 export function settleEstimates(db: Database, e: ForgeEvent): void {
5172 if (e.repoId === null) return;
52 const number = numberOf(e);
73 settleKind(db, { ...e, repoId: e.repoId }, numberOf(e));
74 if (e.kind === "pull.merged" || e.kind === "issue.closed" || e.kind === "release.verified_live") settleDue(db);
75 }
76
77 function settleKind(db: Database, e: ForgeEvent & { repoId: string }, number: number | undefined): void {
5378 switch (e.kind) {
5479 case "pull.opened": {
5580 const pull = number === undefined ? undefined : pullFacts(db, e.repoId, number);
@@ -68,7 +93,7 @@ export function settleEstimates(db: Database, e: ForgeEvent): void {
6893 return;
6994 }
7095 case "release.verified_live":
71 settleLive(db, servedReleases(db), (repoId, n) => mergedPullClosing(db, repoId, n)?.mergedAt ?? undefined);
96 settleLiveNow(db);
7297 return;
7398 }
7499 }
src/contract/openapi/routes/postApiV1ReposOwnerNameIssuesNumberEstimates.ts +1 −1
@@ -3,7 +3,7 @@ import { REPO, type Route } from "../shared.ts";
33 export const route: Route = {
44 method: "post",
55 path: `${REPO}/issues/{number}/estimates`,
6 summary: "Record an estimate against an issue: event (pull_opened, merged, live), p50 and p90 instants, a one-line basis; the forge resolves and scores it",
6 summary: "Record an estimate against an issue: event (pull_opened, merged, live, done), p50 and p90 instants, a one-line basis; the forge resolves and scores it",
77 auth: true,
88 role: "member",
99 body: { event: "string", p50: "string", p90: "string", basis: "string" },
src/core/estimates.ts +35 −1
@@ -10,7 +10,7 @@ import type { Database } from "bun:sqlite";
1010 import type { User } from "./users.ts";
1111 import { repoVisible } from "./visibility.ts";
1212
13 export const EVENTS = ["pull_opened", "merged", "live"] as const;
13 export const EVENTS = ["pull_opened", "merged", "live", "done"] as const;
1414 export type EstimateEvent = (typeof EVENTS)[number];
1515 export const isEvent = (v: unknown): v is EstimateEvent => typeof v === "string" && (EVENTS as readonly string[]).includes(v);
1616
@@ -224,3 +224,37 @@ export function voidOpen(db: Database, issueId: string, at: string = new Date().
224224 .query(`UPDATE issue_estimate SET voided_at = ?2 WHERE issue_id = ?1 AND actual_at IS NULL AND voided_at IS NULL`)
225225 .run(issueId, at).changes;
226226 }
227
228 /** How long a closed issue stays closed before it counts as finished. */
229 export const DONE_AFTER_MS = 24 * 3_600_000;
230
231 /**
232 * Open `done` estimates on issues that closed as done (#11955): the issue stayed closed 24 h, or a
233 * release went live after the last pull that closed it merged, whichever the issue reached first.
234 * `liveAfter` says when the first release cut after an instant went live; this module reads no
235 * releases. Returns how many it settled.
236 */
237 export function settleDone(
238 db: Database,
239 now: string,
240 liveAfter: (mergedAt: string) => string | undefined,
241 lastMerged: (repoId: string, number: number) => string | undefined,
242 ): number {
243 const open = db
244 .query<{ id: number; repo_id: string; number: number; closed_at: string }, []>(
245 `SELECT e.id, e.repo_id, i.number, i.closed_at FROM issue_estimate e JOIN issue i ON i.id = e.issue_id
246 WHERE e.event = 'done' AND e.actual_at IS NULL AND e.voided_at IS NULL
247 AND i.state = 'closed' AND i.close_reason = 'done' AND i.closed_at IS NOT NULL`,
248 )
249 .all();
250 let n = 0;
251 for (const r of open) {
252 const stayed = new Date(Date.parse(r.closed_at) + DONE_AFTER_MS).toISOString();
253 const merged = lastMerged(r.repo_id, r.number);
254 const live = merged === undefined ? undefined : liveAfter(merged);
255 const at = live !== undefined && live < stayed ? (live < r.closed_at ? r.closed_at : live) : stayed;
256 if (at > now) continue;
257 n += db.query(`UPDATE issue_estimate SET actual_at = ?2 WHERE id = ?1 AND actual_at IS NULL AND voided_at IS NULL`).run(r.id, at).changes;
258 }
259 return n;
260 }
src/core/migration-pins/20261114000000-estimate-done.pin added +1
@@ -0,0 +1,1 @@
1 0d8a77a0f8d93b0a916c7ba170f068484c6f4e02fed8a9f14d2cafff2e15f35e
src/core/migration-pins/20261114000100-release-how.pin added +1
@@ -0,0 +1,1 @@
1 a4b933abb956477215d033b24949f7667a7ed891ac919993e249b550fdf66347
src/core/migrations/20261114000000-estimate-done.ts added +28
@@ -0,0 +1,28 @@
1 // An estimate can mean "finished" (#11955): the event `done` joins pull_opened, merged and live,
2 // for an issue that takes several pulls. SQLite cannot widen a CHECK in place, so the table is
3 // copied into a new one and its indexes made again; nothing refers to it.
4 export default `
5 CREATE TABLE issue_estimate_next (
6 id INTEGER PRIMARY KEY AUTOINCREMENT,
7 repo_id TEXT NOT NULL REFERENCES repo(id) ON DELETE CASCADE,
8 issue_id TEXT NOT NULL REFERENCES issue(id) ON DELETE CASCADE,
9 event TEXT NOT NULL CHECK (event IN ('pull_opened','merged','live','done')),
10 p50_at TEXT NOT NULL,
11 p90_at TEXT NOT NULL,
12 made_at TEXT NOT NULL,
13 by_id TEXT NOT NULL REFERENCES user(id),
14 by_kind TEXT NOT NULL CHECK (by_kind IN ('person','agent')),
15 basis TEXT NOT NULL DEFAULT '',
16 base_json TEXT NOT NULL DEFAULT 'null',
17 size TEXT NOT NULL,
18 kind TEXT NOT NULL DEFAULT '',
19 actual_at TEXT,
20 voided_at TEXT
21 );
22 INSERT INTO issue_estimate_next SELECT * FROM issue_estimate;
23 DROP TABLE issue_estimate;
24 ALTER TABLE issue_estimate_next RENAME TO issue_estimate;
25 CREATE INDEX issue_estimate_issue ON issue_estimate(issue_id, event, id);
26 CREATE INDEX issue_estimate_open ON issue_estimate(event, repo_id) WHERE actual_at IS NULL AND voided_at IS NULL;
27 CREATE INDEX issue_estimate_repo ON issue_estimate(repo_id, made_at);
28 `;
src/core/migrations/20261114000100-release-how.ts added +6
@@ -0,0 +1,6 @@
1 // How a release went live (#11955): 'keeper' for one the release keeper cut, 'by hand' for one the
2 // steward shipped (a tag pushed, an image built on the host, the container restarted with
3 // FORGE_VERSION and FORGE_COMMIT) that the forge recorded at its own boot. Additive: one column.
4 export default `
5 ALTER TABLE keeper_release ADD COLUMN how TEXT NOT NULL DEFAULT 'keeper';
6 `;
src/crew/brief.ts +1 −1
@@ -374,7 +374,7 @@ const MCP_SIZE_RULE =
374374 * not the system brief, to keep that short.
375375 */
376376 export const ESTIMATE_RULE =
377 'A "by <time>" you say to the owner is recorded first with the `estimate` tool (ask it for the issue\'s base rate, then record `event` pull_opened, merged or live with `p50` and `p90` instants and a one-line `basis`), or it is not said. The forge resolves it when the event happens and scores it.';
377 'A "by <time>" you say to the owner is recorded first with the `estimate` tool (ask it for the issue\'s base rate, then record `event` pull_opened, merged, live or done (an issue that takes several pulls) with `p50` and `p90` instants and a one-line `basis`), or it is not said. The forge resolves it when the event happens and scores it.';
378378
379379 /** A plan on the issue is how a person sees where a run is without reading its log; only the forge serves the tool. */
380380 const PLAN_RULE =
src/flow/estimate_base.ts +50 −11
@@ -34,6 +34,15 @@ export interface HistorySample {
3434 readonly merged: number;
3535 readonly live: number | undefined;
3636 readonly mergedAtMs: number;
37 /** Start of work to the close (never before the merge); absent in old samples. */
38 readonly done?: number;
39 /** Closed once and opened again: the work was not finished when it was called finished. */
40 readonly reopened?: boolean;
41 /**
42 * Not closed now, because it was reopened: counts toward the reopen rate, not toward a
43 * duration.
44 */
45 readonly open?: boolean;
3746 }
3847
3948 export interface EventRate {
@@ -53,6 +62,11 @@ export interface BaseRate {
5362 readonly widened: readonly string[];
5463 readonly basis: string;
5564 readonly events: Record<EstimateEvent, EventRate | null>;
65 /**
66 * Of the issues like these that a pull closed, how many were reopened after: the cost of
67 * "merged is not works".
68 */
69 readonly reopened: { readonly n: number; readonly of: number };
5670 }
5771
5872 /** Nearest-rank quantile of an ascending array. */
@@ -76,15 +90,23 @@ export function historySamples(db: Database, repoId: string, sinceMs: number, no
7690 if (closed.size === 0) return [];
7791 const numbers = JSON.stringify([...closed.keys()].slice(0, PULLS_MAX * 4));
7892 const issues = db
79 .query<{ id: string; number: number; title: string; body: string; created_at: string; assignee: string | null; started: string | null }, [string, string]>(
80 `SELECT i.id, i.number, i.title, i.body, i.created_at, a.username AS assignee,
93 .query<{ id: string; number: number; title: string; body: string; created_at: string; assignee: string | null; started: string | null; state: string; close_reason: string | null; closed_at: string | null }, [string, string]>(
94 `SELECT i.id, i.number, i.title, i.body, i.created_at, a.username AS assignee, i.state, i.close_reason, i.closed_at,
8195 (SELECT MIN(r.created_at) FROM run r WHERE r.issue_id = i.id AND r.kind = 'agent') AS started
8296 FROM issue i LEFT JOIN user a ON a.id = i.assignee_id
83 WHERE i.repo_id = ?1 AND i.kind = 'issue' AND i.state = 'closed' AND i.close_reason = 'done'
97 WHERE i.repo_id = ?1 AND i.kind = 'issue'
8498 AND i.number IN (SELECT value FROM json_each(?2))`,
8599 )
86100 .all(repoId, numbers)
87101 .filter((i) => closed.has(i.number));
102 const reopened = new Set(
103 db
104 .query<{ n: number | null }, [string, string]>(
105 `SELECT DISTINCT json_extract(payload, '$.number') AS n FROM event WHERE repo_id = ?1 AND kind = 'issue.reopened' AND created_at >= ?2`,
106 )
107 .all(repoId, since)
108 .flatMap((r) => (r.n === null ? [] : [r.n])),
109 );
88110 const labels = labelsOf(db, issues.map((i) => i.id));
89111 const releases = db
90112 .query<{ cut_at: string; live_at: string }, [string]>(
@@ -93,6 +115,11 @@ export function historySamples(db: Database, repoId: string, sinceMs: number, no
93115 .all(since);
94116 const out: HistorySample[] = [];
95117 for (const i of issues) {
118 const wasReopened = reopened.has(i.number);
119 const finished = i.state === "closed" && i.close_reason === "done";
120 // An issue that is open again is no duration, but it is a reopen; any other open one is
121 // neither.
122 if (!finished && !wasReopened) continue;
96123 const c = closed.get(i.number)!;
97124 const start = Date.parse(i.started ?? i.created_at);
98125 const merged = Date.parse(c.merged);
@@ -109,6 +136,9 @@ export function historySamples(db: Database, repoId: string, sinceMs: number, no
109136 merged: merged - start,
110137 live: live === undefined || !Number.isFinite(live) || live < merged ? undefined : live - start,
111138 mergedAtMs: merged,
139 ...(finished && i.closed_at !== null && Number.isFinite(Date.parse(i.closed_at)) ? { done: Math.max(merged, Date.parse(i.closed_at)) - start } : {}),
140 reopened: wasReopened,
141 open: !finished,
112142 });
113143 }
114144 return out;
@@ -139,16 +169,20 @@ function fromNow(durations: readonly number[], elapsedMs: number, nowMs: number)
139169 * The base rate of `target` among `samples` (all within 90 days), widening until {@link MIN_BASE}
140170 * are found.
141171 */
142 export function baseRate(samples: readonly HistorySample[], target: Target, where: string, nowMs: number): BaseRate {
172 export function baseRate(allSamples: readonly HistorySample[], target: Target, where: string, nowMs: number): BaseRate {
173 const samples = allSamples.filter((s) => s.open !== true);
143174 const inDays = (days: number) => samples.filter((s) => s.mergedAtMs >= nowMs - days * DAY);
144 const levels: { widen: string | null; pick: () => HistorySample[]; who: string }[] = [
175 const bySize = (s: HistorySample) => s.size === target.size;
176 const byKind = (s: HistorySample) => bySize(s) && s.kind === target.kind;
177 const byBuilder = (s: HistorySample) => byKind(s) && s.builder === target.builder;
178 const levels: { widen: string | null; pick: () => HistorySample[]; who: string; match: (s: HistorySample) => boolean; days: number }[] = [
145179 ...(target.builder === null
146180 ? []
147 : [{ widen: null, who: `${target.size} ${target.kind === "" ? "" : `${target.kind} `}issues by ${target.builder}, last 30 days`, pick: () => inDays(30).filter((s) => s.size === target.size && s.kind === target.kind && s.builder === target.builder) }]),
148 { widen: target.builder === null ? null : "dropped the builder", who: `${target.size} ${target.kind === "" ? "" : `${target.kind} `}issues, last 30 days`, pick: () => inDays(30).filter((s) => s.size === target.size && s.kind === target.kind) },
149 { widen: "dropped the kind", who: `${target.size} issues, last 30 days`, pick: () => inDays(30).filter((s) => s.size === target.size) },
150 { widen: "dropped the size", who: "issues of any size, last 30 days", pick: () => inDays(30) },
151 { widen: "widened to 90 days", who: "issues of any size, last 90 days", pick: () => inDays(90) },
181 : [{ widen: null, who: `${target.size} ${target.kind === "" ? "" : `${target.kind} `}issues by ${target.builder}, last 30 days`, pick: () => inDays(30).filter(byBuilder), match: byBuilder, days: 30 }]),
182 { widen: target.builder === null ? null : "dropped the builder", who: `${target.size} ${target.kind === "" ? "" : `${target.kind} `}issues, last 30 days`, pick: () => inDays(30).filter(byKind), match: byKind, days: 30 },
183 { widen: "dropped the kind", who: `${target.size} issues, last 30 days`, pick: () => inDays(30).filter(bySize), match: bySize, days: 30 },
184 { widen: "dropped the size", who: "issues of any size, last 30 days", pick: () => inDays(30), match: () => true, days: 30 },
185 { widen: "widened to 90 days", who: "issues of any size, last 90 days", pick: () => inDays(90), match: () => true, days: 90 },
152186 ];
153187 const widened: string[] = [];
154188 let used = levels[levels.length - 1]!;
@@ -159,11 +193,15 @@ export function baseRate(samples: readonly HistorySample[], target: Target, wher
159193 used = level;
160194 if (group.length >= MIN_BASE) break;
161195 }
196 // The same group, plus the issues a pull closed that were reopened and are open again.
197 const reads = allSamples.filter((s) => used.match(s) && s.mergedAtMs >= nowMs - used.days * DAY);
198 const reopenedOf = { n: reads.filter((s) => s.reopened === true).length, of: reads.length };
162199 const thin = group.length < MIN_BASE;
163200 const events: Record<EstimateEvent, EventRate | null> = {
164201 pull_opened: fromNow(group.map((s) => s.pull), target.elapsedMs, nowMs),
165202 merged: fromNow(group.map((s) => s.merged), target.elapsedMs, nowMs),
166203 live: fromNow(group.flatMap((s) => (s.live === undefined ? [] : [s.live])), target.elapsedMs, nowMs),
204 done: fromNow(group.flatMap((s) => (s.done === undefined ? [] : [s.done])), target.elapsedMs, nowMs),
167205 };
168206 const merged = events.merged;
169207 const live = events.live;
@@ -171,9 +209,10 @@ export function baseRate(samples: readonly HistorySample[], target: Target, wher
171209 const parts = [
172210 merged === null ? "nothing merged yet" : `${target.elapsedMs > 0 ? "now→merged" : "work start→merged"} p50 ${durationWords(merged.p50Ms)}, p90 ${durationWords(merged.p90Ms)}`,
173211 stage.length === 0 || live === null ? "no release has gone live since" : `merged→live p50 ${durationWords(quantile(stage, 0.5))}, p90 ${durationWords(quantile(stage, 0.9))}`,
212 reopenedOf.of === 0 ? "reopened: no history" : `reopened after closing: ${reopenedOf.n} of ${reopenedOf.of}`,
174213 ];
175214 const basis = thin
176215 ? `only ${group.length} ${used.who} in ${where} (fewer than ${MIN_BASE}; ${widened.length === 0 ? "nothing wider to try" : widened.join(", ")}): ${parts.join("; ")}`
177216 : `${group.length} ${used.who} in ${where}${widened.length === 0 ? "" : ` (${widened.join(", ")})`}: ${parts.join("; ")}`;
178 return { n: group.length, thin, widened, basis, events };
217 return { n: group.length, thin, widened, basis, events, reopened: reopenedOf };
179218 }
src/flow/estimate_words.ts +2 −1
@@ -31,8 +31,9 @@ export const EVENT_WORDS: Record<EstimateEvent, string> = {
3131 pull_opened: "pull opened",
3232 merged: "merged",
3333 live: "live",
34 done: "done",
3435 };
35 const WENT: Record<EstimateEvent, string> = { pull_opened: "pull opened", merged: "merged", live: "went live" };
36 const WENT: Record<EstimateEvent, string> = { pull_opened: "pull opened", merged: "merged", live: "went live", done: "finished" };
3637
3738 /**
3839 * What an estimate was: actual over the p50 duration, both from the moment it was made; undefined
src/http/mcp/tools/estimate.ts +2 −2
@@ -4,14 +4,14 @@ import { mayView } from "../../../core/sealed.ts";
44
55 export const tool: Tool = {
66 name: "estimate",
7 doc: "`{ repo, issue, event?, p50?, p90?, basis? }`: with no times, the base rate for that issue from the forge's own history (p50/p90 instants and durations per event, sample size, one line of basis, what is true now, and your scorecard); with `event` (`pull_opened`, `merged`, `live`), `p50` and `p90` (future instants, p50 ≤ p90) it records an estimate, which the forge resolves when the event happens and a new one supersedes",
7 doc: "`{ repo, issue, event?, p50?, p90?, basis? }`: with no times, the base rate for that issue from the forge's own history (p50/p90 instants and durations per event, sample size, one line of basis, what is true now, and your scorecard); with `event` (`pull_opened`, `merged`, `live`, `done`: the issue closed and stayed closed 24 h, or its last pull is live), `p50` and `p90` (future instants, p50 ≤ p90) it records an estimate, which the forge resolves when the event happens and a new one supersedes",
88 description: "Ask an issue's base rate, or record `event` with `p50`, `p90` (instants) and `basis`.",
99 inputSchema: {
1010 type: "object",
1111 properties: {
1212 repo: REPO_ARG,
1313 issue: { type: "integer", minimum: 1 },
14 event: { type: "string", enum: ["pull_opened", "merged", "live"] },
14 event: { type: "string", enum: ["pull_opened", "merged", "live", "done"] },
1515 p50: { type: "string", maxLength: 40 },
1616 p90: { type: "string", maxLength: 40 },
1717 basis: { type: "string", maxLength: 300 },
src/keeper/by_hand.ts added +52
@@ -0,0 +1,52 @@
1 /**
2 * A release shipped by hand is still a release (#11955). When the steward tags, builds the image on
3 * the host and restarts the container with `FORGE_VERSION` and `FORGE_COMMIT` set, the keeper cut
4 * nothing, and `keeper_release` knew of none, so "live" could not resolve. At boot a forge whose
5 * stamp is newer than the newest release that served records itself: version, commit, went live =
6 * the boot time, `how` = "by hand". The same stamp is what `/changelog` reads
7 * (`kernel/version.ts`).
8 * A forge with no stamp, a preview (its stamp is not inherited) or one running an older version
9 * than the record records nothing.
10 */
11 import type { Database } from "bun:sqlite";
12 import { running } from "../kernel/version.ts";
13
14 const NUMERIC = /^\d+(?:\.\d+){0,3}$/;
15
16 /**
17 * Dotted numbers compared part by part: negative when `a` is older; undefined when either is not
18 * dotted numbers.
19 */
20 export function compareVersions(a: string, b: string): number | undefined {
21 if (!NUMERIC.test(a) || !NUMERIC.test(b)) return undefined;
22 const x = a.split(".").map(Number);
23 const y = b.split(".").map(Number);
24 for (let i = 0; i < Math.max(x.length, y.length); i++) {
25 const d = (x[i] ?? 0) - (y[i] ?? 0);
26 if (d !== 0) return d;
27 }
28 return 0;
29 }
30
31 /** The version recorded, or undefined when this boot records nothing. */
32 export function recordByHand(
33 db: Database,
34 env: Readonly<Record<string, string | undefined>> = process.env,
35 now: Date = new Date(),
36 ): string | undefined {
37 const stamp = running({ FORGE_VERSION: env["FORGE_VERSION"], FORGE_COMMIT: env["FORGE_COMMIT"] });
38 if (stamp.commit === null || !NUMERIC.test(stamp.version)) return undefined;
39 const served = db
40 .query<{ version: string }, []>(`SELECT version FROM keeper_release WHERE live_at IS NOT NULL AND stage != 'rolled_back'`)
41 .all();
42 if (served.some((r) => r.version.replace(/^v(?=\d)/, "") === stamp.version)) return undefined;
43 if (served.some((r) => (compareVersions(stamp.version, r.version.replace(/^v(?=\d)/, "")) ?? 0) < 0)) return undefined;
44 const at = now.toISOString();
45 const made = db
46 .query(
47 `INSERT OR IGNORE INTO keeper_release (version, sha, known_good_version, known_good_image, stage, cut_at, live_at, finished_at, reason, how)
48 VALUES (?1, ?2, '', '', 'verified', ?3, ?3, ?3, 'recorded at boot: shipped by hand', 'by hand')`,
49 )
50 .run(stamp.version, stamp.commit, at);
51 return made.changes === 1 ? stamp.version : undefined;
52 }
src/main.ts +11
@@ -98,6 +98,8 @@ import { walkJourney } from "./release/journey.ts";
9898 import { CHECK_EVERY_MS, setBehind, watchBehind } from "./release/behind.ts";
9999 import { readLandings } from "./release/landings.ts";
100100 import { ownTagsReader } from "./release/own_tags.ts";
101 import { recordByHand } from "./keeper/by_hand.ts";
102 import { settleDue, settleLiveNow } from "./actions/estimates_settle.ts";
101103 import { rollProbe, tickStep as keeperTick } from "./keeper/keeper.ts";
102104 import { stepWithBeat } from "./keeper/beat.ts";
103105 import { keeperEnv, keeperPorts } from "./keeper/wire.ts";
@@ -364,6 +366,14 @@ const bootSignup = signupFromEnv(process.env.FORGE_SIGNUP);
364366 if (bootSignup.ok && bootSignup.value !== null)
365367 setSignup(app.db, bootSignup.value, BOOT_ACTOR);
366368 startPerfWatcher(app, perfRecord, Date.now, TICK_MS, gitRecord);
369 // A release shipped by hand is recorded as live at boot, so live estimates and the base rate
370 // resolve whichever way a release ships; estimates of "done" come due as time passes (#11955).
371 const byHand = recordByHand(app.db);
372 if (byHand !== undefined) {
373 app.log.info("recorded the release this forge runs: shipped by hand", { version: byHand });
374 settleLiveNow(app.db);
375 }
376 const dueTimer = setInterval(() => during("estimates-due", () => void settleDue(app.db)), 300_000);
367377 const flagTimer = setInterval(
368378 () => during("flag-uses", () => flushFlagUses(app.db)),
369379 10_000,
@@ -1002,6 +1012,7 @@ async function close(): Promise<void> {
10021012 ssh?.stop();
10031013 servers.stop();
10041014 clearInterval(flagTimer);
1015 clearInterval(dueTimer);
10051016 flushFlagUses(app.db);
10061017 recordCleanExit(app.config.dataDir);
10071018 app.close();
src/tui/screens/estimates.ts +1 −1
@@ -75,7 +75,7 @@ export function estimatesScreen(f: Session, ctx: Ctx, repo: Repo): Screen {
7575 ctx.push(
7676 promptScreen(ctx, screen, {
7777 title: `Estimate #${issue.number}`,
78 label: "event p50 p90 basis: event is pull_opened, merged or live; times like 2026-11-14T07:15:00Z",
78 label: "event p50 p90 basis: event is pull_opened, merged, live or done; times like 2026-11-14T07:15:00Z",
7979 onSubmit: (raw) => {
8080 const input = parseEstimate(raw);
8181 if (input === undefined) return "event, p50 and p90 first, then the basis";
test/estimates_done.test.ts added +156
@@ -0,0 +1,156 @@
1 /**
2 * Estimates, second pass (#11955): `done`, the refusal that says what can still be estimated, the
3 * reopen rate, a release shipped by hand.
4 */
5 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
6 import { mkdtemp, rm } from "node:fs/promises";
7 import { tmpdir } from "node:os";
8 import { join } from "node:path";
9 import { createApp, type App } from "../src/app.ts";
10 import { recordEstimate } from "../src/actions/estimates.ts";
11 import { settleDue, settleLiveNow } from "../src/actions/estimates_settle.ts";
12 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
13 import { estimatesOf } from "../src/core/estimates.ts";
14 import { createIssue } from "../src/core/issues.ts";
15 import type { Repo } from "../src/core/repos.ts";
16 import { createUser, type User } from "../src/core/users.ts";
17 import { baseRate, historySamples, type HistorySample } from "../src/flow/estimate_base.ts";
18 import { compareVersions, recordByHand } from "../src/keeper/by_hand.ts";
19 import { listKeeperReleases } from "../src/keeper/record.ts";
20
21 const H = 3_600_000;
22 const NOW = new Date("2026-11-13T12:00:00.000Z");
23 const at = (ms: number) => new Date(NOW.getTime() + ms).toISOString();
24
25 describe("estimates: done, reopens, by-hand releases", () => {
26 let dir: string;
27 let app: App;
28 let alice: User;
29 let repo: Repo;
30 const issue = (title: string, body = "") => {
31 const o = createIssue(app.db, { repoId: repo.id, authorId: alice.id, title, body });
32 if (!o.ok) throw new Error("issue");
33 return o.value;
34 };
35 const pull = (closes: number, mergedAt: string | null) => {
36 const pr = createIssue(app.db, { repoId: repo.id, authorId: alice.id, title: `pr for ${closes}`, body: `Closes #${closes}`, kind: "pull" });
37 if (!pr.ok) throw new Error("pull");
38 app.db.query(`INSERT INTO pull (issue_id, base_branch, head_branch, base_oid, head_oid, merged_at) VALUES (?1,'main','b','a','b',?2)`).run(pr.value.id, mergedAt);
39 };
40 const close = (id: string, closedAt: string) =>
41 app.db.query(`UPDATE issue SET state='closed', close_reason='done', closed_at=?2 WHERE id=?1`).run(id, closedAt);
42
43 beforeAll(async () => {
44 dir = await mkdtemp(join(tmpdir(), "forge-estimates-done-"));
45 app = createApp({ dataDir: dir, port: 0 });
46 const made = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
47 if (!made.ok) throw new Error(made.error);
48 alice = made.value;
49 const r = await createRepo(app, alice, { name: "forge", visibility: "public" });
50 if (!r.ok) throw new Error(String(r.error));
51 repo = r.value;
52 await commitToBranch(app, alice, repo, { branch: "main", message: "base", files: [{ path: "a.txt", content: "a\n" }] });
53 }, 30_000);
54
55 afterAll(async () => {
56 app.close();
57 await rm(dir, { recursive: true, force: true, maxRetries: 20, retryDelay: 100 });
58 });
59
60 test("a merged pull refuses `merged` and names the events that can still be estimated", () => {
61 const t = issue("several pulls");
62 pull(t.number, at(-H));
63 const refused = recordEstimate(app, alice, repo, t, { event: "merged", p50: at(H), p90: at(2 * H) }, NOW);
64 expect(refused.ok).toBe(false);
65 if (refused.ok) return;
66 expect(refused.error.message).toContain("merged has already happened");
67 expect(refused.error.message).toContain("Events that can still be estimated: live, done.");
68 // `done` can still be estimated, and says so in the shape message too.
69 expect(recordEstimate(app, alice, repo, t, { event: "done", p50: at(H), p90: at(2 * H) }, NOW).ok).toBe(true);
70 const shape = recordEstimate(app, alice, repo, t, { event: "finished", p50: at(H), p90: at(2 * H) }, NOW);
71 if (!shape.ok) expect(shape.error.message).toContain("live or done");
72 });
73
74 test("done resolves when the issue stays closed 24 h; before that it is open; a reopen keeps it open", () => {
75 const t = issue("closes later");
76 expect(recordEstimate(app, alice, repo, t, { event: "done", p50: at(H), p90: at(2 * H) }, NOW).ok).toBe(true);
77 close(t.id, at(-2 * H));
78 expect(settleDue(app.db, NOW)).toBe(0);
79 expect(estimatesOf(app.db, t.id)[0]!.state).toBe("open");
80 expect(settleDue(app.db, new Date(NOW.getTime() + 21 * H))).toBe(0);
81 expect(settleDue(app.db, new Date(NOW.getTime() + 22 * H + 1))).toBe(1);
82 // The actual instant is when the condition became true: 24 h after the close.
83 expect(estimatesOf(app.db, t.id)[0]).toMatchObject({ state: "resolved", actualAt: at(22 * H) });
84 });
85
86 test("done resolves at the release that carries the last pull, if that comes before 24 h", () => {
87 const t = issue("lands and ships");
88 expect(recordEstimate(app, alice, repo, t, { event: "done", p50: at(H), p90: at(2 * H) }, NOW).ok).toBe(true);
89 pull(t.number, at(-3 * H));
90 close(t.id, at(-3 * H));
91 app.db.query(`INSERT INTO keeper_release (version, sha, known_good_version, known_good_image, stage, cut_at, live_at) VALUES ('0.0.1','s','k','i','verified',?1,?2)`).run(at(-2 * H), at(-H));
92 expect(settleDue(app.db, NOW)).toBe(1);
93 expect(estimatesOf(app.db, t.id)[0]).toMatchObject({ state: "resolved", actualAt: at(-H) });
94 });
95
96 test("the basis says how often such issues were reopened after closing", () => {
97 const base = (n: number, extra: Partial<HistorySample> = {}): HistorySample => ({
98 number: n, size: "large", kind: "", builder: null, pull: H, merged: 2 * H, live: undefined, mergedAtMs: NOW.getTime() - 86_400_000, done: 2 * H, ...extra,
99 });
100 const samples = [1, 2, 3, 4, 5, 6, 7].map((n) => base(n, n <= 2 ? { reopened: true } : {}));
101 // One more reopened and open again: counts toward the rate, never toward a duration.
102 samples.push(base(8, { reopened: true, open: true, merged: 99 * H }));
103 const r = baseRate(samples, { size: "large", kind: "", builder: null, elapsedMs: 0 }, "trevato/forge", NOW.getTime());
104 expect(r.n).toBe(7);
105 expect(r.reopened).toEqual({ n: 3, of: 8 });
106 expect(r.basis).toContain("reopened after closing: 3 of 8");
107 expect(r.events.merged!.p90Ms).toBe(2 * H);
108 expect(r.events.done!.p50Ms).toBe(2 * H);
109 expect(baseRate([], { size: "large", kind: "", builder: null, elapsedMs: 0 }, "x", NOW.getTime()).basis).toContain("reopened: no history");
110 });
111
112 test("history reads the reopen events: a closed-then-reopened issue is counted, even when open again", () => {
113 const now = Date.now();
114 const iso = (ms: number) => new Date(ms).toISOString();
115 const a = issue("reopened and closed again");
116 const b = issue("reopened and still open");
117 const c = issue("never reopened");
118 for (const t of [a, b, c]) {
119 pull(t.number, iso(now - 2 * H));
120 app.db.query(`UPDATE issue SET created_at = ?2 WHERE id = ?1`).run(t.id, iso(now - 5 * H));
121 }
122 close(a.id, iso(now - H));
123 close(c.id, iso(now - H));
124 for (const t of [a, b]) app.bus.append({ kind: "issue.reopened", repoId: repo.id, actorId: alice.id, payload: { number: t.number } });
125 const got = historySamples(app.db, repo.id, now - 30 * 86_400_000, now);
126 const by = (n: number) => got.find((s) => s.number === n);
127 expect(by(a.number)).toMatchObject({ reopened: true, open: false });
128 expect(by(b.number)).toMatchObject({ reopened: true, open: true });
129 expect(by(c.number)).toMatchObject({ reopened: false, open: false });
130 expect(by(c.number)!.done).toBeGreaterThan(0);
131 });
132
133 test("a release shipped by hand is recorded at boot, once, and only when newer", () => {
134 const env = { FORGE_VERSION: "v0.7.560", FORGE_COMMIT: "abcdef1234567" };
135 const t = issue("live by hand");
136 pull(t.number, at(-H));
137 expect(recordEstimate(app, alice, repo, t, { event: "live", p50: at(H), p90: at(2 * H) }, NOW).ok).toBe(true);
138 expect(recordByHand(app.db, env, NOW)).toBe("0.7.560");
139 const row = listKeeperReleases(app.db).find((r) => r.version === "0.7.560")!;
140 expect(row).toMatchObject({ sha: "abcdef1234567", stage: "verified" });
141 expect(app.db.query<{ how: string }, []>(`SELECT how FROM keeper_release WHERE version = '0.7.560'`).get()!.how).toBe("by hand");
142 expect(row.liveAt).not.toBeNull();
143 // The open `live` estimate resolves at the boot time.
144 expect(settleLiveNow(app.db)).toBeGreaterThanOrEqual(1);
145 expect(estimatesOf(app.db, t.id)[0]).toMatchObject({ state: "resolved", actualAt: row.liveAt });
146 // Again, and older, and unstamped: nothing.
147 expect(recordByHand(app.db, env)).toBeUndefined();
148 expect(recordByHand(app.db, { ...env, FORGE_VERSION: "0.7.559", FORGE_COMMIT: "1234567" })).toBeUndefined();
149 expect(recordByHand(app.db, {})).toBeUndefined();
150 expect(recordByHand(app.db, { FORGE_VERSION: "dev", FORGE_COMMIT: "1234567" })).toBeUndefined();
151 expect(recordByHand(app.db, { FORGE_VERSION: "0.7.561" })).toBeUndefined();
152 expect(recordByHand(app.db, { FORGE_VERSION: "0.7.561", FORGE_COMMIT: "1234567" })).toBe("0.7.561");
153 expect(compareVersions("0.7.560", "0.7.99")).toBeGreaterThan(0);
154 expect(compareVersions("0.7.560", "x")).toBeUndefined();
155 });
156 });
test/job-slices.test.ts +1
@@ -241,6 +241,7 @@ const UNTIMED: Record<string, string> = {
241241 probes: "probes each app over HTTP after a deploy and nightly; the fixture has no apps",
242242 prompts: "a closure in startPromptRoutine; the fixture has no prompt routines",
243243 timers: "fires due app timers over HTTP to the apps; the fixture has no apps",
244 "estimates-due": "settles due 'done' estimates every five minutes from one indexed query on open estimates; the fixture records none",
244245 snapshot: "copies the database file to disk (VACUUM INTO): I/O off the loop, nightly",
245246 "snapshot-mirror": "uploads a snapshot to the mirror: network, nightly",
246247 drill: "restores a snapshot into a scratch forge: nightly, I/O",
test/schema-additive.test.ts +1
@@ -161,6 +161,7 @@ const RETIRED: Record<string, { readonly drops: readonly string[]; readonly sinc
161161 const REBUILT: Record<string, string> = {
162162 "20261103120000-conversations-many": "conversation loses UNIQUE(user_id) (#8814); same columns, plus title and archived_at",
163163 "20261104100000-conversation-run-stopped-state": "conversation_run widens its state CHECK to admit stopped (#8972); same columns, same rows, the stopped column kept",
164 "20261114000000-estimate-done": "issue_estimate widens its event CHECK to admit done (#11955); same columns, same rows, the indexes made again; the release before never writes or reads a done row",
164165 "20261110170000-fork-follow-dont-follow": "fork_follow widens its mode CHECK to admit dont-follow (#608); same columns, same rows; the release before reads dont-follow as not take-updates",
165166 };
166167

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs