Public: anyone can read this; the crew can change it.
A repository can carry a beat the forge owns, and a direct push is gated on its tests before the switch #13729
Check · Review · Land · Live done — Landed and live in production.
scout/forge:scout/13709-a-repository-can-carry-a-beat-the-forge → main 7 commits fork 4 threads · 4 unresolved
18 files
changed +688
−16
AGENTS.md +1
| @@ -52,3 +52,4 @@ The check queue hands work to a runner through `src/checks/runner.ts` (#13652, R | ||
| 52 | 52 | |
| 53 | 53 | An app token's requests go behind people's and repeat reads are 304s (`src/http/app_turn.ts`, `repeat_reads.ts`, `perf/asking.ts`): see `docs/features/app-asking.md`. |
| 54 | 54 | `src/core/reports.ts`: `closedSet`, the ids of repositories whose owner closed reports from outside (#12321), read once per database and kept by `setReportsClosed`, the only writer, so every repository page's head link costs no statement; at most one id per closed repository, gone on restart (the rows are `report_closed`). `src/core/contribute_token.ts`: `openSet`, the ids of repositories that allow outside agents, read once per database and kept by `setContributeOpen`, the only writer, at most one id per allowing repository, gone on restart; a raw SQL write to `contribute_open` bypasses it. The terminal door for reports is `r` on the outside agents screen (`setOutsideReports`, admins). |
| 55 | `src/crew/beat.ts`: `lastBeat`, when each repository's beat last ran, and `declared`, the `beat` last read from each default branch with the commit it came from (#13709); at most one entry each per repository, gone on restart (a restart beats once at once; the declaration is a file in git). |
docs/features/README.md +1
| @@ -210,6 +210,7 @@ Read with the guide page [Ship and undo](../guide-ship-and-undo.md). | ||
| 210 | 210 | - [](stale-pulls.md) |
| 211 | 211 | - [](ranked-checks.md) |
| 212 | 212 | - [](stuck-pile.md) |
| 213 | - [](beat.md) | |
| 213 | 214 | - [](release-heartbeat.md) |
| 214 | 215 | - [](release-keeper.md) |
| 215 | 216 | - [](release-later.md) |
docs/features/beat.md added +19
| @@ -0,0 +1,19 @@ | ||
| 1 | # A beat the forge owns, and a gate on the push | |
| 2 | ||
| 3 | ## The beat | |
| 4 | ||
| 5 | A repository can carry a routine in its default branch's `forge.json`: | |
| 6 | ||
| 7 | json | |
| 8 | "beat": { "every": "1h" } | |
| 9 | ||
| 10 | ||
| 11 | Every `every` (`10m` to `7d`, default an hour; a misspelt key or a bad value is refused with the field named, `src/apps/beat_spec.ts`), the forge takes one turn of the repository's autonomy loop with its own agent: the next ready issue goes to the crew, or the planner is asked when the map is thin, or the feed says why not (`beat.ran`: every builder busy, the crew at its pace, autonomy paused, nothing ready). It needs nothing on anyone's machine and stops where the autonomy stops: a person turned autonomy on and set its week's share, and a spent share starts nothing. | |
| 12 | ||
| 13 | In the autonomy tick, a repository with a beat takes its turn of the loop only when the beat is due; the passes around it (finishing, stuck pulls, requested changes) still run each minute. The feed hears when the outcome changes or something started, not every interval, and a `beat` that cannot be read is said once in the feed (`beat not kept: ...`) while the repository stays on the minute loop. A repository whose autonomy is off has no beat, and says nothing: autonomy is the switch a person owns. The last run is held in process memory (`src/crew/beat.ts`), so a restart beats once at once. | |
| 14 | ||
| 15 | ## The gate | |
| 16 | ||
| 17 | A commit pushed straight to the default branch of an app that declares `app.test` no longer switches before it is tested. The new tree starts beside the old one and its tests run first; green switches, red does not (the tests get a budget, `GATE_BUDGET_MS`, ten minutes; a suite that outlives it counts as red): the last green commit keeps serving, the deploy run fails with `held back: its tests failed (...); <sha> keeps serving`, the feed carries an `app.gated` post, and the owner's notification address (Account) gets the same line. The first deploy of an app and a merged pull (already checked) switch at once, as before; previews are unchanged (`src/apps/push_gate.ts`). | |
| 18 | ||
| 19 | The app template's `AGENTS.md` states the ritual: one change, its test, its README line, its version, one commit. |
src/apps/beat_spec.ts added +38
| @@ -0,0 +1,38 @@ | ||
| 1 | /** | |
| 2 | * `beat` (top level of `forge.json`): a routine the forge owns for this repository. | |
| 3 | * | |
| 4 | * "beat": { "every": "1h" } | |
| 5 | * | |
| 6 | * Every `every`, the forge's own agent ships the next ready thing, or the repository's feed says | |
| 7 | * why not (`crew/beat.ts`). `every` is a number and a unit, `m` or `h` or `d`, from ten minutes to | |
| 8 | * a week; unset means an hour. Pure: plain data in, plain data or one sentence naming the field | |
| 9 | * out. Unknown keys are refused so a misspelt `evry` fails the manifest instead of meaning an hour. | |
| 10 | */ | |
| 11 | ||
| 12 | import { err, ok, type Result } from "../kernel/result.ts"; | |
| 13 | ||
| 14 | export interface BeatSpec { | |
| 15 | readonly every: string; | |
| 16 | readonly everyMs: number; | |
| 17 | } | |
| 18 | ||
| 19 | export const BEAT_MIN_MS = 10 * 60_000; | |
| 20 | export const BEAT_MAX_MS = 7 * 86_400_000; | |
| 21 | export const BEAT_DEFAULT = "1h"; | |
| 22 | ||
| 23 | const UNIT = { m: 60_000, h: 3_600_000, d: 86_400_000 } as const; | |
| 24 | ||
| 25 | export function parseBeat(value: unknown): Result<BeatSpec | undefined, string> { | |
| 26 | if (value === undefined) return ok(undefined); | |
| 27 | if (typeof value !== "object" || value === null || Array.isArray(value)) | |
| 28 | return err('forge.json: beat must be an object like { "every": "1h" }'); | |
| 29 | const block = value as Record<string, unknown>; | |
| 30 | const unknown = Object.keys(block).find((k) => k !== "every"); | |
| 31 | if (unknown !== undefined) return err(`forge.json: beat.${unknown.slice(0, 40)} is not a field; the only one is every`); | |
| 32 | const every = block["every"] ?? BEAT_DEFAULT; | |
| 33 | const match = typeof every === "string" ? /^([1-9]\d{0,3})([mhd])$/.exec(every) : null; | |
| 34 | if (match === null) return err('forge.json: beat.every must be a number and m, h or d, like "30m" or "1h"'); | |
| 35 | const everyMs = Number(match[1]) * UNIT[match[2] as keyof typeof UNIT]; | |
| 36 | if (everyMs < BEAT_MIN_MS || everyMs > BEAT_MAX_MS) return err("forge.json: beat.every must be from 10m to 7d"); | |
| 37 | return ok({ every: `${match[1]}${match[2]}`, everyMs }); | |
| 38 | } |
src/apps/gate_notice.ts added +26
| @@ -0,0 +1,26 @@ | ||
| 1 | /** | |
| 2 | * Telling the owner a push did not switch (#13709): one line to the notification address they gave | |
| 3 | * the forge, if any. A line that cannot be sent is dropped; it never fails the deploy. | |
| 4 | */ | |
| 5 | ||
| 6 | import type { Database } from "bun:sqlite"; | |
| 7 | import { notifyUrlOf, post } from "../crew/notify.ts"; | |
| 8 | import { createLog } from "../kernel/log.ts"; | |
| 9 | ||
| 10 | const log = createLog("gate"); | |
| 11 | ||
| 12 | export async function tellGated( | |
| 13 | db: Database, | |
| 14 | ownerId: string, | |
| 15 | line: string, | |
| 16 | send: typeof post = post, | |
| 17 | ): Promise<boolean> { | |
| 18 | try { | |
| 19 | const url = notifyUrlOf(db, ownerId); | |
| 20 | if (url === undefined) return false; | |
| 21 | return await send(url, line, { title: "forge" }); | |
| 22 | } catch (error) { | |
| 23 | log.warn("could not tell the owner a push was gated", { error: String(error) }); | |
| 24 | return false; | |
| 25 | } | |
| 26 | } |
src/apps/manifest.ts +9
| @@ -20,6 +20,7 @@ import { isLaunch, LAUNCHES, type Launch } from "../git/builds.ts"; | ||
| 20 | 20 | import { parseAdapter, type AdapterSpec } from "./adapter.ts"; |
| 21 | 21 | import { parseToolchain, type ToolchainPin } from "./toolchain.ts"; |
| 22 | 22 | import { parseSchedules, type ScheduleSpec } from "./schedule_spec.ts"; |
| 23 | import { parseBeat, type BeatSpec } from "./beat_spec.ts"; | |
| 23 | 24 | import { parseFeature, scheduleSpecs } from "./feature.ts"; |
| 24 | 25 | |
| 25 | 26 | export interface AppManifest { |
| @@ -104,6 +105,11 @@ export interface AppManifest { | ||
| 104 | 105 | * (`apps/schedule_spec.ts`). Unset is none. |
| 105 | 106 | */ |
| 106 | 107 | readonly schedules?: readonly ScheduleSpec[] | undefined; |
| 108 | /** | |
| 109 | * `beat` (top level of `forge.json`): the routine the forge's own agent keeps | |
| 110 | * (`apps/beat_spec.ts`). Unset is none. | |
| 111 | */ | |
| 112 | readonly beat?: BeatSpec | undefined; | |
| 107 | 113 | } |
| 108 | 114 | |
| 109 | 115 | export type ManifestError = |
| @@ -222,6 +228,8 @@ export function parseManifest( | ||
| 222 | 228 | return invalid(`forge.json: adapter.source.hosts names ${unopened}, which egress does not list`); |
| 223 | 229 | const schedules = parseSchedules(parsed.value["schedules"]); |
| 224 | 230 | if (!schedules.ok) return invalid(schedules.error); |
| 231 | const beat = parseBeat(parsed.value["beat"]); | |
| 232 | if (!beat.ok) return invalid(beat.error); | |
| 225 | 233 | // A feature's scheduled tools are schedules like any other (#13533). A feature block that does |
| 226 | 234 | // not parse is the absorb page's to report, as before: it declares none here. |
| 227 | 235 | const feature = parseFeature(parsed.value); |
| @@ -267,6 +275,7 @@ export function parseManifest( | ||
| 267 | 275 | ...(adapter.value === undefined ? {} : { adapter: adapter.value }), |
| 268 | 276 | ...(toolchain.value === undefined ? {} : { toolchain: toolchain.value }), |
| 269 | 277 | ...(allSchedules.length === 0 ? {} : { schedules: allSchedules }), |
| 278 | ...(beat.value === undefined ? {} : { beat: beat.value }), | |
| 270 | 279 | }); |
| 271 | 280 | } |
| 272 | 281 |
src/apps/push_gate.ts added +39
| @@ -0,0 +1,39 @@ | ||
| 1 | /** | |
| 2 | * The gate on a direct push (#13709). A commit pushed straight to the default branch used to | |
| 3 | * switch in and then be tested, so a red push served until someone noticed. Now, when production | |
| 4 | * has a deployment to keep and the manifest declares `test`, the tests run on the new tree | |
| 5 | * before the switch; red keeps the last green commit serving. Plain data in, plain data out. | |
| 6 | */ | |
| 7 | ||
| 8 | import { during } from "../perf/jobs.ts"; | |
| 9 | import { err, type Result } from "../kernel/result.ts"; | |
| 10 | ||
| 11 | /** Whether this deploy waits for its tests before it switches in. */ | |
| 12 | export function gatesPush(env: string, reason: string, hasLive: boolean, declaresTest: boolean): boolean { | |
| 13 | return env === "production" && reason === "push" && hasLive && declaresTest; | |
| 14 | } | |
| 15 | ||
| 16 | /** The run's summary and the owner's line when the gate held a push back. */ | |
| 17 | export function gateSummary(why: string, keptOid: string): string { | |
| 18 | const first = (why.split("\n")[0] ?? "").trim().slice(0, 160); | |
| 19 | return `held back: its tests failed (${first}); ${keptOid.slice(0, 7)} keeps serving`; | |
| 20 | } | |
| 21 | ||
| 22 | /** How long the gate waits for the tests: ten minutes, or `FORGE_GATE_BUDGET_MS` (1 s to 1 h). */ | |
| 23 | export function gateBudgetMs(raw: string | undefined = process.env["FORGE_GATE_BUDGET_MS"]): number { | |
| 24 | const n = raw !== undefined && /^\d{1,7}$/.test(raw) ? Number(raw) : NaN; | |
| 25 | return n >= 1_000 && n <= 3_600_000 ? n : 10 * 60_000; | |
| 26 | } | |
| 27 | ||
| 28 | /** The tests' verdict, or red when they outlive the budget: a hung suite never holds a deploy. */ | |
| 29 | export async function withinBudget(tested: Promise<Result<void, string>>, ms: number): Promise<Result<void, string>> { | |
| 30 | let timer: ReturnType<typeof setTimeout> | undefined; | |
| 31 | const late = new Promise<Result<void, string>>((resolve) => { | |
| 32 | timer = setTimeout(() => during("push-gate-budget", () => resolve(err(`the tests did not finish in ${Math.round(ms / 1000)} s`))), ms); | |
| 33 | }); | |
| 34 | try { | |
| 35 | return await Promise.race([tested, late]); | |
| 36 | } finally { | |
| 37 | clearTimeout(timer); | |
| 38 | } | |
| 39 | } |
src/apps/runtime.ts +35 −1
| @@ -24,6 +24,8 @@ import { commitLog } from "../git/read.ts"; | ||
| 24 | 24 | import { noteRelease, noteReleaseTests, recordsReleases } from "./release_record.ts"; |
| 25 | 25 | import { settleStaleReleases } from "../core/app_releases.ts"; |
| 26 | 26 | import { forgetLanded, noteLanded } from "./landed.ts"; |
| 27 | import { gateBudgetMs, gatesPush, gateSummary, withinBudget } from "./push_gate.ts"; | |
| 28 | import { tellGated } from "./gate_notice.ts"; | |
| 27 | 29 | import { releaseOfCommit } from "../core/release_subject.ts"; |
| 28 | 30 | import { pullDir, runDir } from "../core/pulls.ts"; |
| 29 | 31 | import { accountActive } from "../core/accounts.ts"; |
| @@ -1437,6 +1439,38 @@ export class AppsRuntime { | ||
| 1437 | 1439 | return; |
| 1438 | 1440 | } |
| 1439 | 1441 | |
| 1442 | // A direct push to production is gated (#13709): the declared tests run on the new tree | |
| 1443 | // before the switch, and while they fail the last green commit keeps serving. A merged | |
| 1444 | // pull already passed its check, and a first deploy has nothing to keep, so both swap in | |
| 1445 | // at once as before. | |
| 1446 | let gated: Result<void, string> | undefined; | |
| 1447 | if (gatesPush(state.env, run.reason, state.live !== undefined, outcome.value.manifest.test !== null)) { | |
| 1448 | write(`gate: running the tests on ${run.commitOid.slice(0, 7)} before the switch`); | |
| 1449 | gated = await withinBudget(outcome.tested, gateBudgetMs()); | |
| 1450 | if (!gated.ok) { | |
| 1451 | const kept = state.live!; | |
| 1452 | await outcome.value.handle.stop(); | |
| 1453 | revokeAppToken(this.app.db, outcome.value.tokenId); | |
| 1454 | if (outcome.value.dir !== kept.dir) await rm(outcome.value.dir, { recursive: true, force: true }); | |
| 1455 | state.deploying = undefined; | |
| 1456 | lease(); | |
| 1457 | const summary = gateSummary(gated.error, kept.commitOid); | |
| 1458 | state.lastError = summary; | |
| 1459 | state.lastErrorCommitOid = run.commitOid; | |
| 1460 | const done = finishRun(this.app.db, run.id, { state: "failed", summary }); | |
| 1461 | write(`failed: ${summary}`); | |
| 1462 | if (done) announceRun(this.app, done); | |
| 1463 | this.app.bus.append({ | |
| 1464 | kind: "app.gated", | |
| 1465 | repoId: repo.id, | |
| 1466 | payload: { app: state.app.slug, env: state.env, commitOid: run.commitOid, keptOid: kept.commitOid, why: gated.error.slice(0, 300) }, | |
| 1467 | }); | |
| 1468 | await tellGated(this.app.db, repo.ownerId, `${state.app.slug}: ${summary}\n${this.urlFor(state.app, state.env)}`); | |
| 1469 | return; | |
| 1470 | } | |
| 1471 | write("gate: the tests passed; switching"); | |
| 1472 | } | |
| 1473 | ||
| 1440 | 1474 | // It answers: swap it in now, before its tests are done. `state.deploying` |
| 1441 | 1475 | // stays set — the run is still open, and a stop or a teardown must still |
| 1442 | 1476 | // be able to reach in and kill the test step — until they finish below. |
| @@ -1541,7 +1575,7 @@ export class AppsRuntime { | ||
| 1541 | 1575 | live.manifest.looks, |
| 1542 | 1576 | ); |
| 1543 | 1577 | |
| 1544 | const tested = await outcome.tested; | |
| 1578 | const tested = gated ?? (await outcome.tested); | |
| 1545 | 1579 | state.deploying = undefined; |
| 1546 | 1580 | lease(); |
| 1547 | 1581 | noteReleaseTests(this.app.db, state.env, live.manifest, repo.id, live.commitOid, tested.ok); |
src/core/event_audience.ts +2
| @@ -225,6 +225,8 @@ export function classifiedAudience(kind: string): EventAudience | undefined { | ||
| 225 | 225 | // A hook that keeps failing is for those who can act on it: the host and |
| 226 | 226 | // the reason, never the address (`webhook_streak.ts`). |
| 227 | 227 | if (kind === "webhook.failing") return "writers"; |
| 228 | // What a repository's beat did says how its autonomy and week's share stand (`crew/beat.ts`). | |
| 229 | if (kind === "beat.ran") return "writers"; | |
| 228 | 230 | if ( |
| 229 | 231 | kind.startsWith("webhook.") || |
| 230 | 232 | kind.startsWith("app.env_") || |
src/crew/autonomy.ts +23 −13
| @@ -115,6 +115,7 @@ import { REPEAT_NOTE } from "./repeat_note.ts"; | ||
| 115 | 115 | import { given, pendingReplans } from "../core/limb_replan.ts"; |
| 116 | 116 | import { handedOverPulls, SKIP_SUMMARY, skipsReplan } from "./guard.ts"; |
| 117 | 117 | import { type Held, heldBy, heldWords, replannable } from "./held.ts"; |
| 118 | import { driveOrBeat } from "./beat.ts"; | |
| 118 | 119 | import { forgetBacklog, noteBacklog, waitingBy, waitingWords } from "./backlog.ts"; |
| 119 | 120 | import { ceilingHold, pausedUntil, pauseSentence } from "../core/planner_pause.ts"; |
| 120 | 121 | import { CREW_OFF_WORDS, crewBuildsHere } from "../core/crew_switch.ts"; |
| @@ -394,7 +395,9 @@ function meanRunUsd(db: App["db"], repoId: string): number { | ||
| 394 | 395 | ); |
| 395 | 396 | } |
| 396 | 397 | |
| 397 | /** The issues each repository picked in this tick, by repository id: their runs may not show yet. */ | |
| 398 | /** | |
| 399 | * The issues each repository picked in this tick, by repository id: their runs may not show yet. | |
| 400 | */ | |
| 398 | 401 | export type TickPicks = ReadonlyMap<string, ReadonlySet<string>>; |
| 399 | 402 | |
| 400 | 403 | /** |
| @@ -649,10 +652,10 @@ export function startable( | ||
| 649 | 652 | |
| 650 | 653 | /** |
| 651 | 654 | * Give the planner the direction when the reserve is due a pass — down to |
| 652 | * half of two startable issues per builder slot (`reserve.ts`) — or when the crew has handed it an issue it parked twice, | |
| 653 | * and no planning run is in flight, and the last one is old enough. | |
| 654 | * Returns whether a planning run was set up. Planning runs alongside | |
| 655 | * building: the builders are never waited for. | |
| 655 | * half of two startable issues per builder slot (`reserve.ts`) — or when the crew has handed it an | |
| 656 | * issue it parked twice, and no planning run is in flight, and the last one is old enough. Returns | |
| 657 | * whether a planning run was set up. Planning runs alongside building: the builders are never | |
| 658 | * waited for. | |
| 656 | 659 | * |
| 657 | 660 | * A hand-over is owed one pass, not one a minute: it asks only while it is |
| 658 | 661 | * newer than the last planning run. That one pass is owed even by a planner |
| @@ -1008,7 +1011,10 @@ export function parkForBreaker( | ||
| 1008 | 1011 | return true; |
| 1009 | 1012 | } |
| 1010 | 1013 | |
| 1011 | /** How many times the crew has parked this issue since a person who directs it last spoke (`parks.ts`). */ | |
| 1014 | /** | |
| 1015 | * How many times the crew has parked this issue since a person who directs it last spoke | |
| 1016 | * (`parks.ts`). | |
| 1017 | */ | |
| 1012 | 1018 | export function parkedCount(app: App, repo: Repo, issueId: string): number { |
| 1013 | 1019 | return countParks(app.db, repo, issueId); |
| 1014 | 1020 | } |
| @@ -1305,7 +1311,8 @@ export async function autonomyTick( | ||
| 1305 | 1311 | sweep: (app: App, repo: Repo) => Promise<number> = sweepGreen, |
| 1306 | 1312 | ): Promise<void> { |
| 1307 | 1313 | if (crewOffInPreview()) return; |
| 1308 | // A pull a person holds with changes requested goes back to its builder after a day of silence, in any repository. | |
| 1314 | // A pull a person holds with changes requested goes back to its builder after a day of silence, | |
| 1315 | // in any repository. | |
| 1309 | 1316 | try { |
| 1310 | 1317 | sweepStranded(app, now); |
| 1311 | 1318 | } catch (error) { |
| @@ -1336,7 +1343,8 @@ export async function autonomyTick( | ||
| 1336 | 1343 | // the next batch is formed, not at its gate (#6611). |
| 1337 | 1344 | pruneLine(app, repo); |
| 1338 | 1345 | void startSweep(app, repo, sweep); |
| 1339 | // Each pass below is one slice of the thread: a request waiting is served between them (#8848). | |
| 1346 | // Each pass below is one slice of the thread: a request waiting is served between them | |
| 1347 | // (#8848). | |
| 1340 | 1348 | await breathe(); |
| 1341 | 1349 | // First of all: a slot freed goes to an approved pull that no longer |
| 1342 | 1350 | // merges, before a changes-requested one or a new issue (#3682). |
| @@ -1379,7 +1387,7 @@ export async function autonomyTick( | ||
| 1379 | 1387 | const requested = await moveRequested(app, current, freeSlots(app, current, now), now); |
| 1380 | 1388 | await breathe(); |
| 1381 | 1389 | const unstuck = unstickNow(app, current, now); |
| 1382 | const did = await driveRepo(app, current, now, picked); | |
| 1390 | const did = await driveOrBeat(app, current, now, picked, driveRepo); | |
| 1383 | 1391 | await breathe(); |
| 1384 | 1392 | // Rules only, no run: an issue noted already done on the base closes. |
| 1385 | 1393 | const triaged = await liveTriage(app, current, now).catch((error) => { |
| @@ -1432,7 +1440,8 @@ export async function autonomyTick( | ||
| 1432 | 1440 | // The standing reserve the planner keeps: startable/target (#3237). |
| 1433 | 1441 | reserve: reserveWords(reserveOf(held.startable.length, status.slots)), |
| 1434 | 1442 | held: heldWords(held), |
| 1435 | // What the picker passes over among the startable, so no ready issue goes unexplained (#8924). | |
| 1443 | // What the picker passes over among the startable, so no ready issue goes unexplained | |
| 1444 | // (#8924). | |
| 1436 | 1445 | passed: passedWords(pickFilter(app, current, held.startable, now).passed), |
| 1437 | 1446 | // The files 5 or more open pulls touch, which no new build starts on (#5934). |
| 1438 | 1447 | ...(held.by.hot_file.length === 0 ? {} : { hot: hotWords(hotFiles(current.id)) }), |
| @@ -1448,8 +1457,8 @@ export async function autonomyTick( | ||
| 1448 | 1457 | idle: idleBuilders(activeNow, crewNow), |
| 1449 | 1458 | ...(governor === undefined ? {} : { governor }), |
| 1450 | 1459 | // Approved conflicted pulls: queued, in flight, and why each other is |
| 1451 | // skipped (#3682, #6237), or "0 candidates of N open" (#6263); and the lane for rebases: conflicted N, | |
| 1452 | // rebasing M, room R (#6140). | |
| 1460 | // skipped (#3682, #6237), or "0 candidates of N open" (#6263); and the lane for rebases: | |
| 1461 | // conflicted N, rebasing M, room R (#6140). | |
| 1453 | 1462 | ...finishFields(finishing.moves, readConflictLane(app.db), finishing.open), |
| 1454 | 1463 | ...(requested.length === 0 |
| 1455 | 1464 | ? {} |
| @@ -2256,7 +2265,8 @@ async function admit( | ||
| 2256 | 2265 | if (!again) addComment(app, by, repo, getIssueById(app.db, pull.id) ?? pull, clashSaid(red.comment, together)); |
| 2257 | 2266 | // A red that reproduces on the re-gate is the code's: only then is it the doctor's to hold. |
| 2258 | 2267 | if (again) treatRed(app, repo, by, pull, red, baseOid); |
| 2259 | // Red only on top of pulls that landed first: no single culprit yet, so the class is recorded and the re-gate alone decides. | |
| 2268 | // Red only on top of pulls that landed first: no single culprit yet, so the class is recorded | |
| 2269 | // and the re-gate alone decides. | |
| 2260 | 2270 | else noteRed(app, repo, pull, red, baseOid, together); |
| 2261 | 2271 | }, |
| 2262 | 2272 | }); |
src/crew/beat.ts added +111
| @@ -0,0 +1,111 @@ | ||
| 1 | /** | |
The |
||
| 2 | * The beat a repository carries (#13709): `beat` in its `forge.json` (`apps/beat_spec.ts`) says | |
| 3 | * "every hour, ship the next ready thing, or say why not". The forge keeps it with its own agent: | |
| 4 | * a due beat takes one turn of the repository's autonomy loop (`driveRepo`), which starts the | |
| 5 | * next ready issue on the crew, and then says in the repository's feed what came of it | |
| 6 | * (`beat.ran`, with `beatWords`, when the outcome changes or something started). Spend is the | |
| 7 | * autonomy's, which a person turned on: a repository whose autonomy is off has no beat, and one | |
| 8 | * whose week's share is spent says so and starts nothing. A repository with a beat takes that turn | |
| 9 | * only when its beat is due (`driveOrBeat`), in the tick's own pass, so the passes before it still | |
| 10 | * run. Last run per repository is held in process memory (`kept`), so a restart beats once at once; | |
| 11 | * the declarations are read from the default branch's `forge.json`, re-read only when that branch | |
| 12 | * moves. | |
| 13 | */ | |
| 14 | ||
| 15 | import type { App } from "../app.ts"; | |
| 16 | import { parseBeat, type BeatSpec } from "../apps/beat_spec.ts"; | |
| 17 | import { readFileAt } from "../apps/checkout.ts"; | |
| 18 | import type { Repo } from "../core/repos.ts"; | |
| 19 | import { resolveOid } from "../git/merge.ts"; | |
| 20 | import { repoDir } from "../git/repos.ts"; | |
| 21 | import { createLog } from "../kernel/log.ts"; | |
| 22 | ||
| 23 | const log = createLog("beat"); | |
| 24 | ||
| 25 | /** What one turn of a repository's autonomy loop did (`autonomy.ts` `Drove`). */ | |
| 26 | export type Drove = "off" | "paused" | "waiting" | "full" | "picked" | "planned" | "idle" | "stalled"; | |
| 27 | export type Drive = (app: App, repo: Repo, now: Date, picked: Map<string, Set<string>>) => Promise<Drove>; | |
| 28 | ||
| 29 | /** | |
| 30 | * Per repository: when its beat last ran, the declaration last read with the commit it came from, | |
| 31 | * and what the feed was last told. Held per forge, keyed by the app. | |
| 32 | */ | |
| 33 | type Kept = { | |
| 34 | last: Map<string, number>; | |
| 35 | declared: Map<string, { oid: string; beat: BeatSpec | undefined }>; | |
| 36 | said: Map<string, Drove>; | |
| 37 | }; | |
| 38 | const kept = new WeakMap<App, Kept>(); | |
| 39 | function keptOf(app: App): Kept { | |
| 40 | let k = kept.get(app); | |
| 41 | if (k === undefined) kept.set(app, (k = { last: new Map(), declared: new Map(), said: new Map() })); | |
| 42 | return k; | |
| 43 | } | |
| 44 | ||
| 45 | /** Whether a beat is due: never run, or `everyMs` since the last. */ | |
| 46 | export const beatDue = (last: number | undefined, everyMs: number, now: number): boolean => last === undefined || now - last >= everyMs; | |
| 47 | ||
| 48 | /** What a turn did, in a sentence a person reads in the feed. */ | |
| 49 | export function beatWords(did: Exclude<Drove, "off">): string { | |
| 50 | switch (did) { | |
| 51 | case "picked": return "started the next ready issue"; | |
| 52 | case "planned": return "the map was thin, so the planner was asked for more"; | |
| 53 | case "full": return "not started: every builder is busy"; | |
| 54 | case "waiting": return "not started: the crew is at its pace for now"; | |
| 55 | case "paused": return "not started: autonomy is paused (the week's share is spent, or nobody is set to drive)"; | |
| 56 | case "stalled": return "nothing started: red pulls wait for someone"; | |
| 57 | case "idle": return "nothing ready to ship"; | |
| 58 | } | |
| 59 | } | |
| 60 | ||
| 61 | /** The beat declared on a repository's default branch; re-read only when the branch moves. */ | |
| 62 | export async function beatOf(app: App, repo: Repo): Promise<BeatSpec | undefined> { | |
| 63 | const dir = repoDir(app.reposRoot, repo.ownerName, repo.name); | |
| 64 | const oid = await resolveOid(dir, repo.defaultBranch); | |
| 65 | if (oid === undefined) return undefined; | |
| 66 | const known = keptOf(app).declared.get(repo.id); | |
| 67 | if (known?.oid === oid) return known.beat; | |
| 68 | const text = await readFileAt(dir, oid, "forge.json"); | |
| 69 | let beat: BeatSpec | undefined; | |
| 70 | if (text !== undefined) { | |
| 71 | try { | |
| 72 | const parsed: unknown = JSON.parse(text); | |
| 73 | const read = parseBeat(typeof parsed === "object" && parsed !== null ? (parsed as Record<string, unknown>)["beat"] : undefined); | |
| 74 | if (read.ok) beat = read.value; | |
| 75 | else { | |
| 76 | log.warn("beat not kept", { repo: `${repo.ownerName}/${repo.name}`, why: read.error }); | |
| 77 | // Said once, when the declaration is read (it is re-read only when the branch moves). | |
| 78 | app.bus.append({ kind: "beat.ran", repoId: repo.id, payload: { every: "-", did: "invalid", says: `beat not kept: ${read.error}` } }); | |
| 79 | } | |
| 80 | } catch { | |
| 81 | beat = undefined; | |
| 82 | } | |
| 83 | } | |
| 84 | keptOf(app).declared.set(repo.id, { oid, beat }); | |
| 85 | return beat; | |
| 86 | } | |
| 87 | ||
| 88 | /** | |
| 89 | * One turn of a repository in the autonomy tick: without a beat it is the autonomy loop's own | |
| 90 | * turn (`drive`); with one, the turn is taken only when the beat is due, and the feed hears of it | |
| 91 | * when the outcome changes or something started, not every interval. | |
| 92 | */ | |
| 93 | export async function driveOrBeat(app: App, repo: Repo, now: Date, picked: Map<string, Set<string>>, drive: Drive): Promise<Drove> { | |
| 94 | let beat: BeatSpec | undefined; | |
| 95 | try { | |
| 96 | beat = await beatOf(app, repo); | |
| 97 | } catch (error) { | |
| 98 | log.warn("beat could not be read", { repo: `${repo.ownerName}/${repo.name}`, error: String(error) }); | |
| 99 | } | |
| 100 | if (beat === undefined) return drive(app, repo, now, picked); | |
| 101 | const mine = keptOf(app); | |
| 102 | if (!beatDue(mine.last.get(repo.id), beat.everyMs, now.getTime())) return "waiting"; | |
| 103 | mine.last.set(repo.id, now.getTime()); | |
| 104 | const did = await drive(app, repo, now, picked); | |
| 105 | if (did === "off") return did; | |
| 106 | if (did === "picked" || did === "planned" || mine.said.get(repo.id) !== did) { | |
| 107 | app.bus.append({ kind: "beat.ran", repoId: repo.id, payload: { every: beat.every, did, says: beatWords(did) } }); | |
| 108 | } | |
| 109 | mine.said.set(repo.id, did); | |
| 110 | return did; | |
| 111 | } |
src/feed/items.ts +16
| @@ -780,6 +780,22 @@ function fromEvent( | ||
| 780 | 780 | what: post, |
| 781 | 781 | }; |
| 782 | 782 | } |
| 783 | // A direct push the gate held back, and a repository's beat saying what came of its turn | |
| 784 | // (#13709). | |
| 785 | case "app.gated": { | |
| 786 | const kept = p["keptOid"]; | |
| 787 | return { | |
| 788 | ...common, | |
| 789 | kind: "release", | |
| 790 | urgency: 4, | |
| 791 | link: base(repo), | |
| 792 | what: `a push was held back: its tests failed${typeof kept === "string" && kept !== "" ? `, ${kept.slice(0, 7)} keeps serving` : ""}`, | |
| 793 | }; | |
| 794 | } | |
| 795 | case "beat.ran": { | |
| 796 | const says = p["says"]; | |
| 797 | return { ...common, kind: "run", link: `${base(repo)}/flow`, what: `beat: ${typeof says === "string" ? clip(says) : "ran"}` }; | |
| 798 | } | |
| 783 | 799 | // A tell is a post; a comment is a reply under it, in its own words. |
| 784 | 800 | case "issue.opened": { |
| 785 | 801 | if (number === undefined) return undefined; |
src/web/studio/feed.ts +1 −1
| @@ -44,7 +44,7 @@ import { appsFirst } from "../../actions/first_ask.ts"; | ||
| 44 | 44 | |
| 45 | 45 | /** Events that can change what the feed says. */ |
| 46 | 46 | export const FEED_LIVE = |
| 47 | "run.finished,issue.opened,issue.commented,pull.commented,pull.opened,pull.merged,pull.reviewed,pull.closed,look.taken,app.serving,issue.blocked,issue.unblocked,crew.governed,issue.closed,decision.*,thought.received,thought.placed"; | |
| 47 | "run.finished,issue.opened,issue.commented,pull.commented,pull.opened,pull.merged,pull.reviewed,pull.closed,look.taken,app.serving,app.gated,beat.ran,issue.blocked,issue.unblocked,crew.governed,issue.closed,decision.*,thought.received,thought.placed"; | |
| 48 | 48 | |
| 49 | 49 | export const FEED_FIRST_PAGE = 20; |
| 50 | 50 |
templates/app-template.json +1 −1
Diff not shown — a line 15,877 characters wide (limit 2,000). +1 −1
View filetest/beat.test.ts added +154
| @@ -0,0 +1,154 @@ | ||
| 1 | /** | |
| 2 | * The beat a repository carries (#13709): parsed at the boundary, kept on time, and heard in the | |
| 3 | * feed. | |
| 4 | */ | |
| 5 | ||
| 6 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 7 | import { mkdtemp, rm } from "node:fs/promises"; | |
| 8 | import { tmpdir } from "node:os"; | |
| 9 | import { join } from "node:path"; | |
| 10 | import { autonomyTick } from "../src/crew/autonomy.ts"; | |
| 11 | import { createApp, type App } from "../src/app.ts"; | |
| 12 | import { commitToBranch, createRepo } from "../src/actions/repos.ts"; | |
| 13 | import { parseBeat } from "../src/apps/beat_spec.ts"; | |
| 14 | import { parseManifest } from "../src/apps/manifest.ts"; | |
| 15 | import { beatDue, beatWords, driveOrBeat, type Drive, type Drove } from "../src/crew/beat.ts"; | |
| 16 | import { type Repo } from "../src/core/repos.ts"; | |
| 17 | import { createUser, type User } from "../src/core/users.ts"; | |
| 18 | ||
| 19 | describe("parseBeat", () => { | |
| 20 | test("unset is none; every defaults to an hour", () => { | |
| 21 | expect(parseBeat(undefined)).toEqual({ ok: true, value: undefined }); | |
| 22 | const empty = parseBeat({}); | |
| 23 | expect(empty.ok && empty.value?.everyMs).toBe(3_600_000); | |
| 24 | }); | |
| 25 | test("reads numbers and units", () => { | |
| 26 | const r = parseBeat({ every: "30m" }); | |
| 27 | expect(r.ok && r.value).toEqual({ every: "30m", everyMs: 1_800_000 }); | |
| 28 | const d = parseBeat({ every: "2d" }); | |
| 29 | expect(d.ok && d.value?.everyMs).toBe(172_800_000); | |
| 30 | }); | |
| 31 | test("refuses what it cannot read", () => { | |
| 32 | for (const bad of ["1w", "0h", "5m", "8d", "1h30m", 60, null, ["1h"], "", "-1h"]) { | |
| 33 | expect(parseBeat(bad === null ? null : { every: bad }).ok).toBe(false); | |
| 34 | } | |
| 35 | expect(parseBeat([]).ok).toBe(false); | |
| 36 | expect(parseBeat("1h").ok).toBe(false); | |
| 37 | const typo = parseBeat({ evry: "1h" }); | |
| 38 | expect(!typo.ok && typo.error).toContain("evry"); | |
| 39 | }); | |
| 40 | test("a manifest carries it and refuses a bad one", () => { | |
| 41 | const good = parseManifest(JSON.stringify({ app: { run: "bun x.ts" }, beat: { every: "1h" } }), undefined); | |
| 42 | expect(good.ok && good.value.beat?.every).toBe("1h"); | |
| 43 | expect(parseManifest(JSON.stringify({ app: { run: "bun x.ts" }, beat: { every: "1y" } }), undefined).ok).toBe(false); | |
| 44 | }); | |
| 45 | }); | |
| 46 | ||
| 47 | describe("beatDue and beatWords", () => { | |
| 48 | test("due when never run or the interval has passed", () => { | |
| 49 | expect(beatDue(undefined, 1000, 5)).toBe(true); | |
| 50 | expect(beatDue(0, 1000, 999)).toBe(false); | |
| 51 | expect(beatDue(0, 1000, 1000)).toBe(true); | |
| 52 | }); | |
| 53 | test("every turn has words, and the why-not ones say not", () => { | |
| 54 | expect(beatWords("picked")).toContain("started"); | |
| 55 | expect(beatWords("idle")).toContain("nothing ready"); | |
| 56 | expect(beatWords("paused")).toContain("not started"); | |
| 57 | }); | |
| 58 | }); | |
| 59 | ||
| 60 | describe("runBeats", () => { | |
| 61 | let dataDir: string; | |
| 62 | let app: App; | |
| 63 | let alice: User; | |
| 64 | let repo: Repo; | |
| 65 | ||
| 66 | beforeAll(async () => { | |
| 67 | dataDir = await mkdtemp(join(tmpdir(), "forge-beat-")); | |
| 68 | app = createApp({ dataDir, port: 0 }); | |
| 69 | const a = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 70 | if (!a.ok) throw new Error(a.error); | |
| 71 | alice = a.value; | |
| 72 | const made = await createRepo(app, alice, { name: "beating", visibility: "private" }); | |
| 73 | if (!made.ok) throw new Error(String(made.error)); | |
| 74 | repo = made.value; | |
| 75 | const seeded = await commitToBranch(app, alice, repo, { message: "Start", files: [{ path: "forge.json", content: JSON.stringify({ beat: { every: "1h" } }) }] }); | |
| 76 | if (!seeded.ok) throw new Error(JSON.stringify(seeded.error)); | |
| 77 | }); | |
| 78 | afterAll(async () => { | |
| 79 | app.close(); | |
| 80 | await rm(dataDir, { recursive: true, force: true }); | |
| 81 | }); | |
| 82 | ||
| 83 | const seen: string[] = []; | |
| 84 | const turn = (did: Drove, turns: string[]): Drive => async (_a, r) => { | |
| 85 | turns.push(r.id); | |
| 86 | return did; | |
| 87 | }; | |
| 88 | ||
| 89 | test("a due beat takes one turn, waits its hour, and tells the feed only when the outcome changes", async () => { | |
| 90 | app.bus.subscribe((e) => { | |
| 91 | if (e.kind === "beat.ran") seen.push(String(e.payload["says"])); | |
| 92 | }); | |
| 93 | const turns: string[] = []; | |
| 94 | const t0 = new Date("2026-11-15T10:00:00Z"); | |
| 95 | const at = (min: number): Date => new Date(t0.getTime() + min * 60_000); | |
| 96 | const go = (min: number, did: Drove): Promise<Drove> => driveOrBeat(app, repo, at(min), new Map(), turn(did, turns)); | |
| 97 | expect(await go(0, "idle")).toBe("idle"); | |
| 98 | expect(await go(59, "idle")).toBe("waiting"); | |
| 99 | expect(turns).toHaveLength(1); | |
| 100 | expect(await go(60, "idle")).toBe("idle"); | |
| 101 | expect(turns).toHaveLength(2); | |
| 102 | expect(seen).toEqual(["nothing ready to ship"]); // the same outcome twice: said once | |
| 103 | expect(await go(120, "full")).toBe("full"); | |
| 104 | expect(await go(180, "picked")).toBe("picked"); | |
| 105 | expect(await go(240, "picked")).toBe("picked"); // something started: said each time | |
| 106 | expect(seen).toHaveLength(4); | |
| 107 | expect(seen[3]).toContain("started"); | |
| 108 | // Autonomy off for the turn: nothing is said. | |
| 109 | expect(await go(300, "off")).toBe("off"); | |
| 110 | expect(seen).toHaveLength(4); | |
| 111 | }); | |
| 112 | ||
| 113 | test("a repository without a beat takes the autonomy loop's own turn every time", async () => { | |
| 114 | const made = await createRepo(app, alice, { name: "plain", visibility: "private" }); | |
| 115 | if (!made.ok) throw new Error(String(made.error)); | |
| 116 | const seeded = await commitToBranch(app, alice, made.value, { message: "Start", files: [{ path: "README.md", content: "x\n" }] }); | |
| 117 | if (!seeded.ok) throw new Error(JSON.stringify(seeded.error)); | |
| 118 | const turns: string[] = []; | |
| 119 | const t = new Date("2026-11-15T10:00:00Z"); | |
| 120 | await driveOrBeat(app, made.value, t, new Map(), turn("idle", turns)); | |
| 121 | await driveOrBeat(app, made.value, t, new Map(), turn("idle", turns)); | |
| 122 | expect(turns).toHaveLength(2); | |
| 123 | }); | |
| 124 | ||
| 125 | test("a beat that cannot be read is said once in the feed, and the repository stays on the loop", async () => { | |
| 126 | const made = await createRepo(app, alice, { name: "badbeat", visibility: "private" }); | |
| 127 | if (!made.ok) throw new Error(String(made.error)); | |
| 128 | const seeded = await commitToBranch(app, alice, made.value, { message: "Start", files: [{ path: "forge.json", content: JSON.stringify({ beat: { every: "1y" } }) }] }); | |
| 129 | if (!seeded.ok) throw new Error(JSON.stringify(seeded.error)); | |
| 130 | const before = seen.length; | |
| 131 | const turns: string[] = []; | |
| 132 | const t = new Date("2026-11-15T10:00:00Z"); | |
| 133 | await driveOrBeat(app, made.value, t, new Map(), turn("idle", turns)); | |
| 134 | await driveOrBeat(app, made.value, t, new Map(), turn("idle", turns)); | |
| 135 | expect(turns).toHaveLength(2); | |
| 136 | expect(seen.slice(before)).toHaveLength(1); | |
| 137 | expect(seen[before]).toContain("beat not kept"); | |
| 138 | }); | |
| 139 | ||
| 140 | test("the autonomy tick gives a beat-carrying repository its turn once per beat, in its own pass", async () => { | |
| 141 | app.db.query("UPDATE repo SET autonomy = 'autonomous', autonomy_by = ?2 WHERE id = ?1").run(repo.id, alice.id); | |
| 142 | const heard: string[] = []; | |
| 143 | app.bus.subscribe((e) => { | |
| 144 | if (e.kind === "beat.ran" && e.repoId === repo.id) heard.push(String(e.payload["did"])); | |
| 145 | }); | |
| 146 | const t = new Date("2026-12-01T10:00:00Z"); | |
| 147 | await autonomyTick(app, t, async () => 0); | |
| 148 | expect(heard).toHaveLength(1); | |
| 149 | // A minute later the beat is not due: the repository's turn is "waiting", and nothing new is | |
| 150 | // said. | |
| 151 | await autonomyTick(app, new Date(t.getTime() + 60_000), async () => 0); | |
| 152 | expect(heard).toHaveLength(1); | |
| 153 | }); | |
| 154 | }); |
test/feed-reasons.test.ts +12
| @@ -22,6 +22,7 @@ import { createUser, type User } from "../src/core/users.ts"; | ||
| 22 | 22 | import { Affinity } from "../src/feed/affinity.ts"; |
| 23 | 23 | import { plainWords } from "../src/feed/plain.ts"; |
| 24 | 24 | import { candidatesFor, eventCandidates, type Candidate, type FeedKind } from "../src/feed/items.ts"; |
| 25 | import { FEED_LIVE } from "../src/web/studio/feed.ts"; | |
| 25 | 26 | import { rank, REASON_MAX } from "../src/feed/rank.ts"; |
| 26 | 27 | import { quietLogs } from "./helpers/globals.ts"; |
| 27 | 28 | |
| @@ -113,6 +114,8 @@ const events = (): Record<string, ForgeEvent> => ({ | ||
| 113 | 114 | tell: event("issue.opened", { number: 8 }), |
| 114 | 115 | reply: event("issue.commented", { number: 8, body: `see ${ULID} and ${OID}` }), |
| 115 | 116 | map: event("issue.blocked", { number: 8, blockedBy: 9 }), |
| 117 | gated: event("app.gated", { app: "site", env: "production", commitOid: OID, keptOid: "b".repeat(40), why: "1 fail" }), | |
| 118 | beat: event("beat.ran", { every: "1h", did: "idle", says: "nothing ready to ship" }), | |
| 116 | 119 | syncFailed: event("adapter.sync_failed", { reason: `repo ${UUID} refused ${ULID}` }), |
| 117 | 120 | }); |
| 118 | 121 | |
| @@ -144,6 +147,15 @@ test("the builders emit every kind", () => { | ||
| 144 | 147 | expect(kinds.has(k as FeedKind)).toBe(true); |
| 145 | 148 | }); |
| 146 | 149 | |
| 150 | test("a gated push and a beat are feed items, and the live feed hears them", () => { | |
| 151 | const cs = built(); | |
| 152 | expect(cs.find((c) => c.what.startsWith("a push was held back"))?.what).toContain("bbbbbbb keeps serving"); | |
| 153 | expect(cs.find((c) => c.what === "beat: nothing ready to ship")?.kind).toBe("run"); | |
| 154 | const live = FEED_LIVE.split(","); | |
| 155 | expect(live).toContain("app.gated"); | |
| 156 | expect(live).toContain("beat.ran"); | |
| 157 | }); | |
| 158 | ||
| 147 | 159 | test("an id in an event's payload never reaches the clause", () => { |
| 148 | 160 | const cs = built(); |
| 149 | 161 | const sync = cs.find((c) => c.what.startsWith("sync failed"))!; |
test/job-slices.test.ts +1
| @@ -278,6 +278,7 @@ const UNTIMED: Record<string, string> = { | ||
| 278 | 278 | prompts: "a closure in startPromptRoutine; the fixture has no prompt routines", |
| 279 | 279 | timers: "fires due app timers over HTTP to the apps; the fixture has no apps", |
| 280 | 280 | "ledger-git": "commits the cost ledger chain to the ledger branch at most hourly: a git commit and signature in a data directory; the fixture has no ledger directory, and test/ledger-git.test.ts runs the tick", |
| 281 | "push-gate-budget": "a one-shot timer in apps/push_gate.ts that gives up on a push gate's tests at its budget: one resolve of a promise; push-gate tests drive the budget directly", | |
| 281 | 282 | "run-log-flush": "a one-shot timer in runs/log.ts that appends buffered lines to the run log files: I/O, no fixture runs", |
| 282 | 283 | "spawn-restart": "a one-shot backoff timer in spawn/stopped.ts that restarts a stopped spawn; the fixture has no spawns", |
| 283 | 284 | "origin-broker-sweep": "a one-shot timer a minute after a brokered sign-in, and once on boot: one DELETE of its spent and expired rows; the fixture brokers none", |
test/push-gate.test.ts added +199
| @@ -0,0 +1,199 @@ | ||
| 1 | /** | |
| 2 | * The gate on a direct push (#13709), through a live apps runtime: a push to the default branch | |
| 3 | * whose declared tests fail does not switch, and the last green commit keeps serving. | |
| 4 | */ | |
| 5 | ||
| 6 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 7 | import { mkdtemp, rm } from "node:fs/promises"; | |
| 8 | import { tmpdir } from "node:os"; | |
| 9 | import { join } from "node:path"; | |
| 10 | import { createApp, type App } from "../src/app.ts"; | |
| 11 | import { AppsRuntime } from "../src/apps/runtime.ts"; | |
| 12 | import { gateBudgetMs, gatesPush, gateSummary, withinBudget } from "../src/apps/push_gate.ts"; | |
| 13 | import { newApp } from "../src/actions/apps.ts"; | |
| 14 | import { getAppByRepo } from "../src/core/apps.ts"; | |
| 15 | import { commitToBranch } from "../src/actions/repos.ts"; | |
| 16 | import type { ForgeEvent } from "../src/core/events.ts"; | |
| 17 | import type { Repo } from "../src/core/repos.ts"; | |
| 18 | import { listRuns, type Run } from "../src/core/runs.ts"; | |
| 19 | import { tellGated } from "../src/apps/gate_notice.ts"; | |
| 20 | import { setNotifyUrl } from "../src/crew/notify.ts"; | |
| 21 | import { ok } from "../src/kernel/result.ts"; | |
| 22 | import { createUser, type User } from "../src/core/users.ts"; | |
| 23 | import { serve, type Servers } from "../src/http/server.ts"; | |
| 24 | ||
| 25 | describe("gatesPush", () => { | |
| 26 | test("only a direct push to production with something to keep and a declared test", () => { | |
| 27 | expect(gatesPush("production", "push", true, true)).toBe(true); | |
| 28 | expect(gatesPush("production", "push", false, true)).toBe(false); | |
| 29 | expect(gatesPush("production", "push", true, false)).toBe(false); | |
| 30 | expect(gatesPush("production", "merge", true, true)).toBe(false); | |
| 31 | expect(gatesPush("pull-3", "push", true, true)).toBe(false); | |
| 32 | }); | |
| 33 | test("the summary names the commit that keeps serving", () => { | |
| 34 | const line = gateSummary("test failed: exit 1\nmore", "abcdef0123456789"); | |
| 35 | expect(line).toContain("abcdef0 keeps serving"); | |
| 36 | expect(line).not.toContain("more"); | |
| 37 | }); | |
| 38 | }); | |
| 39 | ||
| 40 | describe("the gate's budget", () => { | |
| 41 | test("a suite that outlives it is red; one inside it keeps its verdict", async () => { | |
| 42 | const hung = await withinBudget(new Promise(() => {}), 20); | |
| 43 | expect(!hung.ok && hung.error).toContain("did not finish"); | |
| 44 | expect((await withinBudget(Promise.resolve(ok()), 1000)).ok).toBe(true); | |
| 45 | }); | |
| 46 | test("the budget is ten minutes unless the environment sets a sane one", () => { | |
| 47 | expect(gateBudgetMs(undefined)).toBe(600_000); | |
| 48 | expect(gateBudgetMs("5000")).toBe(5000); | |
| 49 | for (const bad of ["0", "10", "abc", "-5", "99999999", "1e3"]) expect(gateBudgetMs(bad)).toBe(600_000); | |
| 50 | }); | |
| 51 | }); | |
| 52 | ||
| 53 | describe("tellGated", () => { | |
| 54 | test("sends one line to the owner's address; a failing sender, or no address, never throws", async () => { | |
| 55 | const dataDir = await mkdtemp(join(tmpdir(), "forge-gate-notice-")); | |
| 56 | const app = createApp({ dataDir, port: 0 }); | |
| 57 | try { | |
| 58 | const u = await createUser(app.db, { origin: "seed", trusted: true, username: "bob", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 59 | if (!u.ok) throw new Error(u.error); | |
| 60 | const sent: string[][] = []; | |
| 61 | const send = async (url: string, line: string, h?: Record<string, string>): Promise<boolean> => { | |
| 62 | sent.push([url, line, h?.["title"] ?? ""]); | |
| 63 | return true; | |
| 64 | }; | |
| 65 | expect(await tellGated(app.db, u.value.id, "held back", send)).toBe(false); | |
| 66 | expect(sent).toHaveLength(0); | |
| 67 | expect(setNotifyUrl(app.db, u.value.id, "https://ntfy.example.com/topic").ok).toBe(true); | |
| 68 | expect(await tellGated(app.db, u.value.id, "held back", send)).toBe(true); | |
| 69 | expect(sent).toEqual([["https://ntfy.example.com/topic", "held back", "forge"]]); | |
| 70 | const boom = async (): Promise<boolean> => { | |
| 71 | throw new Error("down"); | |
| 72 | }; | |
| 73 | expect(await tellGated(app.db, u.value.id, "held back", boom)).toBe(false); | |
| 74 | } finally { | |
| 75 | app.close(); | |
| 76 | await rm(dataDir, { recursive: true, force: true }); | |
| 77 | } | |
| 78 | }); | |
| 79 | }); | |
| 80 | ||
| 81 | describe("a direct push is gated on its tests", () => { | |
| 82 | let dataDir: string; | |
| 83 | let app: App; | |
| 84 | let servers: Servers; | |
| 85 | let runtime: AppsRuntime; | |
| 86 | let alice: User; | |
| 87 | let repo: Repo; | |
| 88 | ||
| 89 | const when = <T>(look: (event?: ForgeEvent) => T | undefined, what: string): Promise<T> => | |
| 90 | new Promise((resolve, reject) => { | |
| 91 | const done = (value: T): void => { | |
| 92 | unsubscribe(); | |
| 93 | clearTimeout(timer); | |
| 94 | resolve(value); | |
| 95 | }; | |
| 96 | const unsubscribe = app.bus.subscribe((event) => { | |
| 97 | const value = look(event); | |
| 98 | if (value !== undefined) done(value); | |
| 99 | }); | |
| 100 | const timer = setTimeout(() => { | |
| 101 | unsubscribe(); | |
| 102 | reject(new Error(`${what} did not happen in time`)); | |
| 103 | }, 25_000); | |
| 104 | const now = look(); | |
| 105 | if (now !== undefined) done(now); | |
| 106 | }); | |
| 107 | ||
| 108 | const settledAt = (oid: string): Promise<Run> => | |
| 109 | when( | |
| 110 | () => | |
| 111 | listRuns(app.db, repo.id, { kind: "deploy", limit: 50 }).find( | |
| 112 | (r) => r.env === "production" && r.commitOid === oid && (r.state === "ok" || r.state === "failed"), | |
| 113 | ), | |
| 114 | "the deploy settling", | |
| 115 | ); | |
| 116 | ||
| 117 | const push = async (test: string): Promise<string> => { | |
| 118 | const committed = await commitToBranch(app, alice, repo, { | |
| 119 | branch: repo.defaultBranch, | |
| 120 | base: repo.defaultBranch, | |
| 121 | message: `test: ${test}`, | |
| 122 | files: [{ path: "forge.json", content: `${JSON.stringify({ app: { run: "bun run src/server.ts", test }, note: test })}\n` }], | |
| 123 | }); | |
| 124 | if (!committed.ok) throw new Error(JSON.stringify(committed.error)); | |
| 125 | return committed.value.commitOid; | |
| 126 | }; | |
| 127 | ||
| 128 | beforeAll(async () => { | |
| 129 | dataDir = await mkdtemp(join(tmpdir(), "forge-gate-")); | |
| 130 | app = createApp({ dataDir, port: 0 }); | |
| 131 | servers = await serve(app); | |
| 132 | runtime = new AppsRuntime(app, { domain: "apps.test", scheme: "http", publicPort: servers.port, forgeUrl: servers.url, concurrency: 2 }); | |
| 133 | await runtime.start(); | |
| 134 | const created = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 135 | if (!created.ok) throw new Error(created.error); | |
| 136 | alice = created.value; | |
| 137 | const made = await newApp(app, alice, { name: "gated", visibility: "public" }); | |
| 138 | if (!made.ok) throw new Error(made.error); | |
| 139 | repo = made.value.repo; | |
| 140 | const first = await when( | |
| 141 | () => listRuns(app.db, repo.id, { kind: "deploy", limit: 5 }).find((r) => r.env === "production" && r.state === "ok"), | |
| 142 | "first production deploy", | |
| 143 | ); | |
| 144 | expect(first.state).toBe("ok"); | |
| 145 | }, 40_000); | |
| 146 | ||
| 147 | afterAll(async () => { | |
| 148 | await runtime.stop(); | |
| 149 | servers.stop(); | |
| 150 | app.close(); | |
| 151 | await rm(dataDir, { recursive: true, force: true }); | |
| 152 | }); | |
| 153 | ||
| 154 | test("red tests keep the last green commit serving and say so; green switches", async () => { | |
| 155 | const record = getAppByRepo(app.db, repo.id)!; | |
| 156 | const green = await push("true"); | |
| 157 | expect((await settledAt(green)).state).toBe("ok"); | |
| 158 | const servingBefore = runtime.liveFor(record, "production")?.commitOid; | |
| 159 | expect(servingBefore).toBe(green); | |
| 160 | ||
| 161 | const gated = when( | |
| 162 | (e) => (e?.kind === "app.gated" && e.repoId === repo.id ? e : undefined), | |
| 163 | "the gated event", | |
| 164 | ); | |
| 165 | const red = await push("exit 1"); | |
| 166 | const failed = await settledAt(red); | |
| 167 | expect(failed.state).toBe("failed"); | |
| 168 | expect(failed.summary).toContain("held back"); | |
| 169 | expect(failed.summary).toContain(`${green.slice(0, 7)} keeps serving`); | |
| 170 | const event = await gated; | |
| 171 | expect(event.payload["keptOid"]).toBe(green); | |
| 172 | expect(runtime.liveFor(record, "production")?.commitOid).toBe(green); | |
| 173 | ||
| 174 | }); | |
| 175 | ||
| 176 | test("a suite that outlives the budget is red too, and does not hold the deploy", async () => { | |
| 177 | const record = getAppByRepo(app.db, repo.id)!; | |
| 178 | const green = runtime.liveFor(record, "production")?.commitOid; | |
| 179 | process.env["FORGE_GATE_BUDGET_MS"] = "1000"; | |
| 180 | try { | |
| 181 | const hung = await push("sleep 30"); | |
| 182 | const slow = await settledAt(hung); | |
| 183 | expect(slow.state).toBe("failed"); | |
| 184 | expect(slow.summary).toContain("held back"); | |
| 185 | expect(slow.summary).toContain("did not finish"); | |
| 186 | expect(runtime.liveFor(record, "production")?.commitOid).toBe(green); | |
| 187 | } finally { | |
| 188 | delete process.env["FORGE_GATE_BUDGET_MS"]; | |
| 189 | } | |
| 190 | ||
| 191 | }); | |
| 192 | ||
| 193 | test("a fixed push switches in again", async () => { | |
| 194 | const record = getAppByRepo(app.db, repo.id)!; | |
| 195 | const fixed = await push("true"); | |
| 196 | expect((await settledAt(fixed)).state).toBe("ok"); | |
| 197 | expect(runtime.liveFor(record, "production")?.commitOid).toBe(fixed); | |
| 198 | }, 60_000); | |
| 199 | }); |
new above
offline
The beat is a feature added ahead of the pipeline list the owner set. Split it from the push gate, and explain how it serves the direction.