The throughput keeper (#7645)

The forge watches its own landing rate, names the one thing binding it, and acts where the act is reversible and its own. src/flow/keeper.ts ticks once a minute. It reads the database and asks no model, so it still speaks when the subscription is out.

  1. Measure (keeper_rate.ts): pulls landed per hour over the last 3 h against the best 3 h rate of the week before, and the share of the last 24 hours that ran (a landing or a builder run started). Low is under half the best and under one an hour.
  2. Find (keeper_cause.ts): the first of ceiling, usage ("resets at HH:MM UTC"), unassigned, empty_pillar, stale_reds, red_checks, stale_pile, capacity. It asks the dispatcher's own questions (heldNow, atCeiling, pace).
  3. Act: unassigned sweeps assigned issues with no run. empty_pillar, red_checks and stale_pile become one filed issue in the forge's own repository, once a week per cause (stale_pile's issue lists the ten oldest waiting pulls, red and conflicted ones left out, each with what it waits on: a reviewer, a builder or the merge). ceiling, usage, capacity and stale_reds (the crew's red-pull sweep is already at it) are only said.
  4. Check after 2 h: landings rose and the act stands, or it is undone, the cause is left alone for 6 h and the owner is asked one question with one button (POST /flow/keeper/answer).

Where it shows: one line on home and on the boards panel, keeper in the MCP flow report, the terminal's Flow dialog (the line, the question, and l for its one button after a confirm) and its crew scorecard ("Forge per day" rows), and "Forge per day" (hours running, landed) on the crew scorecard and in scorecard (keeperDays). Every attempt is a row in keeper_attempt.

A red_checks issue lists the red pulls, oldest first, so whoever picks it up knows which to rework.

An empty_pillar issue lists the ready issues and the hold on each (landed, open_pull, stalled...), so a person sees why nothing can start.

Two DORA numbers (keeper_measures.ts, #13110), over the last 7 days, each with the count behind it, on the flow panel (once the week has a reading), in the terminal's Flow dialog (which also says "No reading") and in keeper.measures of the MCP flow report and its API answer. Change-fail rate: landed pulls whose next finished main check was red, or, in the forge's own repository, whose next release was rolled back (a red production proof). Revert rate: landed pulls a revert pull landed for within seven days of their landing. Fewer than five landings in the week is "No reading", never a rate.

Still unmeasured: queue-wait p95 and rework per agent from the issue discussion. The two rates are approximations: "next check" is the first main check created within 24 hours after the landing (a release likewise: cut within 24 hours), so a red that one of several batched landings caused is counted against each; a pull landed in the last seven days may yet be reverted, so the revert rate can only rise; a release proof is matched by time, not by the commits it carried, and only for the forge's own repository; the week is rolling, not a calendar week, and no history of past weeks is kept.

Who reads what: a site admin reads the sentence with repository names and the question with its button. Everyone else, guests and agents included, reads the cause with no repository in it (readKeeper(db, full)). The Flow only switch is retired (#7771), so no act asks a question today; one already on the books is answered with its button, and the answer is a keeper.answered event carrying the owner's id.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs