Public: anyone can read this; the crew can change it.
Absorb: show a feature's health on /system #9910
Check · Review · Land · Live done — Landed and live in production.
torque/forge:torque/9887-absorb-show-a-feature-s-health-on-system → main 4 commits fork 3 threads · 3 unresolved
6 files
changed +202
−3
docs/features/absorb.md +5 −1
| @@ -76,12 +76,16 @@ Settings stay behind on a fork. A fork of another forge's app carries its `featu | ||
| 76 | 76 | |
| 77 | 77 | ## Not done yet |
| 78 | 78 | |
| 79 | Until then an absorbed app is a menu link with kept settings (Settings › Features says so). The forge does not yet fire a feature's schedule (#9883), show its feed items (#9884), keep `secret` settings (#9886) or show its health on `/system` (#9887). | |
| 79 | Until then an absorbed app is a menu link with kept settings (Settings › Features says so). The forge does not yet fire a feature's schedule (#9883), show its feed items (#9884) or keep `secret` settings (#9886). | |
| 80 | 80 | |
| 81 | 81 | ## MCP tools |
| 82 | 82 | |
| 83 | 83 | A feature may declare `"tools": { "scan": { "description": "Run a scan now", "writes": true } }` (at most 12; `writes` is true unless it says `false`). The forge adds no tool to `/mcp` for them: the `features` tool lists them as `tools` with a `slug`, and `{ slug, tool, args }` calls one of an absorbed app. The forge POSTs `{ "args": … }` to the app's `/forge/tools/<name>` over loopback and returns its JSON (at most 64 KB, 10 seconds). A read tool answers at once for anyone who can read the app; a write tool is for someone who can write it, needs an `rw` token and plans first (`dryRun: false` does it). The plan of a write is told only once the call could be made (allowed, absorbed, running). These tools are MCP-only on purpose: they are an agent's door to the app, and the app's own page is the person's, so the terminal has no entry for them (a person reaches the app itself there). Code: `src/apps/feature_tools.ts`, `src/actions/feature_tools.ts`; tests: `test/feature-tools.test.ts`, `test/features.test.ts`. |
| 84 | 84 | |
| 85 | ## Health on /system | |
| 86 | ||
| 87 | The owner's `/system` page has a "Features from apps" list: for each absorbed app, production's state (`running`, `deploying`, `failed`, `asleep`, `stopped`, or `none` when the runtime holds nothing for it), the commit serving, since when, and why it is down or why a newer deploy failed. A newer failed deploy counts as unhealthy though the old one still serves. Unhealthy rows come first. The `system` MCP tool returns the same rows as `features`; an agent granted `telemetry:read` sees only apps with a public repository. It reads the runtime's state: no git read, no model. The terminal's `/system` screen draws them for the owner too (`featuresLines`). Code: `src/system/feature_health.ts`; test: `test/system-feature-health.test.ts`. | |
| 88 | ||
| 85 | 89 | ## Findings |
| 86 | 90 | |
| 87 | 91 | Absorb was built by Ben (benjivers). The review of #9441 found: |
src/system/feature_health.ts added +60
| @@ -0,0 +1,60 @@ | ||
| 1 | /** | |
| 2 | * How the apps absorbed as features are doing, now (#9887): one row per | |
| 3 | * absorbed app with its production environment's state, the commit serving | |
| 4 | * and, when a newer deploy failed or the app is down, why. A read of what | |
| 5 | * the runtime already knows: no git read, no model, nothing started. The | |
| 6 | * owner sees every absorbed app; an agent granted telemetry:read, only those | |
| 7 | * whose repository is public (the same cut `agentView` makes of processes). | |
| 8 | */ | |
| 9 | import type { App } from "../app.ts"; | |
| 10 | import type { Reader } from "../actions/telemetry.ts"; | |
| 11 | import { getAppById } from "../core/apps.ts"; | |
| 12 | import { getRepoById } from "../core/repos.ts"; | |
| 13 | import { listAbsorbed } from "../core/features.ts"; | |
| 14 | import { appsRuntime, type EnvState, type EnvStatus } from "../apps/runtime.ts"; | |
| 15 | ||
| 16 | export interface FeatureHealth { | |
| 17 | readonly slug: string; | |
| 18 | /** The menu label the admin absorbed it under. */ | |
| 19 | readonly name: string; | |
| 20 | readonly repo: string; | |
| 21 | /** Production's state; `none` when the runtime has no production environment for it. */ | |
| 22 | readonly state: EnvState | "none"; | |
| 23 | /** True when production is serving (`running`), the one state that is well. */ | |
| 24 | readonly healthy: boolean; | |
| 25 | /** The first 7 characters of the commit serving or being brought up. */ | |
| 26 | readonly commit: string | null; | |
| 27 | /** When production started serving or last changed state. */ | |
| 28 | readonly since: string | null; | |
| 29 | /** Why it is down, or why a newer deploy did not come up. */ | |
| 30 | readonly problem: string | null; | |
| 31 | } | |
| 32 | ||
| 33 | /** One row from an environment's status; `undefined` is an app the runtime holds nothing for. */ | |
| 34 | export function healthOf(base: Pick<FeatureHealth, "slug" | "name" | "repo">, env: EnvStatus | undefined): FeatureHealth { | |
| 35 | const state = env?.state ?? "none"; | |
| 36 | const problem = env?.error ?? (env?.failedDeploy ? `a newer deploy failed: ${env.failedDeploy.error}` : null); | |
| 37 | return { | |
| 38 | ...base, | |
| 39 | state, | |
| 40 | healthy: state === "running" && env?.failedDeploy === undefined, | |
| 41 | commit: env ? env.commitOid.slice(0, 7) : null, | |
| 42 | since: env?.servingAt ?? env?.since ?? null, | |
| 43 | problem: problem === null ? null : problem.slice(0, 300), | |
| 44 | }; | |
| 45 | } | |
| 46 | ||
| 47 | /** Unhealthy first, then by name. */ | |
| 48 | export function featureHealth(app: App, reader: Reader): FeatureHealth[] { | |
| 49 | const runtime = appsRuntime(app); | |
| 50 | const out: FeatureHealth[] = []; | |
| 51 | for (const row of listAbsorbed(app.db)) { | |
| 52 | const record = getAppById(app.db, row.appId); | |
| 53 | const repo = record && getRepoById(app.db, record.repoId); | |
| 54 | if (!record || !repo) continue; | |
| 55 | if (reader.kind === "agent" && repo.visibility !== "public") continue; | |
| 56 | const env = runtime?.status(record).find((e) => e.env === "production"); | |
| 57 | out.push(healthOf({ slug: record.slug, name: row.name, repo: `${repo.ownerName}/${repo.name}` }, env)); | |
| 58 | } | |
| 59 | return out.sort((a, b) => Number(a.healthy) - Number(b.healthy) || a.name.localeCompare(b.name)); | |
| 60 | } |
src/system/telemetry.ts +3 −2
| @@ -27,6 +27,7 @@ import type { Reader } from "../actions/telemetry.ts"; | ||
| 27 | 27 | import { agentView, live } from "./live.ts"; |
| 28 | 28 | import type { SystemView } from "./view.ts"; |
| 29 | 29 | import { readerPaces, type ReaderPace } from "./telemetry_reads.ts"; |
| 30 | import { featureHealth, type FeatureHealth } from "./feature_health.ts"; | |
| 30 | 31 | |
| 31 | 32 | export function systemFor(app: App, reader: Reader): SystemView { |
| 32 | 33 | const view = live().read(); |
| @@ -38,8 +39,8 @@ export function systemFor(app: App, reader: Reader): SystemView { | ||
| 38 | 39 | * rows /system draws (#9088). Names are accounts, so only a telemetry reader gets |
| 39 | 40 | * here. Call it before `countRead` so the read does not count into its own answer. |
| 40 | 41 | */ |
| 41 | export function systemWithReaders(app: App, reader: Reader): SystemView & { readers: ReaderPace[] } { | |
| 42 | return { ...systemFor(app, reader), readers: readerPaces() }; | |
| 42 | export function systemWithReaders(app: App, reader: Reader): SystemView & { readers: ReaderPace[]; features: FeatureHealth[] } { | |
| 43 | return { ...systemFor(app, reader), readers: readerPaces(), features: featureHealth(app, reader) }; | |
| 43 | 44 | } |
| 44 | 45 | |
| 45 | 46 | /** |
src/tui/screens/system.ts +22
| @@ -34,6 +34,7 @@ import { dialog } from "../ui/widgets.ts"; | ||
| 34 | 34 | import { Scroll, confirmScreen, drawLines, hitScroll, later, pickerScreen } from "./common.ts"; |
| 35 | 35 | import type { PaletteEntry } from "./entries.ts"; |
| 36 | 36 | import { readHealth } from "../../actions/health.ts"; |
| 37 | import { featureHealth } from "../../system/feature_health.ts"; | |
| 37 | 38 | import { readerPaces } from "../../system/telemetry_reads.ts"; |
| 38 | 39 | import { live as liveMachine } from "../../system/live.ts"; |
| 39 | 40 | |
| @@ -170,6 +171,25 @@ export function readerLines(session: Session, now = Date.now()): string[] { | ||
| 170 | 171 | ]; |
| 171 | 172 | } |
| 172 | 173 | |
| 174 | /** | |
| 175 | * Absorbed features and their production health, as /system's "Features from | |
| 176 | * apps" says it (#9887): one line each, the problem after it only when there | |
| 177 | * is one. Empty for anyone but the owner, and when nothing is absorbed. | |
| 178 | */ | |
| 179 | export function featuresLines(session: Session): string[] { | |
| 180 | const reader = telemetryReader(session.app, session.user); | |
| 181 | if (reader?.kind !== "owner") return []; | |
| 182 | const rows = featureHealth(session.app, reader); | |
| 183 | if (rows.length === 0) return []; | |
| 184 | return [ | |
| 185 | "Features from apps", | |
| 186 | ...rows.slice(0, READER_ROWS).map((r) => | |
| 187 | truncate(`${pad(r.name, 20)} ${pad(r.state, 8)} ${[r.commit, r.since].filter((x) => x !== null).join(" · ")}${r.problem === null ? "" : ` ${r.problem}`}`.trimEnd(), WIDTH - 6), | |
| 188 | ), | |
| 189 | ...(rows.length > READER_ROWS ? [`… and ${rows.length - READER_ROWS} more`] : []), | |
| 190 | ]; | |
| 191 | } | |
| 192 | ||
| 173 | 193 | /** The agents holding the grant, in one line. */ |
| 174 | 194 | export function grantLine(session: Session): string { |
| 175 | 195 | const held = grantable(session.db).filter((a) => a.granted).map((a) => a.username); |
| @@ -219,6 +239,7 @@ export function systemScreen(ctx: Ctx, lines: readonly string[], session: Sessio | ||
| 219 | 239 | title: "System", |
| 220 | 240 | keys: [binding(["h"], "window"), ...(offered ? [binding(["g"], "grant or take back telemetry")] : []), binding(["enter", "esc", "q"], "close")], |
| 221 | 241 | render(area, s) { |
| 242 | const features = featuresLines(session); | |
| 222 | 243 | const buttons = dialog(s, area, ctx.theme.c, { |
| 223 | 244 | title: "System", |
| 224 | 245 | body: [ |
| @@ -226,6 +247,7 @@ export function systemScreen(ctx: Ctx, lines: readonly string[], session: Sessio | ||
| 226 | 247 | "", |
| 227 | 248 | ...history, |
| 228 | 249 | ...(offered ? ["", ...readerLines(session)] : []), |
| 250 | ...(features.length === 0 ? [] : ["", ...features]), | |
| 229 | 251 | ...(offered ? ["", grantLine(session), "g: grant or take back telemetry:read"] : []), |
| 230 | 252 | ], |
| 231 | 253 | actions: [{ label: "Close" }], |
src/web/pages/system.ts +11
| @@ -36,6 +36,7 @@ import { systemLiveJs } from "../system-live.ts"; | ||
| 36 | 36 | import { RECENT_BYTES, recentEncodings } from "../../http/compress.ts"; |
| 37 | 37 | import { KEPT_PAGE_BYTES, keptPageStats } from "./kept_page.ts"; |
| 38 | 38 | import { COLUMN_BYTES, PARTS_BYTES, keptPartsStats, treeCacheStats } from "./tree_cache.ts"; |
| 39 | import { featureHealth, type FeatureHealth } from "../../system/feature_health.ts"; | |
| 39 | 40 | import { badge, chartBox, meter, pill, row } from "../ui.ts"; |
| 40 | 41 | |
| 41 | 42 | /** A group's memory, and a cell's beside its cap and who holds the cap. */ |
| @@ -132,6 +133,15 @@ export function cachesSection(rows: readonly CacheRow[]): Html { | ||
| 132 | 133 | </table></div></section>`; |
| 133 | 134 | } |
| 134 | 135 | |
| 136 | /** Absorbed apps and whether production is serving (#9887); `featureHealth`'s rows. Nothing absorbed, nothing drawn. */ | |
| 137 | export function featuresSection(rows: readonly FeatureHealth[]): Html { | |
| 138 | if (rows.length === 0) return html``; | |
| 139 | return html`<section class="settings-block" id="sys-features"><h2>Features from apps</h2> | |
Principle 1: the h2 already says what this is; this sentence restates it. Cut it, or keep only the link to Settings › Features. |
||
| 140 | <ul class="list" aria-label="Absorbed features and their health">${rows.map((r) => html`<li data-feature="${r.slug}" data-healthy="${String(r.healthy)}"><strong>${r.name}</strong> ${r.healthy ? html`<span class="small muted">${r.state}</span>` : badge(r.state, { tone: "danger" })} | |
| 141 | <span class="small muted mono">${[r.commit, r.since].filter((x) => x !== null).join(" · ")}</span>${r.problem === null ? null : html` <span class="small muted">${r.problem}</span>`}</li>`)}</ul> | |
outdated
Principle 7: six columns, mostly empty (Problem is blank for healthy rows). Draw each feature as a row: name and state first, then commit, since and the problem as muted text only when there is one. Also check principle 5: the danger badge must not add a second accent. |
||
| 142 | <p class="small"><a href="/settings/features">Settings › Features</a></p></section>`; | |
| 143 | } | |
| 144 | ||
| 135 | 145 | /** The last hour's stalls by route, most milliseconds first; the rows are `stallHour`'s. */ |
| 136 | 146 | export function stallSection(rows: readonly RouteStall[]): Html { |
| 137 | 147 | return html`<section class="settings-block"><h2>Main-thread stalls, last hour</h2> |
| @@ -253,6 +263,7 @@ export function systemPage(ctx: Ctx): Response { | ||
| 253 | 263 | <div class="js-only small muted">Sort by <button class="btn" data-size="sm" type="button" data-sort="cpu" aria-pressed="true"><kbd>c</kbd> CPU</button> |
| 254 | 264 | <button class="btn" data-size="sm" type="button" data-sort="mem" aria-pressed="false"><kbd>m</kbd> memory</button></div> |
| 255 | 265 | <div id="sys-procs">${at("sys-procs")}</div></section> |
| 266 | ${featuresSection(timePhase(ctx.request, "features", () => featureHealth(ctx.app, reader)))} | |
| 256 | 267 | <div id="sys-stalls">${stallSection(stalls)}</div> |
| 257 | 268 | <div id="sys-readers">${readersSection(readerPaces())}</div> |
| 258 | 269 | <div id="sys-caches">${cachesSection(pageCaches())}</div> |
test/system-feature-health.test.ts added +101
| @@ -0,0 +1,101 @@ | ||
| 1 | /** | |
| 2 | * An absorbed app's health on /system and in the `system` MCP answer (#9887): | |
| 3 | * the row says what production's environment says; an app the runtime does | |
| 4 | * not hold is "none", not healthy; a failed newer deploy shows although an | |
| 5 | * older one serves; unhealthy rows come first; an agent sees public apps only. | |
| 6 | */ | |
| 7 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 8 | import { mkdtemp, rm } from "node:fs/promises"; | |
| 9 | import { tmpdir } from "node:os"; | |
| 10 | import { join } from "node:path"; | |
| 11 | import { createApp, type App } from "../src/app.ts"; | |
| 12 | import { createRepo } from "../src/actions/repos.ts"; | |
| 13 | import { createApp as insertApp } from "../src/core/apps.ts"; | |
| 14 | import { absorb } from "../src/core/features.ts"; | |
| 15 | import { createUser, type User } from "../src/core/users.ts"; | |
| 16 | import { systemWithReaders } from "../src/system/telemetry.ts"; | |
| 17 | import { featureHealth, healthOf } from "../src/system/feature_health.ts"; | |
| 18 | import { featuresSection } from "../src/web/pages/system.ts"; | |
| 19 | import { Session } from "../src/tui/session.ts"; | |
| 20 | import { featuresLines } from "../src/tui/screens/system.ts"; | |
| 21 | import { getUserByName } from "../src/core/users.ts"; | |
| 22 | import type { EnvStatus } from "../src/apps/runtime.ts"; | |
| 23 | import { quietLogs } from "./helpers/globals.ts"; | |
| 24 | ||
| 25 | quietLogs(); | |
| 26 | ||
| 27 | const env = (over: Partial<EnvStatus>): EnvStatus => ({ env: "production", state: "running", commitOid: "abcdef0123456789", url: "http://x", ...over }); | |
| 28 | const base = { slug: "radar", name: "Radar", repo: "cy/radar" }; | |
| 29 | ||
| 30 | describe("healthOf", () => { | |
| 31 | test("running with nothing failed is healthy, with a short commit", () => { | |
| 32 | const h = healthOf(base, env({ servingAt: "2026-11-08T10:00:00Z" })); | |
| 33 | expect(h).toMatchObject({ state: "running", healthy: true, commit: "abcdef0", since: "2026-11-08T10:00:00Z", problem: null }); | |
| 34 | }); | |
| 35 | test("a failed environment is down and says why", () => { | |
| 36 | const h = healthOf(base, env({ state: "failed", error: "port never answered" })); | |
| 37 | expect(h).toMatchObject({ healthy: false, state: "failed", problem: "port never answered" }); | |
| 38 | }); | |
| 39 | test("a newer deploy that failed makes it unhealthy although the old one serves", () => { | |
| 40 | const h = healthOf(base, env({ failedDeploy: { commitOid: "1234567890", runId: "r", error: "tests red" } })); | |
| 41 | expect(h.state).toBe("running"); | |
| 42 | expect(h.healthy).toBe(false); | |
| 43 | expect(h.problem).toBe("a newer deploy failed: tests red"); | |
| 44 | }); | |
| 45 | test("an app the runtime holds nothing for is none, not healthy", () => { | |
| 46 | expect(healthOf(base, undefined)).toMatchObject({ state: "none", healthy: false, commit: null }); | |
| 47 | }); | |
| 48 | }); | |
| 49 | ||
| 50 | describe("the rows and where they show", () => { | |
| 51 | let dataDir: string; | |
| 52 | let app: App; | |
| 53 | beforeAll(async () => { | |
| 54 | dataDir = await mkdtemp(join(tmpdir(), "forge-fh-")); | |
| 55 | app = createApp({ dataDir, port: 0, host: "forge.test" }); | |
| 56 | const made = await createUser(app.db, { origin: "seed", trusted: true, username: "ada", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 57 | if (!made.ok) throw new Error(made.error); | |
| 58 | const ada: User = made.value; | |
| 59 | for (const [name, visibility] of [["pubapp", "public"], ["privapp", "private"]] as const) { | |
| 60 | const repo = await createRepo(app, ada, { name, visibility, description: "" }); | |
| 61 | if (!repo.ok) throw new Error(String(repo.error)); | |
| 62 | const row = insertApp(app.db, { repoId: repo.value.id, slug: name, access: visibility }); | |
| 63 | if (!row.ok) throw new Error(row.error); | |
| 64 | absorb(app.db, row.value.id, ada.id, name === "pubapp" ? "Public" : "Private", "/"); | |
| 65 | } | |
| 66 | }); | |
| 67 | afterAll(async () => { | |
| 68 | app.db.close(); | |
| 69 | await rm(dataDir, { recursive: true, force: true }); | |
| 70 | }); | |
| 71 | ||
| 72 | test("the owner sees every absorbed app; with no runtime each is none", () => { | |
| 73 | const rows = featureHealth(app, { kind: "owner" }); | |
| 74 | expect(rows.map((r) => r.name)).toEqual(["Private", "Public"]); | |
| 75 | expect(rows.every((r) => r.state === "none" && !r.healthy)).toBe(true); | |
| 76 | }); | |
| 77 | test("an agent sees public apps only", () => { | |
| 78 | expect(featureHealth(app, { kind: "agent" }).map((r) => r.name)).toEqual(["Public"]); | |
| 79 | }); | |
| 80 | test("the system answer carries them", () => { | |
| 81 | expect(systemWithReaders(app, { kind: "owner" }).features).toHaveLength(2); | |
| 82 | }); | |
| 83 | test("the section names each feature, escaped, and is empty with none", () => { | |
| 84 | const text = featuresSection([healthOf({ slug: "x", name: "<b>Radar</b>", repo: "cy/radar" }, env({}))]).text; | |
| 85 | expect(text).toContain("<b>Radar"); | |
| 86 | expect(text).not.toContain("<b>Radar"); | |
| 87 | expect(featuresSection([]).text).toBe(""); | |
| 88 | }); | |
| 89 | test("a healthy feature is a line, not a table row; an unhealthy one carries its problem", () => { | |
| 90 | const text = featuresSection([healthOf(base, env({})), healthOf({ ...base, slug: "b", name: "Bad" }, env({ state: "failed", error: "boom" }))]).text; | |
| 91 | expect(text).not.toContain("<table"); | |
| 92 | expect(text).toContain("boom"); | |
| 93 | }); | |
| 94 | test("the terminal draws them for the owner only", () => { | |
| 95 | const id = (user?: User) => ({ sshUsername: user?.username ?? "git", remoteAddress: "127.0.0.1", fingerprint: "SHA256:t", user }); | |
| 96 | const lines = featuresLines(new Session(app, id(getUserByName(app.db, "ada")))); | |
| 97 | expect(lines[0]).toBe("Features from apps"); | |
| 98 | expect(lines.join("\n")).toContain("Private"); | |
| 99 | expect(featuresLines(new Session(app, id(undefined)))).toEqual([]); | |
| 100 | }); | |
| 101 | }); |
Both doors: the terminal's
/systemscreen does not drawfeatures(src/tui/screens/system.ts). Add it there, owner-gated like the page, with a test.