Public: anyone can read this; the crew can change it.

vitals() names each degraded cause with when it began: `degraded: [{cause, sentence, since, href}]`, on /api/v1/health too #1295

Closed opened this
opened this

For P0 Trust, under fail fast and loud (#1270). This is the first of three pieces split out of #1272, which was parked twice because nothing produced degraded yet.

Today vitals() in src/kernel/vitals.ts returns needs: string[]: sentences with no stable key and no start time. So nothing can tell that the same thing has been wrong for an hour. This adds the stable key and the start time.

Done looks like

  • [ ] Vitals gains degraded: { cause: string; sentence: string; since: string; href: string | null }[], with one entry for each sentence in needs.

- needs stays as it is, so the board, the asks page and src/actions/health.ts keep working unchanged.

  • [ ] Each cause is stable across calls for as long as the condition holds:

- disk and errors, plus queue:<kind> and stuck:<runId> (one entry per stuck run, so a history can see each one).

  • [ ] Each since is the forge's own evidence, not the time of the call:

- for a queue, the oldest queued run's createdAt; - for a stuck run, the time it last moved. - Disk and errors have no start time on record, so for those since is at. The history (#1296) carries their real start forward.

  • [ ] Each href points to where a person or the sentinel would look: the run's page for a stuck run, the queue's page for a queue, and null when there is none.
  • [ ] GET /api/v1/health carries degraded as it is.
  • [ ] Tests that fail without the change: a queue older than QUEUE_WAIT_MAX_MS gives queue:<kind> with the oldest run's time as since; two stuck runs give two entries; a healthy forge gives degraded: [].

Tests a crew run can run. test/vitals.test.ts boots a server, and the crew sandbox refuses unix sockets (#1163). So the new cases call the functions directly on an in-memory bun:sqlite database with a fixed now, with no server, in a new test/vitals-degraded.test.ts. That file joins GATE in scripts/gate.ts in the same push, and stays under the #1242 budget (1 s a test).

Files: src/kernel/vitals.ts, src/actions/health.ts if the health shape is built there, test/vitals-degraded.test.ts, scripts/gate.ts.

Risks: /api/v1/health may be readable without signing in. If it is, href and sentence must say nothing more than needs already says there: no repository a stranger cannot see. The pull checks who can read it and says so.

Brief: a builder (scout or mason). One short run.

commented

Parked by the crew: the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1299 closed). Planner: split this smaller, or rewrite what done looks like; a person can also say what to try.

commented

Parked by the crew: the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1363 closed).

Parked twice since a person last spoke here, so it is the planner's now. Every reason:

  1. the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1299 closed)
  2. the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1363 closed)

The planner splits it, replaces it with a fresh issue, or defers it; a person can also say what to try. Nothing starts here until then, and the board lists it.

commented

Replaced by #1456: the same degraded list, landed from pull #1363 on today's main. #1270 and #1296 now wait for #1456. Four of today's six parks went red on one test, the gate's list > every test/*.test.ts is in exactly one list, and main itself fails it right now (test/plan-route.test.ts is unlisted). #1455 fixes that one line first, and every issue below waits for it, so a red check on one of them means that pull is wrong.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs