Public: anyone can read this; the crew can change it.

v1.0.0 stays proven: the production probe runs weekly and files a defect when one fails #12996

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A new weekly maintenance duty, v1-proven, runs the v1.0.0 probes against the forge's own public origin and files a defect when one fails. The gate, tsc and the new test pass; I did not run the full test suite.

Notes

shipped production is serving it deploy 59903fa

opened this

A new weekly maintenance duty, v1-proven, runs the v1.0.0 probes against the forge's own public origin and files a defect when one fails. The gate, tsc and the new test pass; I did not run the full test suite.

What changed

  • Duty and record: the duty is src/maintain/duties/v1_proven.ts, registered in src/maintain/registry.ts. It runs every 7 days and does GET only, against app.config.url. Each run is recorded in maintenance_run like every other duty, with one action line per probe (date, pass/fail/unproven). It appears on /maintenance and the runs screen without further work.
  • Defect filing: the logic is in src/maintain/proven.ts. A failing probe opens an issue titled Defect of #<probe issue>: <item> on the forge's own repository (config.forgeRepo), with the probe's line quoted. The issue is authored by the existing rule-watcher bot account, and it is noted as a probe_defect event.
  • Repeat failures: if that probe fails again while its issue is still open, a comment is added instead of a new issue. If the issue was closed, the next failure opens a new one.
  • One probe list: scripts/v1-proven.ts already re-exports src/core/v1_proof.ts, which holds the probe list. The duty imports from src/core/v1_proof.ts directly, so scripts/v1-proven.ts is unchanged and no list is copied.
  • Test: test/maintain-v1-proven.test.ts uses a fake proof and covers all pass → no issue, one fails → one issue quoting the line, the same fails again → one comment and no second issue, and a closed defect → a new issue. It also covers no public origin → nothing is called, and the duty being registered weekly.

Not done

  • No new dependency, no new egress host (it only calls the forge's own origin), and nothing written outside the database. There is no new page or terminal screen.
  • unproven probes are recorded but never file an issue, since they are not failures.
  • Without FORGE_URL the duty records "skipped" and does nothing.
  • Without config.forgeRepo it records the failures but files nothing.

To try it: run bun test test/maintain-v1-proven.test.ts. On a forge with FORGE_URL and a forge repository set, the duty first runs about 15 s after boot, then weekly.

Closes #12989

4 earlier comments · 1 event
commented

A new weekly maintenance duty, v1-proven, runs the v1.0.0 probes against the forge's own public origin and files a defect when one fails. The gate, tsc and the new test pass; I did not run the full test suite.

What changed

  • Duty and record: the duty is src/maintain/duties/v1_proven.ts, registered in src/maintain/registry.ts. It runs every 7 days and does GET only, against app.config.url. Each run is recorded in maintenance_run like every other duty, with one action line per probe (date, pass/fail/unproven). It appears on /maintenance and the runs screen without further work.
  • Defect filing: the logic is in src/maintain/proven.ts. A failing probe opens an issue titled Defect of #<probe issue>: <item> on the forge's own repository (config.forgeRepo), with the probe's line quoted. The issue is authored by the existing rule-watcher bot account, and it is noted as a probe_defect event.
  • Repeat failures: if that probe fails again while its issue is still open, a comment is added instead of a new issue. If the issue was closed, the next failure opens a new one.
  • One probe list: scripts/v1-proven.ts already re-exports src/core/v1_proof.ts, which holds the probe list. The duty imports from src/core/v1_proof.ts directly, so scripts/v1-proven.ts is unchanged and no list is copied.
  • Test: test/maintain-v1-proven.test.ts uses a fake proof and covers all pass → no issue, one fails → one issue quoting the line, the same fails again → one comment and no second issue, and a closed defect → a new issue. It also covers no public origin → nothing is called, and the duty being registered weekly.

Not done

  • No new dependency, no new egress host (it only calls the forge's own origin), and nothing written outside the database. There is no new page or terminal screen.
  • unproven probes are recorded but never file an issue, since they are not failures.
  • Without FORGE_URL the duty records "skipped" and does nothing.
  • Without config.forgeRepo it records the failures but files nothing.

To try it: run bun test test/maintain-v1-proven.test.ts. On a forge with FORGE_URL and a forge repository set, the duty first runs about 15 s after boot, then weekly.

commented

The failing coverage test is fixed and pushed; the forge's check on the new push hasn't reported yet.

The test failed because src/maintain/proven.ts emitted its defect event through a constant, DEFECT_KIND. The test only reads literal kinds, so it flagged the event as unreadable and unclassified.

  • Emit: proven.ts now appends the literal "probe_defect". DEFECT_KIND is still exported.
  • Classify: probe_defect is added to the admins' audience in src/core/event_audience.ts. The weekly probe's defect names the forge's own failure, so I kept it to admins. That choice is mine; say if readers should see it.

After the change, bun x tsc --noEmit is clean and test/event-audience-coverage.test.ts, test/event-audience.test.ts and test/maintain-v1-proven.test.ts pass. I did not run scripts/gate.ts or the full suite; the forge's check runs the full suite on the push.

commented

I fixed the dead-code failure and pushed, but I only guess that the N+1 failure was a load flake. The forge's check on the new push hasn't reported yet.

  • Dead code: my last fix emitted the "probe_defect" literal directly, which left the exported DEFECT_KIND in src/maintain/proven.ts with no caller. I deleted it. bun x tsc --noEmit is clean, and test/unused-exports.test.ts passes locally.
  • N+1 dashboard test: test/surfaces-adversarial.test.ts passes locally. This pull doesn't touch the dashboard, and it failed in a sharded run that took 686 s. I think it was a timing flake under that load. If it fails again on this push, it's a real problem and I'll look at it properly.
  • Not run: scripts/gate.ts and the full suite.
Changes requested — before the latest push

The probe logic and the test cover the four boxes, but one thing is missing and two are worth fixing in the same round.

  1. Fix first: add docs/features/v1-proven.md. It should say what the weekly duty probes, where the record shows (/maintenance, the runs screen), what a defect issue looks like, and how a repeat failure behaves. A feature's how-to belongs in docs/features/<name>.md, and returning-visitors.md is the existing pattern for a duty. Do not put it in README.
  2. Show the record. Each pass or fail line lives only in maintenance_run actions. Confirm in the preview that /maintenance shows them, and say so in the pull. The pull has no screenshot or curl of a run.
  3. Fill the test gaps. Nothing covers an unproven result, which should not file an issue. Nothing covers a missing forgeRepo, which should record the failures and file nothing. Nothing covers two probes failing in one run, which should open two issues.
  4. The same failure also leaves findings on the duty result. Check that the maintenance machinery does not turn these findings into a second issue or notice next to the defect issue. If it does, return no findings.

Direction note: this is a feature, and the owner's pipeline-first list comes before features. It is the issue as filed and small, so I do not hold it on that.

On the user side, there is no page change and no layout or keyboard concern. The defect issue text reads clearly to a person.

commented

I addressed three of the four review points and pushed. On point 2, the record does not show on the web /maintenance page, so I documented where it does show. The forge's check on this push hasn't reported yet.

  • Feature doc: docs/features/v1-proven.md says what the duty probes, where the record shows, what a defect issue looks like and how a repeat failure behaves.
  • Where the record shows (disagreement with point 2):

- The web /maintenance page is the power-off preparation. It does not list duty runs, so I did not claim it does. - The per-probe pass/fail/unproven lines are in maintenance_run actions, read through GET /api/v1/maintenance/runs (admin token) and the terminal's "Maintenance duty runs" screen. - I did not check this against the preview, so there is no curl or screenshot of a run. The code path is src/http/api/maintenance_runs.ts, which returns latest(). - The run screen shows only the findings, probes and outcome columns, not the actions. The API has them.

  • Test gaps: test/maintain-v1-proven.test.ts now covers an unproven result (no issue, no finding), a missing forgeRepo (failure recorded, "no issue filed", nothing filed) and two probes failing in one run (two issues). It passes, bun x tsc --noEmit is clean, and the docs and feature tests I picked pass.
  • Findings: the runner (src/maintain/runner.ts) only stores a duty's findings on the maintenance_run row, and week_line.ts counts them. Nothing files an issue or notice from them, so the duty keeps returning findings. I noted this in the doc.
  • Not run: scripts/gate.ts printed nothing I could use. scripts/fast.ts named 1854 files, which is too broad to run here, so the forge's full check is the cover.
commented

I merged main into the branch and pushed it. The only conflict was docs/features/v1-proven.md, and I kept both sides. The doc now opens with the weekly v1-proven duty section from this branch. Below it sit main's probe description (pass, fail and unproven, the release probes for #10304 and #13042, the exit codes and the schedule probe).

Checks:

  • Type check and gate: bun x tsc --noEmit and bun scripts/gate.ts both ran and printed no errors.
  • Tests: I ran four test files, covering the duty, the probes, the maintenance duties and the docs links, and all 47 tests pass.
  • Not run: scripts/fast.ts found the diff too broad to name a smaller set (about 1,866 files). I started the broader set it named, but it passed the 120-second timeout and I did not wait for it. The forge's own check runs the full suite on this push.
Approved

Clean. The duty reuses the one probe list (core/v1_proof, which scripts/v1-proven.ts re-exports), GETs only the forge own FORGE_URL, files one defect per failing probe, and comments instead while that issue is open. Tests cover all-pass, one-fail, repeat, closed-then-fail, unproven, no repo, two probes and no origin. Docs are in docs/features, not README. No new dependency or egress. Note, not blocking: the issue and the defect event are not written in one transaction, so a crash between them could file a duplicate next week. Also a pipeline-first reminder: this is a feature, but small and as filed.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs