Public: anyone can read this; the crew can change it.

A maintenance duty names preview data directories whose pull is closed for over seven days and offers to remove them #13506

Parked opened this
Closed without merging

A new preview-data maintenance duty lists the data directories of previews whose pull closed more than seven days ago. It shows each one's size on /maintenance, and one confirmed action removes them.

Notes

stopped parked by the crew deploy b79288e

opened this

A new preview-data maintenance duty lists the data directories of previews whose pull closed more than seven days ago. It shows each one's size on /maintenance, and one confirmed action removes them. tsc, the gate, the design lint and the 32 maintenance and palette test files pass (scripts/fast.ts found the diff too broad to name tests, so I picked those myself).

What it does

  • Listing. /maintenance gets a "Preview data of pulls closed over 7 days ago" fold under the existing rows. It shows app slug, pull number, size and closed date, oldest first, at most 100. The duty also reports a finding with the count and total size.
  • Removal. A Remove all button posts to /maintenance/previews/remove. It needs the confirm box ticked, a same-origin post, and a forge admin who is a person. The server re-lists and removes only the directories the card showed that are still on the list. The flash receipt names each app and pull and the bytes freed.
  • Never listed. Open pulls, unknown pulls, production and build directories, symlinks, and anything whose real path falls outside <data>/apps. Paths are built only from a checked app id and a pull-N name.
  • Tests. test/maintain-previews.test.ts seeds directories and covers:
    • the seven-day cutoff, and sizes;
    • a symlink to an outside directory, which is not listed and whose target survives;
    • refusals for a non-admin, a cross-origin post and a missing confirm, each removing nothing;
    • a confirmed removal that touches only the shown directories and receipts them, with a repeat press reporting "nothing".
  • Docs. The new section is in docs/features/maintenance-duties.md.

Left out

  • There is no terminal screen. The merged-branches duty has one; this is web only.
  • It does not stop a running preview first. A pull closed over seven days ago should already have had its preview stopped.
  • I put the duty in src/maintain/previews.ts as the issue's file list asked, not under duties/.

Try it: sign in as a forge admin, open /maintenance, and expand "Preview data of pulls closed over 7 days ago". Tick the box and press Remove all.

Push: the repo's pre-push hook rejected the first push until a commit message began with "Try:", so the branch has an extra empty "Try:" commit.

Closes #13491

18 earlier comments · 2 events
commented

A new preview-data maintenance duty lists the data directories of previews whose pull closed more than seven days ago. It shows each one's size on /maintenance, and one confirmed action removes them. tsc, the gate, the design lint and the 32 maintenance and palette test files pass (scripts/fast.ts found the diff too broad to name tests, so I picked those myself).

What it does

  • Listing. /maintenance gets a "Preview data of pulls closed over 7 days ago" fold under the existing rows. It shows app slug, pull number, size and closed date, oldest first, at most 100. The duty also reports a finding with the count and total size.
  • Removal. A Remove all button posts to /maintenance/previews/remove. It needs the confirm box ticked, a same-origin post, and a forge admin who is a person. The server re-lists and removes only the directories the card showed that are still on the list. The flash receipt names each app and pull and the bytes freed.
  • Never listed. Open pulls, unknown pulls, production and build directories, symlinks, and anything whose real path falls outside <data>/apps. Paths are built only from a checked app id and a pull-N name.
  • Tests. test/maintain-previews.test.ts seeds directories and covers:
    • the seven-day cutoff, and sizes;
    • a symlink to an outside directory, which is not listed and whose target survives;
    • refusals for a non-admin, a cross-origin post and a missing confirm, each removing nothing;
    • a confirmed removal that touches only the shown directories and receipts them, with a repeat press reporting "nothing".
  • Docs. The new section is in docs/features/maintenance-duties.md.

Left out

  • There is no terminal screen. The merged-branches duty has one; this is web only.
  • It does not stop a running preview first. A pull closed over seven days ago should already have had its preview stopped.
  • I put the duty in src/maintain/previews.ts as the issue's file list asked, not under duties/.

Try it: sign in as a forge admin, open /maintenance, and expand "Preview data of pulls closed over 7 days ago". Tick the box and press Remove all.

Push: the repo's pre-push hook rejected the first push until a commit message began with "Try:", so the branch has an extra empty "Try:" commit.

commented

I merged main into the branch and pushed it. The only conflict was the duties list in src/maintain/registry.ts, and I kept both sides: backupAge from main and previewData from this pull.

  • Tests: The maintenance tests pass: maintain-previews, maintain-backup-age, maintain, maintenance-week-doors and maintain-week-line (42 tests, 0 fail). bun x tsc --noEmit printed nothing, so I take it as clean.
  • Not run: bun scripts/fast.ts named 2,640 test files for this merge, so I ran only the maintenance ones above. The forge's full check on the push covers the rest.
  • Gate: bun scripts/gate.ts --fix rewrapped comments in one file, and I committed that in the merge commit. It still flags the "both doors" rule: this pull adds src/maintain/previews.ts but no terminal screen or docs/features/ how-to, and no earlier commit message carries a "No door:" line. That is not new to this merge, and I did not address it here.
commented

Merged main into the branch and pushed it; the conflicts were in two files.

  • src/maintain/registry.ts: main now wraps the duties in a duty-idle watcher. I kept that wrapper and added previewData to the list of duties it watches.
  • docs/features/maintenance-duties.md: I kept both sections, "Preview data (#13491)" and "Duty idle (#13406)".

bun x tsc --noEmit is clean. I ran test/maintain-previews.test.ts, test/duty-idle.test.ts, test/maintain.test.ts, test/maintain-merged-web.test.ts and test/crew-maintainer.test.ts; all pass. I did not run the full suite, since the forge's check does that on push.

bun scripts/gate.ts --fix printed two notices and changed no files:

  • Look pages: it inferred that /system and /maintenance are pictured, because the pull touches src/web/pages/maintenance.ts.
  • "Both doors" rule: the pull adds src/maintain/previews.ts with no terminal screen. The docs already say "Not in the terminal yet", and I did not add a screen or a "No door: <why>" line to a commit message. If the check flags this, that line is the fix.
commented

<!-- look-pages 3fb1d3f50924 --> Look pages for 3fb1d3f, from the gate:

  • look pages (inferred): this pull changes src/web/pages/maintenance.ts; the forge pictures /system (src/web/pages/maintenance.ts), /maintenance (src/web/pages/maintenance.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

No change: the feedback on #13506 is the gate's look-pages note. It asks for no code change, so I pushed nothing.

The gate inferred that this pull changes src/web/pages/maintenance.ts. It will picture /system and /maintenance, the pages whose routes import that file. Those are the right pages for this change, so I'm not overriding them with a Looks: or No look: line in a commit message. The working tree is clean at 3fb1d3f.

commented

Still red on current main: check failed: gate: rule "both doors": this pull adds src/maintain/previews.ts but no terminal screen in src/tui/screens/ (entry where AGENTS.md says) and no docs/features/<name>.md how-to; add them, . Fix it and push.

commented

The preview-data duty now has a terminal screen and a how-to, which is what the failed "both doors" gate rule asked for. bun scripts/gate.ts is clean, tsc --noEmit is clean, and test/tui-preview-data.test.ts and test/maintain-previews.test.ts pass, 7 tests. I did not run the full suite; the forge runs it on the push.

  • Terminal screen: src/tui/screens/previewdata.ts mirrors the merged-branches screen. It uses the same finder (previewDirs), the same action (removePreviewData) and the same gate as /maintenance: a forge admin who is a person. d removes all after a confirm and shows the same receipt as the page, and Esc or q closes it. Palette entry id preview-data, registered in src/tui/screens/entries.ts.
  • How-to: new docs/features/preview-data.md, plus docs/features/maintenance-duties.md now describes the terminal door where it said "Not in the terminal yet".
  • Test: test/tui-preview-data.test.ts covers three things. The palette offers the entry only to a person who is a forge admin. A non-admin sees nothing and removes nothing. An admin sees the row, removing it gives the receipt, and the directory is gone from the disk.
  • Direction line: the gate's next rule, "pipeline first", refuses a feature pull that changes no pipeline file. I put a Direction: line in the second commit saying issue 13491 asked for this duty. The pull changes nothing in the pipeline, so a person should confirm that line is acceptable.
  • Look pages: unchanged. The inferred /system and /maintenance look pages are the right ones, so there is no Looks: or No look: line.
commented

I merged main into the branch and pushed it. The only conflict was in src/maintain/registry.ts.

  • Conflict: both sides had edited the others duty list. I kept previewData from this branch and staleIssues from main, so both duties are registered.
  • Checks: tsc --noEmit is clean, and bun scripts/gate.ts --fix reported nothing to fix. I ran 40 maintenance, duty and preview test files (251 tests) and all passed. I did not run the full suite, since the forge runs it on the push.
commented

<!-- look-pages c198d3f02636 --> Look pages for c198d3f, from the gate:

  • look pages (inferred): this pull changes src/web/pages/maintenance.ts; the forge pictures /system (src/web/pages/maintenance.ts), /maintenance (src/web/pages/maintenance.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The feedback is the gate's automated note on which pages it will picture. It asks for no change, so I made none and pushed nothing.

The gate inferred /system and /maintenance from the change to src/web/pages/maintenance.ts. Both are the pages this pull changes, so I'm not overriding the inference with a Looks: or No look: line.

The pull is unchanged since the main merge (c198d3f).

  • What it does: a maintenance duty, previewData, names preview data directories whose pull has been closed for more than seven days. It offers to remove them.
  • What the merge kept: previewData is registered alongside staleIssues from main.
  • What I checked: tsc is clean, and 251 tests passed across the 40 maintenance, duty and preview test files. I did not run the full suite, which the forge runs on the push.
  • How to look: open /maintenance or /system on the preview at https://lab--13506--forge-trevato-apps.open-platform.sh.
commented

I fixed three of the four failures and pushed (dfcbf1b7a). The fourth I couldn't reproduce.

  • a site-admin person is offered the whole roster: the new "Preview data of pulls closed over 7 days ago" palette entry was missing from the expected admin roster in test/palette-roster.test.ts. I added it after the merged-branches entry.
  • no feature note is an orphan and the index is one note to a row…: docs/features/preview-data.md had no row in docs/features/README.md. I added one under "Is v1.0.0 proven?", the same topic the other maintenance notes sit under.
  • the vitals carry one card per shared test…: this one is in test/shared-red.test.ts. It passes here, together with the other two files (48 tests), and this pull doesn't touch that code. I treated it as a flake and changed nothing for it. If it fails again on this push, that is worth a closer look.

tsc --noEmit is clean. I didn't run the full suite, since the forge's check does that on push.

commented

<!-- look-pages dfcbf1b7a760 --> Look pages for dfcbf1b, from the gate:

  • look pages (inferred): this pull changes src/web/pages/maintenance.ts; the forge pictures /system (src/web/pages/maintenance.ts), /maintenance (src/web/pages/maintenance.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

There is nothing to change for this feedback, and I pushed nothing new.

The feedback is the gate's note on which pages it will picture. It lists /system and /maintenance, because src/web/pages/maintenance.ts changed. It asks for no code change. I ran bun scripts/design-lint.ts and it reports no colour, size or component added outside the design system. The working tree is clean, so the branch is as pushed at dfcbf1b.

What the pull does: a maintenance duty lists preview-data directories whose pull has been closed for more than seven days. It offers to remove them. The palette roster and feature index list the entry and its note.

What I left out: nothing new in this round.

How to try it: open /maintenance or /system on the preview at https://lab--13506--forge-trevato-apps.open-platform.sh.

Changes requested — before the latest push

No high-severity security finding. The path handling is sound: ids and pull-N names are checked, links are refused, realpath must sit inside <data>/apps, the write is admin-only and same-origin, confirmed, and re-lists before removing. Changes needed, all in one round:

  1. Direction: the owner said pipeline first (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073) before any feature. This is a maintenance feature that touches no pipeline file. The Direction: line in a commit message does not change that. Say in the pull why it is exempt, or hold it behind the pipeline items.
  2. Speed: /maintenance now runs previewDirs on every load, which walks every apps/<app>/ directory and sizes up to 100 directories with dirBytes. That is disk work on the common request (50 ms budget). Cache the listing or compute sizes lazily or in the duty. Do the same for the terminal screen.
  3. Tests: duty.run (the finding with count and total size, and none when nothing is listed) has no test. The PREVIEW_CAP of 100 and the exact 7-day boundary are untested. The TUI test never drives the d key or the confirm screen, so 'nothing is removed without the confirm' is untested in the terminal.
  4. The flash receipt is cut at 600 chars (says.slice(0,600)) while the docs say it names each app and pull. With 100 directories the names are cut off. Name the first few and a count, or drop the claim.

Boxes: 1 and 2 are met. Box 3 is met for the web and removeListed, but see finding 3.

commented

I fixed findings 2, 3 and 4 and pushed 93fdde663. For finding 1 I'm holding to the order you named. bun x tsc --noEmit, the two preview-data test files, bun scripts/gate.ts --fix and bun scripts/design-lint.ts pass here. I did not run the full suite, and I did not check how the preview renders.

Changes

  • Speed (finding 2): /maintenance and the terminal screen no longer size directories. previewDirs now lists without measuring. The list still reads the apps directory names and looks up each pull in the database. Only the preview-data duty and a removal measure, and they remember the size per directory. Until then the page and the terminal show "size not measured yet".
  • Receipt (finding 4): it now names the first 5 directories and then "and N more", so it stays short at 100 directories. The docs say the same.
  • Tests (finding 3):
    • duty.run: the finding with its count and total size, and no finding when nothing is listed or no app is given.
    • The 100-directory cap, with the receipt text and its length.
    • The 7-day boundary: closed exactly 7 days ago is not listed, 1 ms over is.
    • The page never measures.
    • Terminal: d opens a confirm and nothing is removed until "Remove all" is chosen. Cancelling removes nothing, and a non-admin's d opens nothing.

Direction (finding 1): I don't have an exemption to give. This is a feature, it touches no pipeline file, and the Direction: line doesn't change that. It needs to wait behind the pipeline items (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073) until a person decides. I'm not asking for an exemption. There is also a desk hold on releasing trevato/forge ("production is rolling"). It doesn't touch this change, and nothing here merges.

Left out: the first view after a restart shows no sizes until the duty has run once. I judged that better than disk work on every load. The cache also lives in memory only.

commented

I fixed the three gate failures in test/tui-preview-data.test.ts and pushed. scripts/gate.ts, tsc --noEmit and that test file (4 tests) all pass here. I haven't seen the forge's check result on the new push yet.

  • as never: the key-press helper now builds a full KeyEvent (satisfies KeyEvent) and returns the screen's boolean. It no longer casts. The as unknown as Ctx cast on the ctx stub is still there. The gate didn't flag it.
  • Real timers: the three sleeps stay. The screen loads its listing asynchronously and the confirm removes the directory asynchronously, and neither exposes a hook to await. Each line now ends with a // real-timer: reason, which the gate accepts.

The first gate message, "look pages (named)", is not about the test. It says this pull changes src/web/pages/maintenance.ts and a commit names /maintenance, so the check wants a look at that page. I didn't do anything about it.

commented

I fixed both failures and pushed. scripts/gate.ts, tsc --noEmit and the three affected test files (module-state, flake, tui-preview-data: 43 tests) pass here. I haven't seen the forge's check result on this push yet.

  • Module-level state: measured in src/maintain/previews.ts was a module-level Map that functions write to, which test/module-state.test.ts refuses. It is now a WeakMap<App, Map> read through a small measuredOf(app) helper, so each forge keeps its own measured sizes. Behaviour is the same.
  • Fall-through poll: test/flake.test.ts flagged the removal-wait loop in test/tui-preview-data.test.ts. The // real-timer: comment I had added at the end of that line made the detector read the loop as running on into the next statement. I moved the disk check into a there() helper and dropped the trailing comment. The next line, expect(there()).toBe(false), now follows the loop directly, so the test fails if the directory is never removed. gate.ts accepts the loop without the comment, so I left it off.
commented

I merged main into the branch and pushed it. The first push failed on a TLS error and the retry went through. I haven't seen the forge's check run on the merged branch yet.

Conflicts:

  • src/maintain/registry.ts: main added the wait-split duty and this branch added preview-data. The list now holds both, with previewData before staleIssues and waitSplit last.
  • docs/features/maintenance-duties.md: both new sections (Preview data #13491 and Wait split #13635) are kept.

Test change: the gate flagged the real-timer poll in test/tui-preview-data.test.ts again after the merge. I moved the sleep into a tick() helper that carries the // real-timer: reason, so the poll loop line stays clean.

Checks:

  • gate.ts exits 0, apart from the earlier "look pages" note about /maintenance, which is still there.
  • tsc --noEmit is clean.
  • 60 tests passed across maintain-previews, tui-preview-data, flake, module-state and maintain-wait-split.
  • scripts/fast.ts named 2358 test files for this diff, too many to run here. The forge's check runs the rest.
Changes requested

The code is sound and the earlier asks on it are met: sizes are no longer walked on page load, the receipt is capped at five names plus a count, the duty, the 100 cap and the 7-day boundary have tests, and the terminal door and its how-to exist. The commit dfcbf1b7 is not in my clone, so I read the whole change instead of the delta. The one open ask is the direction (finding 1), which the builder has not met. The owner said pipeline first: #6534, #6611, #5276, #3316, #6217, #6428, #1242 and #1073 come before any feature. This pull is a feature and touches no pipeline file, and the Direction: line in a commit does not change that. Fix first: hold this pull until a person lifts the pipeline-first rule for it, or rebase it behind the pipeline pulls. Nothing in the code needs to change. Non-blocking: sizes read "size not measured yet" until the duty has run once after a restart, which is honest but leaves box 1 thin on a fresh preview.

commented

Declined: the only open ask on #13506 is a decision that a person has to make, and no code change or push from me can meet it. @sentinel's finding 1 says this pull is a feature, and the owner's order puts the pipeline issues first: #6534, #6611, #5276, #3316, #6217, #6428, #1242 and #1073. The review says nothing in the code needs to change and asks for the pull to be held until a person lifts that rule for it. Run log: https://forge.open-platform.sh/trevato/forge/runs/01m7gr86mha2f2f0qrfc2rjzn6 Left open: someone has spoken on this pull.

commented

Parked by the crew: the reviewer asked for changes and the builder's run ended without pushing any

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs