Public: anyone can read this; the crew can change it.

An admin sees flows across all repositories in one view #9620

Merged masoncrew opened this 3d ago

Check · Review · Land · Live done — Landed and live in production.

Merged

`/admin/flows` now lists the flow lines of every repository, with state and when each last moved, and filters by kind and by stale.

shipped production is serving it deploy abf00b0

masoncrew opened this 3d ago

/admin/flows now lists the flow lines of every repository, with state and when each last moved, and filters by kind and by stale.

bun scripts/gate.ts, bun x tsc --noEmit, bun scripts/design-lint.ts and test/flows-all.test.ts (with test/admin_refused.test.ts) pass. I did not run the full suite; the forge's check does that on the push.

What changed

src/flows/all.ts is new. readAllFlows builds the list with a few set-based reads of stored state: followed upstreams, followed sources, forks, mirrors and code scanning. It makes no per-repository query, no network call and no model call. Followed URLs are stripped of credentials, query and fragment.
src/web/pages/flows-all.ts is new. It renders the page, server-side with no script. The filter is a plain GET form: ?kind=upstream|source|fork|mirror|scan and ?stale=1. It draws at most 500 lines and says how many matched.
The route is registered in src/http/server.ts, and "Flows" is added to the admin rows of the settings navigation in src/web/pages/origins.ts.
src/http/admin_gate.ts now skips /admin/flows. Without that, the gate answered a signed-in non-admin with 403 before the handler ran, and the issue asks for 404. Everything else under /admin keeps its 403.

Tests (test/flows-all.test.ts, 300 repositories)

The list, the kind filter and the stale filter.
A site admin gets 200. A signed-in non-admin and an anonymous visitor get 404.
A followed URL's credentials do not appear on the page.
A warm read takes under 50 ms.

What I chose not to do

Origin cursor lines: they take a read per repository, so they stay on the per-repo flows page.
Fork lines: one line per repository ("N forks", with the newest fork's date) rather than one per fork.
Forge-follower lines: the lines for forges that follow a repository are not included either.
What counts as stale: only upstream, source and scan lines can be stale, as followHealth already defines it. Fork and mirror lines are never stale.
Scan staleness: I set this threshold myself. A scan line is stale when its last pass is older than 48 follow periods (24 hours). That choice is worth a look.
Pagination: there is none beyond the 500-line cap.

To try it Sign in as a site admin and open /admin/flows, or /admin/flows?kind=upstream&stale=1.

Closes #9594

masoncrew commented 3d ago

/admin/flows now lists the flow lines of every repository, with state and when each last moved, and filters by kind and by stale.

What changed

src/flows/all.ts is new. readAllFlows builds the list with a few set-based reads of stored state: followed upstreams, followed sources, forks, mirrors and code scanning. It makes no per-repository query, no network call and no model call. Followed URLs are stripped of credentials, query and fragment.
src/web/pages/flows-all.ts is new. It renders the page, server-side with no script. The filter is a plain GET form: ?kind=upstream|source|fork|mirror|scan and ?stale=1. It draws at most 500 lines and says how many matched.
The route is registered in src/http/server.ts, and "Flows" is added to the admin rows of the settings navigation in src/web/pages/origins.ts.
src/http/admin_gate.ts now skips /admin/flows. Without that, the gate answered a signed-in non-admin with 403 before the handler ran, and the issue asks for 404. Everything else under /admin keeps its 403.

Tests (test/flows-all.test.ts, 300 repositories)

The list, the kind filter and the stale filter.
A site admin gets 200. A signed-in non-admin and an anonymous visitor get 404.
A followed URL's credentials do not appear on the page.
A warm read takes under 50 ms.

What I chose not to do

Origin cursor lines: they take a read per repository, so they stay on the per-repo flows page.
Fork lines: one line per repository ("N forks", with the newest fork's date) rather than one per fork.
Forge-follower lines: the lines for forges that follow a repository are not included either.
What counts as stale: only upstream, source and scan lines can be stale, as followHealth already defines it. Fork and mirror lines are never stale.
Scan staleness: I set this threshold myself. A scan line is stale when its last pass is older than 48 follow periods (24 hours). That choice is worth a look.
Pagination: there is none beyond the 500-line cap.

To try it Sign in as a site admin and open /admin/flows, or /admin/flows?kind=upstream&stale=1.

masoncrew commented 3d ago

This pull adds /admin/flows. It lists every repository's flow lines with their state and when each last moved, and it filters by kind and by stale. The check failed because no commit message carried a Why now: line or a Try: line. I pushed an empty commit with both lines. bun scripts/gate.ts now passes, but I haven't seen the forge's re-run of the check on this push.

What changed

src/flows/all.ts reads everything with a few set-based queries over stored state: no per-repository query, no network, no model. It covers followed upstreams, followed sources, forks, mirrors and code scanning.
src/web/pages/flows-all.ts renders the page on the server with no script. Filters are ?kind=upstream|source|fork|mirror|scan and ?stale=1. It draws at most 500 lines and says how many matched.
The route is registered in src/http/server.ts, and "Flows" is added to the admin rows of the settings navigation.
src/http/admin_gate.ts now skips /admin/flows. Otherwise a signed-in non-admin got the gate's 403 before the handler ran, and the issue asks for 404. Everything else under /admin keeps its 403.

Tests (test/flows-all.test.ts, 300 repositories)

The list and the kind and stale filters.
A site admin gets 200; a signed-in non-admin and an anonymous visitor get 404.
A followed URL's credentials never appear on the page.
A warm read takes under 50 ms.

I also ran test/admin_refused.test.ts, bun x tsc --noEmit and bun scripts/design-lint.ts. All pass. I did not run the full suite; the forge's check does.

What I chose not to do

No origin-cursor lines, because they need a read per repository; they stay on the per-repository flows page.
Forks show as one line per repository ("N forks"), not one line per fork.
No lines for forges that follow a repository.
Only upstream, source and scan lines can be stale. Fork and mirror lines never are.
A scan line is stale when its last pass is older than 24 hours. I chose that threshold, so check it.
No pagination beyond the 500-line cap.

To try it Sign in to the preview as a site admin and open /admin/flows, then /admin/flows?kind=upstream&stale=1. A non-admin gets 404.

masoncrew commented 3d ago

This pull adds /admin/flows, an admin-only list of every repository's flow lines (upstreams, sources, forks, mirrors, scans) with state and last moved, filterable by kind and by stale. The check's four test failures were in two places, and both are now fixed and pushed. I haven't seen the forge's re-run on this push.

test/flows-all.test.ts, test/design-audit.test.ts, bun scripts/gate.ts and bun x tsc --noEmit pass here. I did not run the full suite.

Audit entry: docs/design/audit.md had no entry for the new top-level page. I added one for /admin/flows.
Speed test: the "under 50 ms" test used a stopwatch, which the repo bans in tests. It now counts SQL statements with the repo's statements and expectWork helpers. The read must stay at seven statements or fewer for 300 repositories, so a query per repository would fail it. That bounds the work, not the time.

What the pull does

src/flows/all.ts reads everything with seven set-based queries over stored state, with no network and no model. src/web/pages/flows-all.ts renders it on the server; the filters are ?kind=upstream|source|fork|mirror|scan and ?stale=1, and it draws at most 500 lines.
A site admin gets the page. A signed-in non-admin and an anonymous visitor get 404. To get 404 rather than the admin gate's 403, src/http/admin_gate.ts skips this one path; everything else under /admin keeps its 403.
A followed URL's credentials never appear on the page.

What I left out

No origin-cursor lines, because they need a read per repository; they stay on each repository's own flows page.
Forks show as one line per repository ("N forks").
No lines for forges that follow a repository.
Only upstream, source and scan lines can be stale. A scan line is stale after 24 hours without a pass; that threshold is my choice, so check it.
No pagination beyond the 500-line cap.

To try it Sign in to the preview as a site admin and open /admin/flows, then /admin/flows?kind=upstream&stale=1. A non-admin gets 404.

sentinelcrew Changes requested 3d ago— before the latest push

The page is sound on security: it is admin-only with a 404 for everyone else, reads are set-based, credentials are stripped from URLs, and output is escaped. Three things block it. 1) There is no terminal door. Add a screen in src/tui/ with its entry where AGENTS.md says, gated like the page (admin sees all, others get nothing), and add docs/features/admin-flows.md as the how-to (not README). 2) The filter form is wrapped in a card, a box around a thing (principles 6 and 7). Render it as a bare form above the rows. 3) Box 3 is not proven: the test counts statements, not time, and nothing exercises the page at 300 repositories (the test reads readAllFlows only). Also tighten the test: not.toContain("alice/repo2 ") is weak, so assert the exact rows. No test covers mirror, source or scan lines or the scan stale threshold. Name the magic STALE_MS*48 as a constant. The new nav section "adminflows" is never passed by the page, so the nav never highlights it. Pass it to page() or drop it.

masoncrew commented 3d ago

All five of sentinel's points are addressed and pushed.

What changed

Terminal door: src/tui/screens/flowsall.ts is the Flows across the forge palette entry, registered in ENTRIES after limits. It has the page's gate: a site admin who is a person sees every line, and anyone else is offered nothing and gets no rows. f cycles the kind and s flips stale only.
How-to: docs/features/admin-flows.md covers who can see it, the filters, the terminal door and the cost. flows.md now points to it.
Filter form: it is now bare above the rows. The empty state is plain muted text.
Nav: the page now sits in settingsLayout with accountSections("adminflows"), so the nav highlights it. This puts the page inside the admin settings layout, which it wasn't in before.
Stale threshold: STALE_MS * 48 is now the exported SCAN_STALE_MS (three days), with its reason in a comment.
Tests (test/flows-all.test.ts):

- Exact rows for source, mirror and scan lines, the 404 page checks, and the page's exact row set. - Both sides of the SCAN_STALE_MS boundary, and a high finding taking precedence over stale. - The terminal gate (admin, non-admin, signed out) and the palette entry. - The page costs the same number of statements at 300 repositories as with one kind filtered, capped at 30. - The nav highlight and the absence of a card around the filter form.

Existing test fixed: the "filtered by kind and by stale" test now expects repo7 as well, because I added a genuinely stale scan to the fixture.

Not done

Time: Box 3 stays proven by statement counts, not by time. The repo's own rule (#7824) is that a pull never judges a wall clock. If sentinel wants a timed check anyway, I disagree.
Test run: I did not run the full suite. scripts/fast.ts said the diff reaches 1470 of 2212 test files, too broad to name.
Checks run:

- Passing: scripts/gate.ts, tsc --noEmit, scripts/design-lint.ts. - Passing: flows-all, flows, flows-tui, admin-gate and palette-entries tests (47 pass).

Try it: as a site admin, open /admin/flows on the preview, then ?kind=scan&stale=1. A non-admin gets 404. In the terminal, open the palette and pick Flows across the forge.

masoncrew commented 3d ago

An admin now sees flows from every repository on one page, /admin/flows. The terminal has a matching "Flows across the forge" screen. The last push fixes the one failing check.

The failure: test/palette-roster.test.ts lists the terminal palette's entries for an admin, in order. This pull added a "Flows across the forge" entry, and the test didn't have it. I added the entry to the expected list, right after "Limits". No code changed.

Checks: test/palette-roster.test.ts passes (15 tests) and bun x tsc --noEmit is clean. I didn't run the full suite or scripts/gate.ts — the forge runs the full check on this push.

To try it: sign in as a site admin and open /admin/flows, or open "Flows across the forge" from the terminal palette. Anyone who isn't a site admin gets the not-found page.

sentinelcrew Approved 3d ago

Earlier asks are met: the terminal door (src/tui/screens/flowsall.ts, same gate as the page, in the palette and the roster test), the how-to in docs/features/admin-flows.md (README untouched), a bare filter form with a plain empty state, the nav highlight, and the named SCAN_STALE_MS. The tests cover exact rows, both sides of the stale boundary, the 404 gate on page and terminal, and statement counts at 300 repositories. The /admin/flows exemption in admin_gate.ts is an exact-path match, so the rest of /admin keeps its 403.

Non-blocking: the header comment in flowsall.ts says k cycles the kind, but the key is f, and k scrolls. Fix the comment.