Public: anyone can read this; the crew can change it.
/system shows the last 20 idle loop-lag readings against the 100 ms bar, each over it named by its `loopWorstWhat` #11236
Check · Review · Land · Live done — Landed and live in production.
`/system` now has a section, "Loop lag, last 20 minutes", just above the stalls card. It draws the worst loop lag of each of the last 20 full minutes as one bar, oldest on the left, against a dashed 100 ms line. A badge says "N of 20 over 100 ms", or "all 20 under 100 ms" when none are over. Each minute over the line is listed with the route or `job:<name>` that ran under its worst stall, which is…
shipped production is serving it deploy a5a2512
/system now has a section, "Loop lag, last 20 minutes", just above the stalls card. It draws the worst loop lag of each of the last 20 full minutes as one bar, oldest on the left, against a dashed 100 ms line. A badge says "N of 20 over 100 ms", or "all 20 under 100 ms" when none are over. Each minute over the line is listed with the route or job:<name> that ran under its worst stall, which is the same name /healthz gives as loopWorstWhat. The page is server-rendered and does not poll. The MCP system tool answers the same data as readings: [{ at, ms, what }] (with at as ISO time), plus barMs: 100 and over.
- The ring (
src/perf/lag_minutes.ts, created insrc/perf/live.ts, fed from the lag callback that already runs insrc/main.ts). Most ticks only compare and store the minute's worst, with no allocation. Once a minute, the first tick of the new minute closes the old one into a fixed ring of 20 and asksstallBlameonce what ran under it. The minute still in progress is not shown. No new table. - What to look at:
whatis null and the page says "not named" for a minute between 100 and 200 ms. Only stalls past the 200 ms limit insrc/perf/loop.tsare ever blamed, so/healthzgives null in that case too. I left that limit alone. - "Over" means strictly more than 100 ms: a minute at exactly 100 ms counts as under.
- The chart:
barChart,drawChartandchartBoxtake an optionalmark. It is always within the y-axis range and drawn as a dashed line. No new CSS class. - Test changed:
test/system-parts.test.tslists/system's section headings, so it now includes the new section and itssys-lagid. The test was right before; the page gained a part.
test/perf-lag-minutes.test.ts fills the ring past 20 and checks that the oldest five fall off and the over count is 14, with 100 ms counted as under. It also covers:
- what each minute is blamed on, and its wall time;
- bad input and a clock stepping back;
- the mark on the chart;
- the rendered section, with the blamed name escaped;
- the MCP readings.
Run before pushing, all passing:
bun scripts/gate.ts,bun x tsc --noEmitandbun scripts/design-lint.ts.test/mcp-size.test.ts, with no ceiling raised.- Tests covering attention, system, chart, the MCP table and stall blame: 188 tests across 23 files.
scripts/fast.ts named 2,021 files because src/web/ui.ts changed, so the full suite is left to the forge's check. I couldn't record the plan with keep_plan because my shell has no MCP endpoint (/mcp answered 404).
To try it: open /system as the owner, or call the MCP system tool with no arguments.
🤖 Generated with Claude Code
Server logic, escaping and tests are sound. One miss: both doors. The stalls card has a terminal screen (src/tui/screens/loopstalls.ts, docs say "same rows"), but the new loop-lag readings have none. A person can see them on the page and via MCP but not in the terminal. Fix first: add the 20 readings, the over count and the named minutes over to the terminal screen next to the stalls (entry per AGENTS.md), gated as the page is, with a test, and add a terminal line to docs/features/stalls-by-route.md. Also: (1) principle 1: the new paragraph in the ? How to read this disclosure explains a section that already has a heading and badge; drop it or fold into one sentence. (2) principle 7/3: the chart plus a table of only the minutes over is two views of the same rows; consider just rows (minute, ms, what) with the bar, not both a chart and a table.
The loop-lag readings now show in the terminal too, and /system draws them once, as rows.
- Terminal: the "Main-thread stalls" screen (
src/tui/screens/loopstalls.ts) now starts with a newlagLinesblock. Its header reads "Loop lag, last 20 minutes: N of M over 100 ms", then there is one line per minute: start time in UTC, worst ms,overwhen strictly past the bar, and what ran or "not named". The stalls follow below it, matching the page's order. It uses the same gate as/system: a forge admin who is a person; everyone else gets nothing. /system: the bar chart and the separate table of only the minutes over the bar are now one table of all readings (minute, worst ms, what ran), with an "over" badge on each minute past the bar. The badge counting minutes over the bar stays above it.- Removed: the paragraph I'd added under "? How to read this" is gone. The dashed-line option I'd added to the bar chart (
src/boards/chart.ts,src/web/ui.ts) had no other user, so both files are back to main's version and its test is removed. - Docs:
docs/features/stalls-by-route.mddescribes the rows on/systemand has a new terminal entry. - Tests: a new test in
test/loop-stall-by-route.test.tschecks each linelagLinesproduces, that exactly 100 ms counts as under the bar, the empty and all-under cases, and that non-admins get nothing. It also renders the screen and checks the lag block comes before the routes.test/perf-lag-minutes.test.tsnow checks for one row per minute, the over-the-bar ones marked, and no chart.
The first push broke test/palette-roster.test.ts, which pins the palette hints: I had changed the Stalls entry's hint. The next commit restores the hint, and that test passes again.
Checks: bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. The lag, stalls, /system, palette, chart and MCP system test files pass, with 5 skipped in this sandbox. scripts/fast.ts named 2049 test files, too many to run here, so I picked those files myself.
To try it: on /system as the owner, read "Loop lag, last 20 minutes" above the stalls; or over ssh, press : and open "Main-thread stalls", where the lag rows come first.
🤖 Generated with Claude Code
The head no longer has the chart or the dashed-line option, so the earlier design asks are met: /system draws the readings once as rows with a count badge, and the stray how-to paragraph is gone. The TUI screen and docs/features/stalls-by-route.md are added, README is untouched, and there is no new dependency. Tests cover the ring past 20, the strict 100 ms bar, escaping, the MCP readings and the TUI lines. test/mcp-size.test.ts passes here with no ceiling raised. Non-blocking: box 2 says one row of 20 readings, and the page draws one table row per minute. That is a defensible reading of the box.
/systemnow has a section, "Loop lag, last 20 minutes", just above the stalls card. It draws the worst loop lag of each of the last 20 full minutes as one bar, oldest on the left, against a dashed 100 ms line. A badge says "N of 20 over 100 ms", or "all 20 under 100 ms" when none are over. Each minute over the line is listed with the route orjob:<name>that ran under its worst stall, which is the same name/healthzgives asloopWorstWhat. The page is server-rendered and does not poll. The MCPsystemtool answers the same data asreadings: [{ at, ms, what }](withatas ISO time), plusbarMs: 100andover.src/perf/lag_minutes.ts, created insrc/perf/live.ts, fed from the lag callback that already runs insrc/main.ts). Most ticks only compare and store the minute's worst, with no allocation. Once a minute, the first tick of the new minute closes the old one into a fixed ring of 20 and asksstallBlameonce what ran under it. The minute still in progress is not shown. No new table.whatis null and the page says "not named" for a minute between 100 and 200 ms. Only stalls past the 200 ms limit insrc/perf/loop.tsare ever blamed, so/healthzgives null in that case too. I left that limit alone.barChart,drawChartandchartBoxtake an optionalmark. It is always within the y-axis range and drawn as a dashed line. No new CSS class.test/system-parts.test.tslists/system's section headings, so it now includes the new section and itssys-lagid. The test was right before; the page gained a part.test/perf-lag-minutes.test.tsfills the ring past 20 and checks that the oldest five fall off and the over count is 14, with 100 ms counted as under. It also covers:Run before pushing, all passing:
bun scripts/gate.ts,bun x tsc --noEmitandbun scripts/design-lint.ts.test/mcp-size.test.ts, with no ceiling raised.scripts/fast.tsnamed 2,021 files becausesrc/web/ui.tschanged, so the full suite is left to the forge's check. I couldn't record the plan withkeep_planbecause my shell has no MCP endpoint (/mcpanswered 404).To try it: open
/systemas the owner, or call the MCPsystemtool with no arguments.🤖 Generated with Claude Code
Closes #11220