Public: anyone can read this; the crew can change it.

A loop stall names the request that was still running when the loop woke, not only the ones that finished #11484

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A loop stall now also names any request that was still running when the loop woke, if that request began at or before the stall did. The stall is charged to it under `<route pattern> (in flight)`, kept separate from the same route's finished requests. That row appears wherever the stall tally shows up: `/system`, MCP `system` `stalls` (same `{cause, ms, count}` shape), `/metrics`, the terminal's S…

shipped production is serving it deploy a5a2512

opened this

A loop stall now also names any request that was still running when the loop woke, if that request began at or before the stall did. The stall is charged to it under <route pattern> (in flight), kept separate from the same route's finished requests. That row appears wherever the stall tally shows up: /system, MCP system stalls (same {cause, ms, count} shape), /metrics, the terminal's Stalls screen and the event loop blocked log line. A stall with a request in flight no longer falls back to a background job or none.

What to look at

  • src/perf/in_flight.ts (new): the set of open requests. It is a Map keyed by the request, so adding and removing one is O(1) and it only ever holds requests still being answered. It is walked only when a stall happens. The route pattern is read at that point (patternOf in src/http/usage_tag.ts), because the router names a request after it starts; a request held before routing reads as (unmatched) (in flight).
  • src/http/server.ts: the public listener's fetch is now inFlight.during(request, performance.now(), () => handle(…)). The begin/end sits outside the public request handler on purpose: test/security-floor-doors.test.ts checks every return in that handler, and I left it unchanged.
  • src/perf/loop.ts: the loop watch takes the open set as a third argument. Stall gains an optional inFlight, and blamed/stallLine charge finished routes first, then in-flight ones (oldest first). whatOf in src/perf/stall_blame.ts follows from that.
  • The caveat sentence in docs/features/stalls-by-route.md is gone, replaced by a paragraph on the in-flight rows. The /metrics help text and the MCP system description now mention the label.

Not done

  • A request held open across many stalls (a slow upload, say) is charged with each one; the doc's caveats now say so.
  • The relay's forge-stall names (heldBy) still accept only bare patterns, so in-flight names are left out there.
  • I couldn't record the plan with keep_plan: that tool wasn't available in this run.

Testing test/stall-in-flight.test.ts covers:

  • a request held open across a stall is charged as in flight;
  • one that started after the stall began is not, and one that started exactly when it began is;
  • an ended request is not charged;
  • the open set is empty after a million requests begin and end;
  • during clears its entry even when the work throws;
  • a real /mcp request with its body still streaming shows up in MCP system stalls as /mcp (in flight).

The gate script and bun x tsc --noEmit pass. I ran 19 test files here, covering stalls, the loop watch, MCP size and system, usage counting, metrics and server timing, plus the security-floor doors and the server, websocket and hosted-child suites; all pass. The fast test picker named 2,090 files because server.ts changed, so the full suite is left to the forge's check.

How to try it As a forge admin, hold a slow request open across a stall, then open /system. The "Main-thread stalls, last hour" card lists <route> (in flight), and MCP system stalls has the same cause.

🤖 Generated with Claude Code

Closes #11478

commented

A loop stall now also names any request that was still running when the loop woke, if that request began at or before the stall did. The stall is charged to it under <route pattern> (in flight), kept separate from the same route's finished requests. That row appears wherever the stall tally shows up: /system, MCP system stalls (same {cause, ms, count} shape), /metrics, the terminal's Stalls screen and the event loop blocked log line. A stall with a request in flight no longer falls back to a background job or none.

What to look at

  • src/perf/in_flight.ts (new): the set of open requests. It is a Map keyed by the request, so adding and removing one is O(1) and it only ever holds requests still being answered. It is walked only when a stall happens. The route pattern is read at that point (patternOf in src/http/usage_tag.ts), because the router names a request after it starts; a request held before routing reads as (unmatched) (in flight).
  • src/http/server.ts: the public listener's fetch is now inFlight.during(request, performance.now(), () => handle(…)). The begin/end sits outside the public request handler on purpose: test/security-floor-doors.test.ts checks every return in that handler, and I left it unchanged.
  • src/perf/loop.ts: the loop watch takes the open set as a third argument. Stall gains an optional inFlight, and blamed/stallLine charge finished routes first, then in-flight ones (oldest first). whatOf in src/perf/stall_blame.ts follows from that.
  • The caveat sentence in docs/features/stalls-by-route.md is gone, replaced by a paragraph on the in-flight rows. The /metrics help text and the MCP system description now mention the label.

Not done

  • A request held open across many stalls (a slow upload, say) is charged with each one; the doc's caveats now say so.
  • The relay's forge-stall names (heldBy) still accept only bare patterns, so in-flight names are left out there.
  • I couldn't record the plan with keep_plan: that tool wasn't available in this run.

Testing test/stall-in-flight.test.ts covers:

  • a request held open across a stall is charged as in flight;
  • one that started after the stall began is not, and one that started exactly when it began is;
  • an ended request is not charged;
  • the open set is empty after a million requests begin and end;
  • during clears its entry even when the work throws;
  • a real /mcp request with its body still streaming shows up in MCP system stalls as /mcp (in flight).

The gate script and bun x tsc --noEmit pass. I ran 19 test files here, covering stalls, the loop watch, MCP size and system, usage counting, metrics and server timing, plus the security-floor doors and the server, websocket and hosted-child suites; all pass. The fast test picker named 2,090 files because server.ts changed, so the full suite is left to the forge's check.

How to try it As a forge admin, hold a slow request open across a stall, then open /system. The "Main-thread stalls, last hour" card lists <route> (in flight), and MCP system stalls has the same cause.

🤖 Generated with Claude Code

commented

When the event loop wakes from a main-thread stall and a request is still running, that stall is now counted under <route> (in flight) instead of whichever request happened to finish first. The Main-thread stalls card on /system and the stalls field of the MCP system tool both show this, and the tool's doc says so.

This push merges main. The only conflict was the system tool's doc string in src/http/mcp/tools/system.ts. The two sides had edited it for different reasons: this branch added the (in flight) wording for stalls, and main added scriptErrors { page, where, message, n, last } to what the csp kind returns. The merged string keeps both.

The gate and bun x tsc --noEmit pass. scripts/fast.ts said the diff was too broad to pick tests from, so I chose the files that cover this change. All 48 tests in these 7 files pass: stall-in-flight, mcp-system-stalls, loop-stall-by-route, loop-stall, csp-report, script-errors and mcp-doc-names-every-argument.

To try it, sign in as a forge admin and open /system while a slow request is held open across a stall. The Main-thread stalls card lists <route> (in flight), and calling the MCP system tool returns the same cause under stalls.

🤖 Generated with Claude Code

Approved

Read every changed line. No security finding: the in-flight name is a router pattern, never client text, the open set is a Map that only holds requests still being answered, and the begin/end sits outside handle so the security-floor door test is untouched. Tests cover held, started-after, boundary, ended, bounded set, throw-safe during, metrics and a real /mcp request through MCP system. Non-blocking: begin, end and size on InFlight are used only by tests (production uses during and since); consider trimming later. A request merely awaiting I/O is charged with every stall it spans; the doc caveat says so.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs