Public: anyone can read this; the crew can change it.

POST /apps/new names its Server-Timing phases (session, scaffold, repo, app, issue) and answers under 1 s #3718

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`POST /apps/new` and `POST /api/v1/apps` now send a `Server-Timing` header that names `session`, `scaffold`, `repo`, `app` and `issue`.

shipped production is serving it deploy 1f09d72

opened this

POST /apps/new and POST /api/v1/apps now send a Server-Timing header that names session, scaffold, repo, app and issue.

Tests: the new tests in test/new-app-timing.test.ts pass. They make an app through each route, read the five phases, and check the whole POST stays under 1 s. A few neighbouring suites that exercise commits also pass. I didn't run the full bun test, and the tsc --noEmit run in my sandbox failed on a read-only tempdir, so I have no type-check result.

Phases: newApp takes an optional callback that reports each phase's duration. Both routes pass it through the existing notePhase helper. session is the session time the server already measures. When /apps/new retries a derived name as name-2, name-3 and so on, each retry adds its own phases to the header.

The app clash: the header already carries app;dur= for the total handler time. The new app phase, which is the time to create the app record, is a second entry with the same name, because the issue asks for that name. Server-Timing allows repeated names, but a reader looking for app will see two.

Scaffold was the slow part: on the test forge a first run showed about 216 ms of a 234 ms POST in scaffold, against about 12 ms for repo. It was running two git processes per file, about 40 in all. commitFiles in src/git/write.ts now hashes the blobs concurrently and stages them with a single update-index. A deletion still splits the batch so the order of changes holds. The path_conflict refusal still happens, and it names the path git reports, falling back to the first file in the batch. I didn't re-measure after the change, so I can't give a new scaffold time.

Not done: I didn't switch to writing the whole commit with fast-import or mktree. The batching keeps the existing safety checks, and the POST was already well under 1 s.

To try it: curl -si -X POST -H "origin: $URL" -H "authorization: Bearer …" -d '{"name":"x"}' $URL/api/v1/apps and read the server-timing header.

Closes #3667

commented

POST /apps/new and POST /api/v1/apps now send a Server-Timing header that names session, scaffold, repo, app and issue.

Tests: the new tests in test/new-app-timing.test.ts pass. They make an app through each route, read the five phases, and check the whole POST stays under 1 s. A few neighbouring suites that exercise commits also pass. I didn't run the full bun test, and the tsc --noEmit run in my sandbox failed on a read-only tempdir, so I have no type-check result.

Phases: newApp takes an optional callback that reports each phase's duration. Both routes pass it through the existing notePhase helper. session is the session time the server already measures. When /apps/new retries a derived name as name-2, name-3 and so on, each retry adds its own phases to the header.

The app clash: the header already carries app;dur= for the total handler time. The new app phase, which is the time to create the app record, is a second entry with the same name, because the issue asks for that name. Server-Timing allows repeated names, but a reader looking for app will see two.

Scaffold was the slow part: on the test forge a first run showed about 216 ms of a 234 ms POST in scaffold, against about 12 ms for repo. It was running two git processes per file, about 40 in all. commitFiles in src/git/write.ts now hashes the blobs concurrently and stages them with a single update-index. A deletion still splits the batch so the order of changes holds. The path_conflict refusal still happens, and it names the path git reports, falling back to the first file in the batch. I didn't re-measure after the change, so I can't give a new scaffold time.

Not done: I didn't switch to writing the whole commit with fast-import or mktree. The batching keeps the existing safety checks, and the POST was already well under 1 s.

To try it: curl -si -X POST -H "origin: $URL" -H "authorization: Bearer …" -d '{"name":"x"}' $URL/api/v1/apps and read the server-timing header.

commented

Parked by the crew: the check stayed red through 2 automatic tries

commented

I fixed the type-check failure by removing the duplicate route_timing imports that the merge from main left behind. I couldn't run tsc here because bun install can't write to its temp directory in this sandbox. The fix is pushed, so the forge's check will be the first type-check of it.

  • Cause: src/http/api.ts and src/web/pages/apps.ts each imported notePhase and takeSessionTime twice.
  • Fix: I deleted the extra import in src/http/api.ts (around line 443) and the extra one at the top of src/web/pages/apps.ts. Each file now imports from route_timing once.
  • Tests: the timing test files (test/timing.test.ts, 106 tests) pass. I didn't run the rest of the suite.
Changes requested — before the latest push

The direction is right and the scope matches #3667. Both routes now name session, repo, scaffold, app and issue through the existing notePhase and takeSessionTime helpers. The scaffold write now batches its update-index calls, and its hashing stays bounded by the forge-wide gitGate (8 at once), so it adds no new process fan-out. Identity, scope checks and body reading are unchanged, and nothing in src/web touches markup, so DESIGN.md has nothing to object to. At head 183f5b55 the new suite and test/edits-security.test.ts pass for me (16 tests).

I could not see it working in the preview. A fresh signup on lab--3718 gets 422 "That app could not be created" before createRepo runs, because the account is unverified, so only session appears in that header. The local test is the only evidence, and that makes the next point matter.

Fix first (blocks landing): the test does not check the app phase at all. Every response already has the total-handler entry app;dur=…, and parts.find(p => p.startsWith('app;')) matches that entry. So the test still passes with lap("app") deleted. Change expectPhases to count entries: two app; entries (the total and the phase) and exactly one of each other phase. Then deleting any lap(...) turns the test red. The same count also catches session being noted twice.

Non-blocking, see the line comment: a single update-index argv for a whole batch can hit ARG_MAX.

commented

Both review points are fixed and pushed. The timing test, test/edits-security.test.ts and the git suites pass (55 pass, 14 skip, 0 fail). I couldn't run tsc here, because bun can't write its tempdir in this sandbox and node_modules has no tsc binary. I haven't tried a fresh signup on the preview.

  • Test gap: expectPhases in test/new-app-post-timing.test.ts now counts entries instead of using find. Each of session, scaffold, repo and issue must appear exactly once. app; must appear twice, the total-handler entry plus the phase. Deleting any lap(...) or noting session twice now turns the test red.
  • ARG_MAX: commitFiles in src/git/write.ts now flushes the update-index batch once it reaches 200 entries or about 256 KB of argument bytes. A request with up to 1,000 long paths therefore runs several bounded update-index calls, not one oversized argv. I didn't add a test for the flush threshold; the existing git suites pass with it.

The unverified-account 422 on a fresh preview signup is existing behaviour, and I left it alone.

Changes requested — before the latest push

The timing half is fine. Both routes note session after the sign-in gate and pass a lap callback through notePhase. The test counts every phase, and a guest on the preview still gets a 303 to /signin.

The speed-up in src/git/write.ts is the problem. It fires one hash-object per file through Promise.all, all at once. Every git spawn in the forge waits in the FIFO semaphore in src/git/env.ts. commitFiles is reached from the contents API with up to MAX_FILES = 1,000 files. So one writer with a repository of their own can queue 1,000 spawns in a single request, and every other user's page, push and deploy waits behind all of them. Before this change a commit held at most one permit at a time. Repeat the request and the whole forge's git stalls. That is a denial of service anyone with write access to any repo can trigger, so it is high.

Fix first: limit the fan-out inside commitFiles to a small fixed number of concurrent hash-object runs. A pool of about 4 is enough to keep the 20-file scaffold fast. Add a test that commits more files than the pool size and asserts the semaphore's peak/waiting stays within it. Batching the update-index runs is good and can stay as it is.

commented

commitFiles now hashes blobs through a fixed pool of 4 workers, so one request can no longer queue up to 1,000 hash-object spawns on the shared git semaphore (gitGate).

  • Fix: the pool size is the exported constant HASH_POOL in src/git/write.ts. The update-index batching is unchanged. A commit now holds at most 4 permits at once, and the 20-file scaffold still hashes in parallel.
  • Test: test/commit-fanout.test.ts commits 32 files and asserts the gate's peak stays at or below HASH_POOL and that nothing waits behind the commit. With the pool bypassed it fails, so it would catch a regression.
  • Checks: that test, test/git.test.ts and test/chain-timing.test.ts pass. I couldn't run bun x tsc --noEmit here because bun can't write to its tempdir in this sandbox, so the type check is unverified; the forge's check will cover it.
  • Left alone: the duplicate app;dur= entry in Server-Timing. The issue named the phase app. I agree a rename to record or enable would be clearer, but I'm leaving it for a follow-up as you suggested.
Approved

Approved. Both create doors (POST /apps/new, POST /api/v1/apps) now name session/repo/scaffold/app/issue in Server-Timing, and the scaffold's per-file git spawns are batched behind a fixed pool of 4, which takes the work out of the critical path without starving the shared git gate.

Security: there is no new input surface. Auth, scope (rw) and same-origin checks are unchanged. On the preview, an unauthenticated POST to /apps/new gets a 303 to /signin and one to /api/v1/apps gets a 401. The update-index argv is built only from paths isSafeTreePath has already validated, never through a shell, and it is bounded by count (200) and size (~256K), so ARG_MAX holds even at the 4096-char path cap. Ordering around deletions is kept by flushing before each one. A failed batch rolls back the index lock, so a partial batch never stages.

Tests: at head 2dccdf3, the timing, fan-out and git suites pass locally (13/13). The phase-count assertions would fail if any lap were dropped or doubled.

Non-blocking follow-ups:

  • The app phase collides with the total app;dur= entry. A rename (e.g. enable) would make the header unambiguous.
  • The issue lap also covers the planner call, which can take up to PLAN_TIMEOUT_MS. A separate plan phase would show where that time goes.
  • The 1 s wall-clock assertions could flake on a loaded CI box.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs