Public: anyone can read this; the crew can change it.

The two slow tests in test/snapshot-hosted-inside.test.ts run under 1 s alone and under 2 s in the full suite #15196

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The test "a preview already seeded gets its settings rows once, and a second seed adds no token" now runs in about 0.2 s, down from 1.19 s.

Notes

shipped production is serving it deploy cfe12e4

opened this

The test "a preview already seeded gets its settings rows once, and a second seed adds no token" now runs in about 0.2 s, down from 1.19 s. The issue names it as part of test/snapshot-hosted-inside.test.ts, but it is in test/preview-looks.test.ts, under "the owner's settings pages (#9781)", which is where this changes it. It seeded a whole preview before testing anything, and that seed was most of its time.

That block now seeds one preview in a beforeAll. Two tests use it:

  • The settings-rows test deletes the token and the preview_settings_seeded mark, as it did before. Its own re-seed puts both back, so the shared preview ends as it began whatever order the tests run in.
  • "the seed gives each a row and answers 200 to the owner" only reads, so it shares the preview too.

The refusal test still seeds its own unseeded forge, because it tests a refusal during a seed. Every assertion still runs, and nothing was added to SLOW in scripts/gate.ts.

The other named test ("a repository git would run something of … left out by name in the manifest") was already brought under budget by #15152, so test/snapshot-hosted-inside.test.ts is unchanged.

Per-test times, bun test <file> --reporter=junit, on a 72-core machine at load average 160–190:

testbeforeafter, run alone (3 runs)after, both files together
a preview already seeded gets its settings rows once…1.19 s0.14 / 0.15 / 0.19 s0.26 s
the seed gives each a row and answers 200 to the owner1.05 s0.03 / 0.04 / 0.04 s0.04 s
a repository git would run something of… (unchanged)0.48 s0.91 / 0.95 / 0.58 s1.10 s

The forge's full check runs after this push, so its log isn't available yet and its times aren't quoted here; the issue's checkbox for them stays open until they're read from it.

To look at: under the same load the refusal test, which this change leaves alone, took 1.7–3.8 s. It was 0.8 s at lower load, and it may go over the 2 s budget in the full suite. Sharing the preview with it would mean testing a refusal on an already-seeded preview rather than on a first seed, so I didn't. If the check log shows it over 2 s, it needs its own issue.

bun scripts/gate.ts --fix changed nothing, bun x tsc --noEmit is clean, and the fast tier's 119 files (from scripts/fast.ts) pass: 1716 pass, 2 skipped, 0 fail. The keep_plan and estimate tools the issue asks for aren't available in this session, so no plan or estimate was recorded.

🤖 Generated with Claude Code

Closes #15191

1 earlier comment · 1 event
commented

The test "a preview already seeded gets its settings rows once, and a second seed adds no token" now runs in about 0.2 s, down from 1.19 s. The issue names it as part of test/snapshot-hosted-inside.test.ts, but it is in test/preview-looks.test.ts, under "the owner's settings pages (#9781)", which is where this changes it. It seeded a whole preview before testing anything, and that seed was most of its time.

That block now seeds one preview in a beforeAll. Two tests use it:

  • The settings-rows test deletes the token and the preview_settings_seeded mark, as it did before. Its own re-seed puts both back, so the shared preview ends as it began whatever order the tests run in.
  • "the seed gives each a row and answers 200 to the owner" only reads, so it shares the preview too.

The refusal test still seeds its own unseeded forge, because it tests a refusal during a seed. Every assertion still runs, and nothing was added to SLOW in scripts/gate.ts.

The other named test ("a repository git would run something of … left out by name in the manifest") was already brought under budget by #15152, so test/snapshot-hosted-inside.test.ts is unchanged.

Per-test times, bun test <file> --reporter=junit, on a 72-core machine at load average 160–190:

testbeforeafter, run alone (3 runs)after, both files together
a preview already seeded gets its settings rows once…1.19 s0.14 / 0.15 / 0.19 s0.26 s
the seed gives each a row and answers 200 to the owner1.05 s0.03 / 0.04 / 0.04 s0.04 s
a repository git would run something of… (unchanged)0.48 s0.91 / 0.95 / 0.58 s1.10 s

The forge's full check runs after this push, so its log isn't available yet and its times aren't quoted here; the issue's checkbox for them stays open until they're read from it.

To look at: under the same load the refusal test, which this change leaves alone, took 1.7–3.8 s. It was 0.8 s at lower load, and it may go over the 2 s budget in the full suite. Sharing the preview with it would mean testing a refusal on an already-seeded preview rather than on a first seed, so I didn't. If the check log shows it over 2 s, it needs its own issue.

bun scripts/gate.ts --fix changed nothing, bun x tsc --noEmit is clean, and the fast tier's 119 files (from scripts/fast.ts) pass: 1716 pass, 2 skipped, 0 fail. The keep_plan and estimate tools the issue asks for aren't available in this session, so no plan or estimate was recorded.

🤖 Generated with Claude Code

Approved

Test-only change, no page or door touched. One seeded preview shared by the two reading tests; the mutating test restores what it removes, so order does not matter; the refusal test keeps its own unseeded forge; every assertion still runs; nothing added to SLOW. Next two things: (1) read the full check log and edit the per-test times into the description (box 3), including the refusal test, which you measured at 1.7-3.8 s under load and which may exceed 2 s: if it does, open its own issue. (2) the snapshot-hosted-inside test is at 0.9-1.1 s alone, close to the 1 s line; confirm it in the check log.

commented

Left #15191 open: 2 unticked items remain ('a repository git would run something of … left out by name in the manifest' runs under 1 s alone and under 2 s in the full suite; the pull quotes the per-test times from bun test test/snapshot-hosted-inside.test.ts and from the full check log).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs