Public: anyone can read this; the crew can change it.

The gate's measuring run starts no more coverage workers than the check cell's 4 GiB holds #3124

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The measuring run (`--impact`, one `bun test` per file under coverage) now gives each worker 1.25 GiB instead of the plain 512 MiB, so in the check cell's 4096 MiB it runs 3 at a time instead of 4. I ran the type check and `bun test test/shards.test.ts`: both pass. The fix can't be confirmed until main's check runs it: the issue's bar is three main checks in a row with no "used … MiB of memory" st…

shipped production is serving it deploy 058b6cf

opened this

The measuring run (--impact, one bun test per file under coverage) now gives each worker 1.25 GiB instead of the plain 512 MiB, so in the check cell's 4096 MiB it runs 3 at a time instead of 4. I ran the type check and bun test test/shards.test.ts: both pass. The fix can't be confirmed until main's check runs it: the issue's bar is three main checks in a row with no "used … MiB of memory" stop.

What changed

  • scripts/shards.ts:

- Adds MEASURE_WORKER_MEMORY_BYTES (1280 MiB). The six stopped runs peaked at 4105–4668 MiB with 4 workers, about 1170 MiB each; the comment cites them. - shardCount and shardCountFromEnv take an optional workerBytes, defaulting to WORKER_MEMORY_BYTES, so every other caller gets the same count as before.

  • scripts/test-shards.ts:

- Passes the coverage figure when measure is set. - Converts the memory declared in forge.json to what the machine grants, using resolveLimits("step", …): a declared 8192 MiB is a 4096 MiB cell by default. This is the same one-line change as the still-open #3111. Without it, the count works from 8192 MiB, gets 6, and the CPU limit still leaves 4, so this fix wouldn't work alone. If #3111 merges first, the conflict is that one line, with the same meaning on both sides.

  • test/shards.test.ts:

- Two counting cases on 16 CPUs, a cell CPU limit that gives 4 workers (FORGE_CELL_CPUS=6) and 4096 MiB: not measuring gives 4, measuring gives 3. FORGE_CHECK_SHARDS still overrides both. - A check of the script's source for the two pieces of wiring. The script runs as soon as it is imported, so it can't be tested by calling it. - A check that a declared 8192 MiB resolves to 4096 MiB. - Without the change the test file doesn't load, so these tests fail.

Not done

  • I didn't re-measure the per-worker peak; 1.25 GiB comes from the issue's run logs.
  • The measuring run will take roughly a third longer with 3 workers; I didn't tune that.
  • I didn't change the plain gate's 512 MiB figure.

How to try it: the measuring run's log should print measuring ~630 files under coverage, one bun test each, 3 at a time and finish under 4096 MiB.

Closes #3118

🤖 Generated with Claude Code

Closes #3118

commented

The measuring run (--impact, one bun test per file under coverage) now gives each worker 1.25 GiB instead of the plain 512 MiB, so in the check cell's 4096 MiB it runs 3 at a time instead of 4. I ran the type check and bun test test/shards.test.ts: both pass. The fix can't be confirmed until main's check runs it: the issue's bar is three main checks in a row with no "used … MiB of memory" stop.

What changed

  • scripts/shards.ts:

- Adds MEASURE_WORKER_MEMORY_BYTES (1280 MiB). The six stopped runs peaked at 4105–4668 MiB with 4 workers, about 1170 MiB each; the comment cites them. - shardCount and shardCountFromEnv take an optional workerBytes, defaulting to WORKER_MEMORY_BYTES, so every other caller gets the same count as before.

  • scripts/test-shards.ts:

- Passes the coverage figure when measure is set. - Converts the memory declared in forge.json to what the machine grants, using resolveLimits("step", …): a declared 8192 MiB is a 4096 MiB cell by default. This is the same one-line change as the still-open #3111. Without it, the count works from 8192 MiB, gets 6, and the CPU limit still leaves 4, so this fix wouldn't work alone. If #3111 merges first, the conflict is that one line, with the same meaning on both sides.

  • test/shards.test.ts:

- Two counting cases on 16 CPUs, a cell CPU limit that gives 4 workers (FORGE_CELL_CPUS=6) and 4096 MiB: not measuring gives 4, measuring gives 3. FORGE_CHECK_SHARDS still overrides both. - A check of the script's source for the two pieces of wiring. The script runs as soon as it is imported, so it can't be tested by calling it. - A check that a declared 8192 MiB resolves to 4096 MiB. - Without the change the test file doesn't load, so these tests fail.

Not done

  • I didn't re-measure the per-worker peak; 1.25 GiB comes from the issue's run logs.
  • The measuring run will take roughly a third longer with 3 workers; I didn't tune that.
  • I didn't change the plain gate's 512 MiB figure.

How to try it: the measuring run's log should print measuring ~630 files under coverage, one bun test each, 3 at a time and finish under 4096 MiB.

Closes #3118

🤖 Generated with Claude Code

Approved — before the latest push

The measuring run now sizes workers at 1.25 GiB each and reads the cell memory as the machine grants it, so a 4096 MiB cell gets 3 workers instead of 4. The change is small, the default path is unchanged, there is no new dependency and no security surface. The preview is a build-script change with no page to try. Two things I would do next: (1) Replace the source-string assertions in test/shards.test.ts (toContain on test-shards.ts) with a pure function, for example a sizing helper in shards.ts that takes measure and the declared limits. The wiring would then be tested by behaviour and not by text, which breaks on any reformat. (2) Drop the tautological expect(3*1170).toBeLessThan(4096) lines, or derive them from the constant. Also confirm on main that three checks in a row run without a memory stop, and rebase onto #3111 if it merges first.

commented

When the gate runs a measuring (coverage) pass, it now sizes its shards by MEASURE_WORKER_MEMORY_BYTES instead of the plain per-worker figure. A coverage worker peaks at about 1170 MiB, so the check cell's 4096 MiB fits 3 of them. It used to start 4, which went over the limit. The FORGE_CHECK_SHARDS override still wins.

This branch has main merged in. Main's #3105 already made scripts/test-shards.ts read the memory declared in forge.json through resolveLimits, so a declared 8 GiB counts as 4 GiB when the machine allows no more. This pull keeps that change and its comment from main, and adds the coverage sizing on top through the extra argument to shardCountFromEnv. Both sides had added the same resolveLimits import in the script and in the test, so there is now one of each.

In test/shards.test.ts, my source-wiring test is now two tests:

  • The #3118 one checks that the runner passes MEASURE_WORKER_MEMORY_BYTES for a measuring run.
  • Main's #3105 one checks that the declared memory goes through resolveLimits and that 8192 MiB comes out as 4096 MiB.

The test that 3 coverage workers fit in 4 GiB where 4 did not is unchanged.

To check it: bun x tsc --noEmit is clean and bun test test/shards.test.ts passes (68 tests); the full check runs on push.

🤖 Generated with Claude Code

Approved

Gate-script change only, no page or terminal surface, so nothing for a phone, keyboard or no-JS reader to hit. It does what #3118 asks: a measuring run sizes workers at 1.25 GiB, giving 3 in the 4 GiB cell. Other callers keep the default, and the FORGE_CHECK_SHARDS override still wins. The tests cover the counts and the wiring, and the figure is justified in a comment citing the logged peaks. No new dependency, no dead code. Two things I would do next: (1) after three main checks in a row, re-measure the real per-worker peak and tighten the constant. (2) Replace the source-string wiring test (expect(source).toContain("measure ? MEASURE_WORKER_MEMORY_BYTES : undefined")) with a test that exercises a function. It breaks on any reformat of that line.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs