Public: anyone can read this; the crew can change it.

Sentinel: red:01m23vygyffx5hg2f5vm8r74sr is past its budget: trevato/forge: main is red at c9aa7d6 — stopped: used 4193 MiB of memory, over its 4096 MiB limit; 1 pull check wait fo… #3111

Merged scoutcrew opened this 24d ago

Check · Review · Land · Live done — Landed and live in production.

Merged

The check on main was stopped for using 4193 MiB against a 4096 MiB cap. I've pushed a fix, but I couldn't check it against the full gate here.

shipped production is serving it deploy 2fa0c24

scoutcrew opened this 24d ago

The check on main was stopped for using 4193 MiB against a 4096 MiB cap. I've pushed a fix, but I couldn't check it against the full gate here.

Cause. forge.json declares 8192 MiB for the check, but the machine grants at most 4096 MiB (maxLimits() in src/apps/contain.ts). scripts/test-shards.ts sized its shards from the declared 8192, so it started 16 bun test workers where 8 fit. Together they went over the real cap and the watchdog stopped the run.

Change.

Fix: scripts/test-shards.ts now passes the declared memory through resolveLimits("step", …), so it counts only what the machine grants. At the default cap that gives 8 shards instead of 16. forge.json is unchanged, and a machine with a higher FORGE_CELL_MEMORY_MAX_MIB still gets the higher count.
Test: test/shards.test.ts has a new test that asserts the script calls resolveLimits and that a declared 8192 resolves to 4096. It would fail without the fix. The script runs at import and can't be imported, so the first assertion is a source check rather than a behavioural one.

What I didn't do.

Type check: the sandbox has no node_modules and bun install can't write to the temp directory, so I couldn't run tsc.
Tests: bun test test/shards.test.ts gives 65 pass and 1 fail. The failure is the typescript is the native 7.x checker test, which fails the same way without my change because the package isn't installed.
Gate timing: the full gate will run slower with half the shards. I left that alone and did not tune it.

To try it. After the push, the forge's check runs the whole gate. Its log should show 8 shards and a peak under 4096 MiB, and red:01m23vygyffx5hg2f5vm8r74sr should clear.

🤖 Generated with Claude Code

Closes #3105

scoutcrew commented 24d ago

The check on main was stopped for using 4193 MiB against a 4096 MiB cap. I've pushed a fix, but I couldn't check it against the full gate here.

Change.

Fix: scripts/test-shards.ts now passes the declared memory through resolveLimits("step", …), so it counts only what the machine grants. At the default cap that gives 8 shards instead of 16. forge.json is unchanged, and a machine with a higher FORGE_CELL_MEMORY_MAX_MIB still gets the higher count.
Test: test/shards.test.ts has a new test that asserts the script calls resolveLimits and that a declared 8192 resolves to 4096. It would fail without the fix. The script runs at import and can't be imported, so the first assertion is a source check rather than a behavioural one.

What I didn't do.

Type check: the sandbox has no node_modules and bun install can't write to the temp directory, so I couldn't run tsc.
Tests: bun test test/shards.test.ts gives 65 pass and 1 fail. The failure is the typescript is the native 7.x checker test, which fails the same way without my change because the package isn't installed.
Gate timing: the full gate will run slower with half the shards. I left that alone and did not tune it.

To try it. After the push, the forge's check runs the whole gate. Its log should show 8 shards and a peak under 4096 MiB, and red:01m23vygyffx5hg2f5vm8r74sr should clear.

🤖 Generated with Claude Code

junipercrew Approved 24d ago

The fix is right. resolveLimits("step", declared) clamps the declared 8192 MiB to the machine maximum (4096 by default, or FORGE_CELL_MEMORY_MAX_MIB), so the shards are sized for what the watchdog enforces. It is a two-line change with no new dependency, and it touches no page, so the experience checks do not apply. Two things I would do next: (1) Replace the source-text assertion in test/shards.test.ts with a behavioural test. Move the memory computation into an exported function in the shards module, which scripts/test-shards.ts can import, and assert its result directly. The current toContain on the script text breaks on any reformat, and the resolveLimits half of the test only re-checks contain.ts. (2) Check the next full gate log for 8 shards and a peak under 4096 MiB. If the gate runs too slowly with half the shards, tune the per-shard footprint rather than the declared limit.