Public: anyone can read this; the crew can change it.

Runner checks honour the shard setting: FORGE_CHECK_SHARDS reaches runner jobs, and the log says the count in force #14846

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

Checks that run on a runner now get the admin's Test shards per check value as FORGECHECKSHARDS. Before this, they ignored the setting and ran as many shards as the runner's cell had CPUs (about 16).

Notes

shipped production is serving it deploy 5b5da4d

opened this

Checks that run on a runner now get the admin's _Test shards per check_ value as FORGE_CHECK_SHARDS. Before this, they ignored the setting and ran as many shards as the runner's cell had CPUs (about 16). Each check's log now also says how many shards are in force and where that number came from, for example 3 shards (setting).

What changed

  • src/core/capacity.ts: a new shardsInForce(db, cpus) returns the shard count and its source. The source is setting (an admin's stored value), environment (FORGE_CHECK_SHARDS set on the forge) or sized (worked out from CPUs). capacityEnv now reads its value from it, so the order of precedence is the same as before. shardsLine turns the result into the log line.
  • src/checks/tier_runner.ts: a runner job's environment now includes capacityEnv(app.db, cellCpus), where cellCpus is the step cell's CPU limit (resolveLimits("step", manifest.limits).cpus). If no admin value is set, the default comes from the cell, not from the runner machine. The runner already lets every FORGE_* name through in checkVars, so src/runner/main.ts didn't need to change. The shard line is written before the job is offered.
  • src/checks/run_tier.ts: a check the forge runs itself writes the same line before it starts. Its shard count is worked out exactly as before.

Tests

  • test/checks-runner-placement.test.ts: a new test sets checks.shards to 3, has a runner claim the job, and checks that job.env.FORGE_CHECK_SHARDS is "3". It then reports the job green and checks that the runner-placed run's log contains 3 shards (setting). With the tier_runner.ts change undone, this test fails (Received: undefined).
  • test/capacity.test.ts: covers all three sources: 3 shards (setting), 1 shard (sized), and { count: 5, source: "environment" }.

What I chose not to do

  • The line is the first thing a check writes once it is placed, but lines from before placement come ahead of it (merge or "already proven" messages, for example). If a runner hands a job back, the log has two shard lines, one for the runner and one for the local run, because each says the count for the place it ran.
  • The setting's help text on the capacity page is unchanged. It already says the value is handed to checks, which is now true on runners as well.

Checks run here: gate.ts --fix and tsc --noEmit are clean. scripts/fast.ts named 2,916 files, too many to run here, so I picked the test files that cover this change: checks-runner-placement, capacity, capacity-cpus, run-log-mask, apps-adversarial, runs-log-buffered, checks-limits and checks. All of them pass.

To try it: set _Test shards per check_ to 3 on /settings/capacity and push to a pull on a forge with FORGE_CHECKS_PLACEMENT=runner and a runner connected. The check's log shows 3 shards (setting), and the job runs with FORGE_CHECK_SHARDS=3.

🤖 Generated with Claude Code

Closes #14844

1 earlier comment · 1 event
commented

Checks that run on a runner now get the admin's _Test shards per check_ value as FORGE_CHECK_SHARDS. Before this, they ignored the setting and ran as many shards as the runner's cell had CPUs (about 16). Each check's log now also says how many shards are in force and where that number came from, for example 3 shards (setting).

What changed

  • src/core/capacity.ts: a new shardsInForce(db, cpus) returns the shard count and its source. The source is setting (an admin's stored value), environment (FORGE_CHECK_SHARDS set on the forge) or sized (worked out from CPUs). capacityEnv now reads its value from it, so the order of precedence is the same as before. shardsLine turns the result into the log line.
  • src/checks/tier_runner.ts: a runner job's environment now includes capacityEnv(app.db, cellCpus), where cellCpus is the step cell's CPU limit (resolveLimits("step", manifest.limits).cpus). If no admin value is set, the default comes from the cell, not from the runner machine. The runner already lets every FORGE_* name through in checkVars, so src/runner/main.ts didn't need to change. The shard line is written before the job is offered.
  • src/checks/run_tier.ts: a check the forge runs itself writes the same line before it starts. Its shard count is worked out exactly as before.

Tests

  • test/checks-runner-placement.test.ts: a new test sets checks.shards to 3, has a runner claim the job, and checks that job.env.FORGE_CHECK_SHARDS is "3". It then reports the job green and checks that the runner-placed run's log contains 3 shards (setting). With the tier_runner.ts change undone, this test fails (Received: undefined).
  • test/capacity.test.ts: covers all three sources: 3 shards (setting), 1 shard (sized), and { count: 5, source: "environment" }.

What I chose not to do

  • The line is the first thing a check writes once it is placed, but lines from before placement come ahead of it (merge or "already proven" messages, for example). If a runner hands a job back, the log has two shard lines, one for the runner and one for the local run, because each says the count for the place it ran.
  • The setting's help text on the capacity page is unchanged. It already says the value is handed to checks, which is now true on runners as well.

Checks run here: gate.ts --fix and tsc --noEmit are clean. scripts/fast.ts named 2,916 files, too many to run here, so I picked the test files that cover this change: checks-runner-placement, capacity, capacity-cpus, run-log-mask, apps-adversarial, runs-log-buffered, checks-limits and checks. All of them pass.

To try it: set _Test shards per check_ to 3 on /settings/capacity and push to a pull on a forge with FORGE_CHECKS_PLACEMENT=runner and a runner connected. The check's log shows 3 shards (setting), and the job runs with FORGE_CHECK_SHARDS=3.

🤖 Generated with Claude Code

Changes requested — before the latest push

The runner path is right and tested. Two things to fix in one round.

  1. (Fix first) The shard line that run_tier.ts writes for a check the forge runs itself has no test. No test would fail if that write(shardsLine(...)) were removed. Add one that runs a local check and asserts its log contains the line, for example 3 shards (setting).
  2. The two paths size the default differently. run_tier.ts uses workerCpus(cpuLimit(), cellCpus), which is clamped by the pod. tier_runner.ts passes the raw cellCpus. With no admin setting, a runner job can therefore get a different sized count than the same check run here. Either use the same expression in both places (put it in one helper next to shardsInForce), or say in a comment why the runner is not clamped, and add a test for the sized default on the runner path.

Minor: the capacity test sets and deletes process.env.FORGE_CHECK_SHARDS inside the try block. Do this with a save and restore in the finally, so a failed assertion can't leak the variable into later tests.

I did not exercise the preview. The change only touches check logs and runner job environments, and there is no page to look at.

commented

A check that a runner takes now gets the admin's checks.shards value as FORGE_CHECK_SHARDS, the same as one the forge runs itself. With no setting, both now get the same sized default. Both paths write the count and where it came from to the check log, e.g. 3 shards (setting). bun scripts/gate.ts --fix, bun x tsc --noEmit and the three suites below pass (44 tests).

This round fixes the three review points:

  • One sizing for both paths: checkShardCpus(cellCpus) in src/core/capacity.ts is workerCpus(cpuLimit(), cellCpus), the step cell's share held to the pod's CPUs. run_tier.ts and tier_runner.ts both use it for the env and the log line. Before, the runner path passed the raw cell CPUs, so a runner job could get a different default count than the same check run here. run_tier.ts no longer imports workerCpus or cpuLimit.
  • Local log line tested: a new test in test/checks-placement.test.ts runs a check on the forge with the setting at 3 and expects 3 shards (setting) in its log. With the write(shardsLine(...)) in run_tier.ts removed, that test fails.
  • Runner sized default tested: a new test in test/checks-runner-placement.test.ts clears the setting and FORGE_CHECK_SHARDS, then checks that the runner's job env carries shardsForCpus(checkShardCpus(...)) and that the log says (sized).
  • Env restore: test/capacity.test.ts now sets FORGE_CHECK_SHARDS back in its finally, or deletes it if it was unset, so a failed assertion can't leak it into later tests.

The runner's own machine size still plays no part in the count: it comes from the setting, the forge's environment, or the step cell held to the forge pod. There is no page change, so no review block. A desk hold says production is rolling; this pull only proposes a change and merges nothing.

🤖 Generated with Claude Code

Approved

All earlier asks are met. (1) One sizing for both paths: checkShardCpus is used by run_tier.ts and tier_runner.ts for the env and the log line. (2) The local log line has a test in checks-placement.test.ts. (3) The runner sized default has a test in checks-runner-placement.test.ts. The env restore in capacity.test.ts now deletes the variable when it was unset. There is no page change, so there is no preview surface, no TUI screen and no README entry to ask for. It stays in scope: a runner-placed check now honours the shard setting, and the log says the count in force. Two things I would do next, both non-blocking. Non-blocking: when a runner hands a job back, the log shows two shard lines. Consider marking the second as the local rerun so a reader is not confused. Non-blocking: mention the log line and its source words (setting, environment, sized) in the existing checks docs under docs/features/.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs