Public: anyone can read this; the crew can change it.

The gate waits on its slowest shard: split test files by how long they take, not how many bytes they are #515

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The gate now assigns test files to shards by how long each one took, not by its size in bytes. Run locally, the three shards finished within 1.07× of each other. The under-75s target can't be met: the gate has grown to 41 files on 3 CPUs, and `test/studio.test.ts` alone takes about 190–200s.

shipped production is serving it deploy d5b2fe2

opened this

The gate now assigns test files to shards by how long each one took, not by its size in bytes. Run locally, the three shards finished within 1.07× of each other. The under-75s target can't be met: the gate has grown to 41 files on 3 CPUs, and test/studio.test.ts alone takes about 190–200s.

What changed

  • Weighing (scripts/shards.ts):

- parseTimes reads scripts/test-times.json (test file → seconds). - weigher gives each file its measured time. A file missing from the table is weighed by its size in bytes, converted to seconds at the rate the table's own files show. - splitFiles still places the longest file first into the least-loaded shard, and the same file list and table always give the same split.

  • Bad table: if the table is not valid JSON, has a key that is not a test file, or has a value that is not seconds from 0 to a day, the whole table is refused. The log says so and the split falls back to bytes, so the check still runs.
  • Recording: bun run check:record (or scripts/check.ts --record) runs the same check.run string from forge.json, with --record added to scripts/test-shards.ts. Each file then runs as its own bun test, as many at a time as the gate runs shards. The table is rewritten only if every file passes, and files the run didn't cover keep their old times. The file list still lives only in forge.json, and I didn't touch it.
  • Log: at the start, the log prints the expected seconds for each shard. At the end, it adds Slowest shard Xs, fastest Ys.
  • Tests (test/shards.test.ts): they pin that:

- a known slow file sits alone when a fast file of the same size exists; - a file missing from the table still lands in a shard, with its weight checked against a hand calculation (7.5s); - eleven kinds of malformed table are refused, and the split still runs on bytes; - the checked-in table parses and names only files that exist.

  • README: says how the weighing works and when to refresh the table: after adding a slow test file, or when the slowest shard is well over 1.5× the fastest.

Measured I recorded the table on this machine with --record, then ran the full gate with bun run check:

Slowest shardFastest shardRatio
This branch, local, 3 shards445.9s417.7s1.07×
Last forge check before this (pull #479), split by bytes339.6s207.4s1.64×

On this machine's timings, the old byte split would have given 208/432/390s across three shards.

One test failed in that local gate run: "a mirror whose git is not ready yet is fetched again, and lands" in test/fork-flow.test.ts. The file passed 15/15 run alone, both in the record run and in a separate run, and its test depends on timing. I didn't change it. The forge numbers for this branch will appear in this pull's own check log on the Slowest shard … fastest … line.

What I chose not to do

  • Measured on the forge, test/studio.test.ts takes about 190s on its own, so the whole check can't get under the issue's 75s target. That target was set when the gate had 23 files. I didn't shorten that file or any other test.
  • The table was recorded here, not on the forge. Only the ratios between files affect the split, but the absolute seconds are this machine's.

To try it: bun test test/shards.test.ts, then bun run check and read the two summary lines at the end of the log.

🤖 Generated with Claude Code

Closes #307

Closes #307

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs