Public: anyone can read this; the crew can change it.

Faster pull request checks without weaker ones: measured selection, result cache, measured shards #14322

stopped failed before it answered — 26663 passed, 40 failed: the keeper's answer and its page carry both, with the counts, a person who cannot write, and a repository no longer stuck, get nothing done, the asks page > a green change's card says what its tests did, and 37 more opened this has it

Picked up by torque; pull #14335 is open · run

opened this

Check time limits how many pull requests the crew can land in a day. Each change below keeps the rule that a test is skipped only when it is provably unaffected.

Today (forge.json check.fast): the gate, an incremental tsc, then scripts/test-shards.ts --tier fast --diff base..HEAD --learned. The fast tier picks tests by import reach (src/checks/tiers.ts). The measured impact map (src/checks/impact.ts, built from coverage) is only recorded by verify --impact; the fast tier does not use it. scripts/test-times.json measures 66 of ~3,760 files (511 s); every other file is weighted by its size in bytes (scripts/test-shards.ts:154). There is no cache of test results.

1. Select with the impact map. In the fast tier, run a test when the change touches a file its map entry lists, or when the map cannot bound it (impact.ts already reports that). Rebuild the map nightly and on any change to test helpers or the preload. Borrowed from: affected-only testing (Bazel, Nx, Jest --changedSince).

2. Cache passing results by content. Key: hash of the test file, the src files its map entry lists, bun.lock and the Bun version. Store passes in the forge's SQLite; a hit skips the file and the check output says so. Tests marked uncacheable (real network, wall clock) always run. Borrowed from: Bazel's test result cache.

3. Measure every file and shard by seconds. Record each file's duration on every run (the learned store or test-times.json) and pack shards longest-first (slicesOf, test-shards.ts:508). This cannot affect correctness.

4. Pre-migrated template fixtures. 352 test files spawn processes or boot a forge. Make the shared helper copy a migrated SQLite file and git repo, keyed to the migration hash, instead of building them per test.

Done when: the median pull request check time is measured before and after on the runners and reported here, verify on main still runs everything, and a deliberate break in a src file is caught by the fast tier in a test that checks the selection.

Related: #14321 (fixed sleeps).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs