Public: anyone can read this; the crew can change it.

A pull's test selection says why it chose each test, and a changed doc no longer blinds it to the impact map #14940

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

A pull's check log now says why it chose each test. A pull that changes a doc like docs/THREATS.md no longer turns its selection blind. What changed The log says why (src/checks/tiers.ts).

Notes

shipped production is serving it deploy 6b7374e

opened this

A pull's check log now says why it chose each test. A pull that changes a doc like docs/THREATS.md no longer turns its selection blind.

What changed

  • The log says why (src/checks/tiers.ts). Under the plan line, test-shards.ts and fast.ts print a count per reason, for example by reason: 40 quick, 12 executes, 11 repo-wide rule (reasonLine). When the plan is blind, they also name the files that made it blind and why, for example blind (every quick test, guesses for every test) by static/app.js (outside src/ and test/), src/x.ts (no test executes it) (blindLine). FastPlan now carries blindBy, which holds those files.
  • Where the width came from. Blind mode adds every test under 1 s, and that rule is what reaches about 2,000 files. The import guess was already limited to the files the map can't see.
  • A doc is not blind (src/checks/scans.ts). The new docReaders finds who can read a changed .md under docs/ or at the root, the same way scriptReaders does for scripts. That is the tests that name it, directly or through a helper, plus the src/ files that name it or list its folder (src/view/docs.ts, src/checks/resilience.ts). The impact map then picks the tests that run those src/ files, with the reason reads docs/THREATS.md through src/view/docs.ts.
  • What still blinds. A doc stays blind only if the suite's preload reads it, or a manifest, chart or the Dockerfile names it. On this tree that is AGENTS.md (named by templates/app-template.json), docs/DESIGN.md (copied by the Dockerfile) and README.md. The log names each of them.
  • Shared code. scriptReaders and docReaders now share one closure (besideReaders). Each round only checks names added since the last round, which took README.md from 25 s to 6 s on the real tree. isDoc lives in impact.ts so tiers.ts and scans.ts don't import each other.
  • Files the suite's loading runs. They are still guessed by each test's imports, as #12178 requires, but they no longer add every quick test.

What I didn't do

  • Step 3 is now #14941. That is the Bun.Transpiler.scanImports parser and the logged bun test --changed comparison, which needs two weeks of real pulls. #14845 now waits for it.
  • The 10% target is only shown on a test fixture. The new test in test/check-tiers.test.ts gives exactly the same selection plus the doc's readers, and no quick tests. I couldn't measure the real pull (one src/crew/ file plus THREATS.md) because the impact map isn't in the sandbox.
  • The before/after median count isn't posted. It needs check logs from after this lands. The by reason line makes it readable from any log.
  • Scripts still blind. A changed script whose readers are known still turns the plan blind, and the log now says so. Treating it like a doc is the obvious next step; I left it out of scope.

Tests

  • Every existing test passes unchanged, including the #12178 loading test.
  • New fixtures in test/check-script-readers.test.ts and test/check-tiers.test.ts. A whole-tree docReaders test took 3 s, over the per-test budget, so I dropped it.
  • bun x tsc --noEmit passes, and scripts/gate.ts --fix reported its comment rewraps, which are committed.
  • scripts/fast.ts exits 2 for this diff because it touches scripts, so I picked the files myself: 17 files, 293 pass, 0 fail. They are the tier and reader suites, scripts-fast, tree-scans, allow-lists, unused-exports, changed-compare, kept, the shards suites, docs-link and the rule tests.
  • keep_plan and estimate were not available to me, so there is no plan on the issue and no recorded estimate.

To try it: stage any docs/features/*.md change and run bun scripts/fast.ts --why. Stderr ends with the by reason: line, and the reads docs/… reasons appear on stdout.

🤖 Generated with Claude Code

Closes #14845

commented

A pull's check log now says why it chose each test. A pull that changes a doc like docs/THREATS.md no longer turns its selection blind.

What changed

  • The log says why (src/checks/tiers.ts). Under the plan line, test-shards.ts and fast.ts print a count per reason, for example by reason: 40 quick, 12 executes, 11 repo-wide rule (reasonLine). When the plan is blind, they also name the files that made it blind and why, for example blind (every quick test, guesses for every test) by static/app.js (outside src/ and test/), src/x.ts (no test executes it) (blindLine). FastPlan now carries blindBy, which holds those files.
  • Where the width came from. Blind mode adds every test under 1 s, and that rule is what reaches about 2,000 files. The import guess was already limited to the files the map can't see.
  • A doc is not blind (src/checks/scans.ts). The new docReaders finds who can read a changed .md under docs/ or at the root, the same way scriptReaders does for scripts. That is the tests that name it, directly or through a helper, plus the src/ files that name it or list its folder (src/view/docs.ts, src/checks/resilience.ts). The impact map then picks the tests that run those src/ files, with the reason reads docs/THREATS.md through src/view/docs.ts.
  • What still blinds. A doc stays blind only if the suite's preload reads it, or a manifest, chart or the Dockerfile names it. On this tree that is AGENTS.md (named by templates/app-template.json), docs/DESIGN.md (copied by the Dockerfile) and README.md. The log names each of them.
  • Shared code. scriptReaders and docReaders now share one closure (besideReaders). Each round only checks names added since the last round, which took README.md from 25 s to 6 s on the real tree. isDoc lives in impact.ts so tiers.ts and scans.ts don't import each other.
  • Files the suite's loading runs. They are still guessed by each test's imports, as #12178 requires, but they no longer add every quick test.

What I didn't do

  • Step 3 is now #14941. That is the Bun.Transpiler.scanImports parser and the logged bun test --changed comparison, which needs two weeks of real pulls. #14845 now waits for it.
  • The 10% target is only shown on a test fixture. The new test in test/check-tiers.test.ts gives exactly the same selection plus the doc's readers, and no quick tests. I couldn't measure the real pull (one src/crew/ file plus THREATS.md) because the impact map isn't in the sandbox.
  • The before/after median count isn't posted. It needs check logs from after this lands. The by reason line makes it readable from any log.
  • Scripts still blind. A changed script whose readers are known still turns the plan blind, and the log now says so. Treating it like a doc is the obvious next step; I left it out of scope.

Tests

  • Every existing test passes unchanged, including the #12178 loading test.
  • New fixtures in test/check-script-readers.test.ts and test/check-tiers.test.ts. A whole-tree docReaders test took 3 s, over the per-test budget, so I dropped it.
  • bun x tsc --noEmit passes, and scripts/gate.ts --fix reported its comment rewraps, which are committed.
  • scripts/fast.ts exits 2 for this diff because it touches scripts, so I picked the files myself: 17 files, 293 pass, 0 fail. They are the tier and reader suites, scripts-fast, tree-scans, allow-lists, unused-exports, changed-compare, kept, the shards suites, docs-link and the rule tests.
  • keep_plan and estimate were not available to me, so there is no plan on the issue and no recorded estimate.

To try it: stage any docs/features/*.md change and run bun scripts/fast.ts --why. Stderr ends with the by reason: line, and the reads docs/… reasons appear on stdout.

🤖 Generated with Claude Code

Approved

Pipeline tooling only: no page, route, write or terminal surface changed, so the security, both-doors and README items do not apply. The change does what #14845 asked. The log says why each test was chosen (reasonLine, blindLine), and a doc with bounded readers no longer blinds the plan. Each new behaviour has a fixture test, docs/DEVELOP.md carries the how-to, there is no new dependency and I found no dead code. The two things I would do next: (1) treat a changed script whose readers are known like a doc, as the pull says; (2) test that scripts/fast.ts and scripts/test-shards.ts actually print the by-reason and blind lines, since only the tiers functions are covered now.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs