Public: anyone can read this; the crew can change it.
The test runner has a looks lane: chromium test files run only where chromium starts, named *.looks.test.ts, out of the gate budget #12506
Check · Review · Land · Live done — Landed and live in production.
test/.looks.test.ts files now run in a looks lane of their own, only where a Chromium actually starts. The gate tiers no longer see them.
shipped production is serving it deploy 7612a18
3 earlier comments
test/*.looks.test.ts files now run in a looks lane of their own, only where a Chromium actually starts. The gate tiers no longer see them. Gate, tsc, the gate and shard tests and the two moved files all pass; the two moved files' browser tests skipped in this sandbox because no Chromium starts here, so the lane running them for real is not exercised here. I did not run the full suite — the forge's check runs it.
- Runner (
scripts/test-shards.ts): on--gatewith the full tier, the runner appends the looks files iffindChromium()finds a binary andchromiumRuns()shows it starts. It prints eitherlooks lane, N file(s) with <binary>orlooks lane skipped: no Chromium that starts on this machine. The lane is exempt from the per-test budget. The fast tier and a box without a working browser leave the lane out. - Gate (
scripts/gate.ts):
- presentTests() skips looks files, so the exclusion and SLOW audits and the fast tier never count them.
- New looksTests() and isLooksFile() list them.
- I removed the two SLOW rows instead of renaming them, because a SLOW row for a file the gate no longer lists is itself an audit failure.
- Moves:
test/oauth-connect-browser.test.tsandtest/loading-chrome.test.tsare renamed to*.looks.test.tswith their contents unchanged, so no assertion is dropped.loading-chromefits the lane because it drives a real Chromium too. Thedocs/ORIGINS.mdreference is updated. - Tests (
test/test-budget-run.test.ts): three new tests prove the lane:
- With a stand-in Chromium that starts, the gate runs a looks file that is over the budget, and it stays green.
- With none that starts, the lane is skipped and the skip is named.
- The fast tier and presentTests leave looks files out.
I did not give the lane its own time budget; it is simply unbudgeted. outsideFiles and the scheduled run do not include looks files either, since they derive from presentTests.
To try it: bun scripts/test-shards.ts --gate and look for the looks-lane line near the top. FORGE_CHROMIUM=/path/to/chromium points it at a particular binary.
Pushed a fix for two of the four failures; I can't see the other two, so the next check will show whether they're gone. Locally the two fixed tests now pass, as do the tree-scan, review-rules, release-checkout, spawn-host-push and crew-sandbox-escape tests. bun scripts/gate.ts is clean.
- Cause: the two failing tests,
test/check-list.test.tsandtest/flake.test.ts, each built their own list of test files from the directory. That list included the two renamed*.looks.test.tsfiles, which the gate now leaves out, so the lists disagreed withgateFiles(). Both tests now skip.looks.test.tswhen they build the list. The gate's behaviour is unchanged. - Not mine: the check log shows
test/boundary-tenancy-cell.test.tsis red on main too. It failed with an egress refusal and is the file with the "two hosted forges on one machine…" test, so I left it alone. - Unknown: the log was cut off after the third failure name. If a fourth still fails on this push, I'll need its name from the new log.
The test runner now has a looks lane: *.looks.test.ts files run only where Chromium starts, outside the gate tiers and the test budget. The two Chromium tests moved into it, and the gate-list tests list present files as the gate does, leaving the looks lane's files aside. I merged the current main into the branch and pushed; the check has not yet run on the merged branch.
I found no failure that this pull caused:
test/deadends.test.ts("every instruction names a script or a tool that exists"): it fails withtool one at src/http/mcp.ts:287after the merge. It fails the same way on a plainorigin/main, so it is not this pull's. The check log also saystest/deadends.test.tsis red onmain.- "a stranger calling every listed tool…": it lives in
test/mcp-refusal-hints.test.ts, which the check log reports as red onmain. I never located or ran that test file, so I did not confirm it. test/explore-rank.test.ts: this was the one file the check log said is green onmainand red here. It passes here, both before and after the merge. It measures cold and warm request times, so I suspect load on the check machine, but I did not confirm that.test/holds.test.ts: passes here.
Before pushing, bun scripts/gate.ts and bun x tsc --noEmit were clean, and explore-rank and holds passed.
I did not touch the main failures, since fixing them is outside this change.
The test runner now has a looks lane: *.looks.test.ts files run only where a Chromium starts, outside the gate tiers and the per-test budget. loading-chrome and oauth-connect-browser moved into it.
The failing check did not come from this change, and I changed no code to fix it. The one test the log blames on this pull is test/spawn-stranger-cell.test.ts, "a stranger claims a small child…". It builds a whole forge inside a cell. The log shows that forge's requests refused: egress refused: kid--forge.test:443 is not in this repository's egress, and 127.0.0.1:… is a private destination. Those are cell-egress failures, not anything the runner or the renamed files touch. I couldn't run that test here, because it is skipped under FORGE_TEST_SOCKET.
The other failures in the list were red on main too: mirror-worker, snapshot-incremental, snapshot-repositories, boundary-tenancy-cell and studio-app-timing.
To get a fresh check run, I merged current main into the branch and pushed. Before pushing, bun scripts/gate.ts and bun x tsc --noEmit were clean. test/test-budget-run.test.ts, test/check-list.test.ts and test/flake.test.ts pass, 63 tests.
If spawn-stranger-cell fails again with the same egress lines, the cause is the check box's egress for cell children, not this pull. A person would need to look at that.
Try it: bun scripts/test-shards.ts --gate prints test-shards: looks lane, N file(s) with <chromium> where a browser starts. Elsewhere it prints looks lane skipped: no Chromium that starts on this machine.
🤖 Generated with Claude Code
Clean. The looks lane runs only on the full gate where findChromium() and chromiumRuns() say a browser starts, and it names the skip when none does. The fast tier and presentTests leave the files out, and the lane is exempt from the per-test budget. Three runner tests prove it: runs and stays green over budget, skipped and named, and fast tier excludes. The renames keep content unchanged, and the stale SLOW rows are rightly removed because a SLOW row for an unlisted file fails the audit. No security surface touched. I did not run the lane against a real Chromium here.
test/*.looks.test.tsfiles now run in a looks lane of their own, only where a Chromium actually starts. The gate tiers no longer see them. Gate,tsc, the gate and shard tests and the two moved files all pass; the two moved files' browser tests skipped in this sandbox because no Chromium starts here, so the lane running them for real is not exercised here. I did not run the full suite — the forge's check runs it.scripts/test-shards.ts): on--gatewith the full tier, the runner appends the looks files iffindChromium()finds a binary andchromiumRuns()shows it starts. It prints eitherlooks lane, N file(s) with <binary>orlooks lane skipped: no Chromium that starts on this machine. The lane is exempt from the per-test budget. The fast tier and a box without a working browser leave the lane out.scripts/gate.ts):-
presentTests()skips looks files, so the exclusion and SLOW audits and the fast tier never count them. - NewlooksTests()andisLooksFile()list them. - I removed the two SLOW rows instead of renaming them, because a SLOW row for a file the gate no longer lists is itself an audit failure.test/oauth-connect-browser.test.tsandtest/loading-chrome.test.tsare renamed to*.looks.test.tswith their contents unchanged, so no assertion is dropped.loading-chromefits the lane because it drives a real Chromium too. Thedocs/ORIGINS.mdreference is updated.test/test-budget-run.test.ts): three new tests prove the lane:- With a stand-in Chromium that starts, the gate runs a looks file that is over the budget, and it stays green. - With none that starts, the lane is skipped and the skip is named. - The fast tier and
presentTestsleave looks files out.I did not give the lane its own time budget; it is simply unbudgeted.
outsideFilesand the scheduled run do not include looks files either, since they derive frompresentTests.To try it:
bun scripts/test-shards.ts --gateand look for the looks-lane line near the top.FORGE_CHROMIUM=/path/to/chromiumpoints it at a particular binary.Closes #12496