One night of the forge's flake hunt and outside gate is compared with Forgejo's, and the result is written down #9056

This adds scripts/flake-compare.ts, which puts one night of the forge's flake hunt and outside-gate run beside the Forgejo jobs' run and names every difference. The second box on the issue is still open: no real night has been compared, so no output is posted and the Forgejo jobs stay on for now.

What it reads. Both sides run scripts/flake-hunt.ts and print the same lines, so one reader handles both kinds of log. Forgejo's timestamp prefixes don't get in the way. From each side's logs it collects:

• which files outside the gate were red or green;
• Red: issues filed, updated or closed;
• Flaky: issues filed or found already open;
• files red in every run (broken);
• files quarantined, and the flake-hunt/<date> quarantine pull.

It prints each side, then each difference with the file it concerns, and exits 1 when there is one. If one side filed and the other said nothing, that counts as agreement: the second side found the first side's issue already open.

How to run it: bun scripts/flake-compare.ts --forge hunt.log --forge outside.log --forgejo slice-1.log … --forgejo outside.log. The forge's job log doesn't name its quarantine pull; that branch is only on the job's Resilience line (quarantine pull on flake-hunt/<date>), so paste that line into the forge log first.

Tests. test/flake-compare.test.ts covers:

• reading a night;
• the same night on both sides (one with timestamps) giving no differences;
• one side silent where the other filed giving no differences;
• a flaky test found by only one side being named;
• a file red on one side and green on the other, and a quarantine or pull on only one side, being named.

Docs. docs/DEVELOP.md now says the Forgejo jobs can't be switched off yet. They go (#9020) once one compared night agrees. A file red outside the gate on one side and green on the other blocks the switch until explained; a flaky file only one side found can be down to the seed. docs/features/nightly.md points to the script.

Why no real night is posted. The forge's /resilience/nightly/<job> returns 404 to this token because it is admin-only, and the Forgejo mirror's Actions logs can't be reached from here. An admin needs to run the script on one night's logs and post its output on #9045.

bun scripts/gate.ts and bun x tsc --noEmit pass, as do test/flake-compare.test.ts, test/flake.test.ts and test/check-list.test.ts (53 tests). scripts/fast.ts said the change was too broad to name its tests (exit 2), so those were picked by hand. The keep_plan tool wasn't available in this run, so the issue has no plan.

🤖 Generated with Claude Code

Closes #9045

This adds scripts/flake-compare.ts, which puts one night of the forge's flake hunt and outside-gate run beside the Forgejo jobs' run and names every difference. The second box on the issue is still open: no real night has been compared, so no output is posted and the Forgejo jobs stay on for now.

What it reads. Both sides run scripts/flake-hunt.ts and print the same lines, so one reader handles both kinds of log. Forgejo's timestamp prefixes don't get in the way. From each side's logs it collects:

• which files outside the gate were red or green;
• Red: issues filed, updated or closed;
• Flaky: issues filed or found already open;
• files red in every run (broken);
• files quarantined, and the flake-hunt/<date> quarantine pull.

It prints each side, then each difference with the file it concerns, and exits 1 when there is one. If one side filed and the other said nothing, that counts as agreement: the second side found the first side's issue already open.

How to run it: bun scripts/flake-compare.ts --forge hunt.log --forge outside.log --forgejo slice-1.log … --forgejo outside.log. The forge's job log doesn't name its quarantine pull; that branch is only on the job's Resilience line (quarantine pull on flake-hunt/<date>), so paste that line into the forge log first.

Tests. test/flake-compare.test.ts covers:

• reading a night;
• the same night on both sides (one with timestamps) giving no differences;
• one side silent where the other filed giving no differences;
• a flaky test found by only one side being named;
• a file red on one side and green on the other, and a quarantine or pull on only one side, being named.

Docs. docs/DEVELOP.md now says the Forgejo jobs can't be switched off yet. They go (#9020) once one compared night agrees. A file red outside the gate on one side and green on the other blocks the switch until explained; a flaky file only one side found can be down to the seed. docs/features/nightly.md points to the script.

Why no real night is posted. The forge's /resilience/nightly/<job> returns 404 to this token because it is admin-only, and the Forgejo mirror's Actions logs can't be reached from here. An admin needs to run the script on one night's logs and post its output on #9045.

bun scripts/gate.ts and bun x tsc --noEmit pass, as do test/flake-compare.test.ts, test/flake.test.ts and test/check-list.test.ts (53 tests). scripts/fast.ts said the change was too broad to name its tests (exit 2), so those were picked by hand. The keep_plan tool wasn't available in this run, so the issue has no plan.

🤖 Generated with Claude Code

Left #9045 open: 1 unticked item remain (Its output for a real night is posted on this issue, and docs/DEVELOP.md says whether the Forgejo jobs can be switched off.).