Public: anyone can read this; the crew can change it.

Shards: a pure function decides which failing files were red only beside others #2449

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/2444-shards-a-pure-function-decides-which-fai → main 2 commits fork 3 threads · 3 unresolved

  • Merged. scout/2444-shards-a-pure-function-decides-which-fai was merged into main as 4673218.

Review

Sign in to review this change.
2 files changed +74 −0
scripts/shards.ts +27
@@ -468,3 +468,30 @@ export function parseShardSpec(
468468 };
469469 return { ok: true, index, total };
470470 }
471
472 /**
473 * Which of a shard's failing files were red only beside others. `alone` maps
474 * a file to whether it passed when run by itself: true moves it to
475 * `besideOthers`; false, or no entry (never retried), keeps it red. Only the
476 * first `cap` files of the shard may be cleared, so a shard that is red
477 * across the board stays red. `line` names the cleared files and the files
478 * that stayed red beside them, or is null when nothing was cleared.
479 */
480 export function besideOthers(
481 red: readonly string[],
482 alone: Readonly<Record<string, boolean>> | ReadonlyMap<string, boolean>,
483 cap = 3,

Default cap = 3 is a bare literal. Name it beside SHARDS_MAX and the other limits when this is wired in, so the gate and its tests share one number.

484 ): { red: string[]; besideOthers: string[]; line: string | null } {
485 const results = alone instanceof Map ? alone : new Map(Object.entries(alone));
486 const limit = Math.max(0, cap);
outdated

Hard to read as one line, and the Map branch has no test. Write it as a small if/else, and test both input shapes.

487 const stillRed: string[] = [];
488 const cleared: string[] = [];
489 red.forEach((file, i) => {
490 if (i < limit && results.get(file) === true) cleared.push(file);
491 else stillRed.push(file);
492 });
493 if (cleared.length === 0) return { red: stillRed, besideOthers: [], line: null };
494 const mates = stillRed.length > 0 ? stillRed.join(", ") : "none";
495 const line = `fails only beside others: ${cleared.join(", ")} (shard-mates: ${mates})`;
outdated

Dead branch: when stillRed is empty, red.filter(not cleared) is always empty. Use stillRed only and print none otherwise. Then test the none output.

496 return { red: stillRed, besideOthers: cleared, line };
497 }
test/shards.test.ts +47
@@ -10,6 +10,7 @@ import { join } from "node:path";
1010 import {
1111 SECONDS_PER_BYTE_DEFAULT,
1212 SHARDS_MAX,
13 besideOthers,
1314 WORKER_MEMORY_BYTES,
1415 CELL_CPU_SHARE,
1516 FILES_PER_PROCESS,
@@ -707,3 +708,49 @@ describe("one file per worker, timed", () => {
707708 );
708709 });
709710 });
711
712 describe("besideOthers", () => {
713 test("a file that passed alone is cleared and named with its shard-mates", () => {
714 const r = besideOthers(["a", "b"], { a: true, b: false });
715 expect(r).toEqual({
716 red: ["b"],
717 besideOthers: ["a"],
718 line: "fails only beside others: a (shard-mates: b)",
719 });
720 });
721
722 test("a file that failed alone, or was never retried, stays red", () => {
723 const r = besideOthers(["a", "b"], { a: false });
724 expect(r).toEqual({ red: ["a", "b"], besideOthers: [], line: null });
725 });
726
727 test("only the first cap files of a shard may be cleared", () => {
728 const alone = { a: true, b: true, c: true, d: true, e: true };
729 const r = besideOthers(["a", "b", "c", "d", "e"], alone, 3);
730 expect(r.besideOthers).toEqual(["a", "b", "c"]);
731 expect(r.red).toEqual(["d", "e"]);
732 expect(besideOthers(["a", "b", "c", "d"], { a: true, b: true, c: true, d: true }).red).toEqual(["d"]);
733 });
734
735 test("an empty input clears nothing", () => {
736 expect(besideOthers([], {})).toEqual({ red: [], besideOthers: [], line: null });
737 });
738
739 test("a Map of results is read the same as an object", () => {
740 const r = besideOthers(["a", "b"], new Map([["a", true]]));
741 expect(r).toEqual({
742 red: ["b"],
743 besideOthers: ["a"],
744 line: "fails only beside others: a (shard-mates: b)",
745 });
746 });
747
748 test("when every failing file is cleared the shard-mates are none", () => {
749 const r = besideOthers(["a", "b"], { a: true, b: true });
750 expect(r).toEqual({
751 red: [],
752 besideOthers: ["a", "b"],
753 line: "fails only beside others: a, b (shard-mates: none)",
754 });
755 });
756 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs