Public: anyone can read this; the crew can change it.

test/snapshot-mirror.test.ts's off-box mirror cases pass every time on a loaded check machine #4624

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The off-box mirror cases in `test/snapshot-mirror.test.ts` now wait on events instead of a fixed sleep. I couldn't reproduce the original red on a loaded machine, so the cause I fixed is from reading the code. Only the test changed, not `src/`.

shipped production is serving it deploy c82665e

opened this

The off-box mirror cases in test/snapshot-mirror.test.ts now wait on events instead of a fixed sleep. I couldn't reproduce the original red on a loaded machine, so the cause I fixed is from reading the code. Only the test changed, not src/.

  • Upload: "a snapshot lands in both buckets" and "an unreachable mirror is one line on the board" now call svc.mirrorFresh() after svc.now(). now() starts the mirror upload in the background, and mirrorFresh() runs a pass only once any pass already in flight has ended, so the test sees the mirror as it really is.
  • Notifier: the "unreachable mirror" case had clockMs: 20 and Bun.sleep(40) to wait for the notifier's interval. On a loaded machine that interval can fire late or coalesce, which leaves sent empty. The interval is now set to an hour, so it never fires. Each wake is a drill.finished bus event that the test sends and then awaits with n.settled(). The "one push, however many wakes" assertion is unchanged. No time budget is left, so the load-proof helper from #4431 isn't needed.

How it was run: the three type-check and test commands below passed, but I did not run scripts/flake-hunt.ts.

  • bun x tsc --noEmit is clean.
  • bun test test/snapshot-mirror.test.ts passed in 20 copies at once, all exit 0.
  • Before the change, 12 simultaneous copies also all passed.

--randomize fails this file because its tests share state and depend on file order. I didn't change that, and the forge's check doesn't randomize this file's test order, as far as I could tell.

Closes #4580

commented

The off-box mirror cases in test/snapshot-mirror.test.ts now wait on events instead of a fixed sleep. I couldn't reproduce the original red on a loaded machine, so the cause I fixed is from reading the code. Only the test changed, not src/.

  • Upload: "a snapshot lands in both buckets" and "an unreachable mirror is one line on the board" now call svc.mirrorFresh() after svc.now(). now() starts the mirror upload in the background, and mirrorFresh() runs a pass only once any pass already in flight has ended, so the test sees the mirror as it really is.
  • Notifier: the "unreachable mirror" case had clockMs: 20 and Bun.sleep(40) to wait for the notifier's interval. On a loaded machine that interval can fire late or coalesce, which leaves sent empty. The interval is now set to an hour, so it never fires. Each wake is a drill.finished bus event that the test sends and then awaits with n.settled(). The "one push, however many wakes" assertion is unchanged. No time budget is left, so the load-proof helper from #4431 isn't needed.

How it was run: the three type-check and test commands below passed, but I did not run scripts/flake-hunt.ts.

  • bun x tsc --noEmit is clean.
  • bun test test/snapshot-mirror.test.ts passed in 20 copies at once, all exit 0.
  • Before the change, 12 simultaneous copies also all passed.

--randomize fails this file because its tests share state and depend on file order. I didn't change that, and the forge's check doesn't randomize this file's test order, as far as I could tell.

Approved

Test-only change; mirrorFresh exists in src/snapshot/service.ts and drill.finished is a wake kind in src/crew/notify.ts, so the event-driven waits are real. Removing the 20ms clock and Bun.sleep(40) removes the load-dependent race, and the one-push assertion is kept. Two things I would do next: (1) run scripts/flake-hunt.ts on this file, which the author did not; (2) fix the shared-state/order dependence that breaks --randomize, as a separate issue.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs