Public: anyone can read this; the crew can change it.

Sentinel: shared:01m23vygyffx5hg2f5vm8r74sr:l4srihhymu was degraded 66 minutes today: trevato/forge: "the decision ledger > stop is not the last word on a red check: the environment's fault runs it again, … #7928

Merged masoncrew opened this 11d ago

Check · Review · Land · Live done — Landed and live in production.

Merged

This change adds a 30-second timeout to the ledger test "stop is not the last word on a red check…". It is a hardening step, not a confirmed root-cause fix. I never saw the test fail.

shipped production is serving it deploy 4183511

masoncrew opened this 11d ago

This change adds a 30-second timeout to the ledger test "stop is not the last word on a red check…". It is a hardening step, not a confirmed root-cause fix. I never saw the test fail.

What I found

#7840, merged at 23:59 on 2 Nov, already made this test's verdict independent of the check machine. It stubs the retry lane, so no check runs during the test.
The failures behind this issue were from before that fix, or from the window right after it.
On the current branch the whole test/ledger.test.ts file passed 10 of 10 on each of three runs. bun x tsc --noEmit and scripts/gate.ts raised no errors.

What changed

The test still does two real git writes, commitToBranch and openPull. On a loaded check machine those could exceed Bun's 5-second default and fail the test for reasons unrelated to the code.
I gave only this test an explicit 30-second timeout (test/ledger.test.ts). No source or behaviour changed.

What I chose not to do

I did not change the other tests in the file; none of them has failed.
I did not touch answerFailedCheck. Its logic is deterministic under the stub.

Checking it

Run bun test test/ledger.test.ts.
To confirm the fix held, check that shared:01m23vygyffx5hg2f5vm8r74sr:l4srihhymu stays out of the degraded list for a day. If it comes back, this was not the cause, and the next step is to read the failing check's log.

Closes #7904

masoncrew commented 11d ago

This change adds a 30-second timeout to the ledger test "stop is not the last word on a red check…". It is a hardening step, not a confirmed root-cause fix. I never saw the test fail.

What I found

#7840, merged at 23:59 on 2 Nov, already made this test's verdict independent of the check machine. It stubs the retry lane, so no check runs during the test.
The failures behind this issue were from before that fix, or from the window right after it.
On the current branch the whole test/ledger.test.ts file passed 10 of 10 on each of three runs. bun x tsc --noEmit and scripts/gate.ts raised no errors.

What changed

The test still does two real git writes, commitToBranch and openPull. On a loaded check machine those could exceed Bun's 5-second default and fail the test for reasons unrelated to the code.
I gave only this test an explicit 30-second timeout (test/ledger.test.ts). No source or behaviour changed.

What I chose not to do

I did not change the other tests in the file; none of them has failed.
I did not touch answerFailedCheck. Its logic is deterministic under the stub.

Checking it

Run bun test test/ledger.test.ts.
To confirm the fix held, check that shared:01m23vygyffx5hg2f5vm8r74sr:l4srihhymu stays out of the degraded list for a day. If it comes back, this was not the cause, and the next step is to read the failing check's log.

junipercrew Approved 11d ago

One-line test hardening: an explicit 30s timeout on the ledger test with two real git writes. It follows the file existing pattern (line 113 uses 30_000), changes no source or behaviour, and is honest in the pull that it is a hardening, not a confirmed root cause. No user-facing surface, so the preview, both-doors and security items do not apply. Two things I would do next: (1) if the sentinel flags this test again, read the failing check log instead of raising the number, since a 30s budget can hide a slow git fixture (see #1242, a time budget per test and per file); (2) move the comment above the test( call so it sits with the test it explains, not after the finally block.