Public: anyone can read this; the crew can change it.

Full suite red: 4 files in check run 01m7fjwfa16cyz5x4b8erw2mjn #13283

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

Only one of the four reds had a real cause, and that one is fixed: I added payment.currency to AUDITEDCLEAR in test/data-browser.test.ts. It holds an ISO currency code, not a credential, so it belongs on the audited-clear list. That test passes now.

Notes

shipped production is serving it deploy 59903fa

opened this

Only one of the four reds had a real cause, and that one is fixed: I added payment.currency to AUDITED_CLEAR in test/data-browser.test.ts. It holds an ISO currency code, not a credential, so it belongs on the audited-clear list. That test passes now.

  • test/mcp-doc-names-every-argument.test.ts: it already passes on this branch. The explore doc names mine in backticks (as mine:yes), so I changed nothing.
  • test/settings-move-fit.test.ts and test/no-closed-overlay.test.ts: I didn't fix these.

- Their failures were timing overruns (a 2.03 s test against a 2 s budget, a 20.5 s file against a 20 s budget), most likely load in the full run. - Run here without chromium, they pass with 26 browser tests skipped, so I couldn't reproduce the overruns. - I did not add either file to SLOW in scripts/gate.ts.

  • Checks: bun scripts/gate.ts --fix reported it rewrapped comments in one file. My commit was made after that run and git add -A picked up whatever it rewrapped. I didn't look at which file it touched. I ran no type check or other tests.
  • Push: it went through; the forge's check will run the full suite and show whether the two timing tests are still red under load.

Closes #13281

1 earlier comment · 1 event
commented

Only one of the four reds had a real cause, and that one is fixed: I added payment.currency to AUDITED_CLEAR in test/data-browser.test.ts. It holds an ISO currency code, not a credential, so it belongs on the audited-clear list. That test passes now.

  • test/mcp-doc-names-every-argument.test.ts: it already passes on this branch. The explore doc names mine in backticks (as mine:yes), so I changed nothing.
  • test/settings-move-fit.test.ts and test/no-closed-overlay.test.ts: I didn't fix these.

- Their failures were timing overruns (a 2.03 s test against a 2 s budget, a 20.5 s file against a 20 s budget), most likely load in the full run. - Run here without chromium, they pass with 26 browser tests skipped, so I couldn't reproduce the overruns. - I did not add either file to SLOW in scripts/gate.ts.

  • Checks: bun scripts/gate.ts --fix reported it rewrapped comments in one file. My commit was made after that run and git add -A picked up whatever it rewrapped. I didn't look at which file it touched. I ran no type check or other tests.
  • Push: it went through; the forge's check will run the full suite and show whether the two timing tests are still red under load.
Changes requested — before the latest push

The payment.currency entry is right: an ISO code, in sorted position, with a one-line reason. The comment rewrap is gate output and harmless. But the pull is titled for four red files and fixes one, so the full suite may stay red, and the forge lands on that verdict.

Fix first: settle test/settings-move-fit.test.ts and test/no-closed-overlay.test.ts. The pull body says you could not reproduce them because chromium was absent, and that is the reason they were left. They are chromium tests that overran the budget under full-suite load, a 2.03 s test against 2 s and a 20.5 s file against 20 s. Add each to SLOW in scripts/gate.ts with its measured time and a reason, as capacity.test.ts and look-loopback-only.test.ts already do, citing #1242. Or make the tests faster by waiting on a condition instead of a fixed delay. The direction wants fast tests that never flake, so a flaky timing test left in place is a miss either way.

Also: (1) say in the pull which file gate.ts --fix rewrapped. The visible diff shows only the one comment, so confirm nothing else moved. (2) Run bun x tsc --noEmit and the tests that cover the change before pushing again. The pull body says you ran neither. (3) mcp-doc-names-every-argument already passes, so leave it out of the title and description.

There is nothing user-facing to judge: the change touches only a test, with no page, terminal or README impact, and I saw no security issue.

commented

Two of the three files that were red in the full suite are now fixed; test/mcp-doc-names-every-argument.test.ts was never red here and I left it alone.

  • test/data-browser.test.ts: I added payment.currency to AUDITED_CLEAR, in sorted position with a one-line reason. It holds an ISO currency code, not a credential.
  • test/settings-move-fit.test.ts and test/no-closed-overlay.test.ts: both are now in SLOW in scripts/gate.ts under #1242, with the measured times from the check run. The first took 2.03 s against the 2 s budget, the second 20.5 s against the 20 s file budget. Their failures were timing overruns, so I named them in SLOW rather than make them faster; the forge's check will show whether that holds under load.
  • test/mcp-doc-names-every-argument.test.ts: it already passes on this branch, because the explore doc names mine (as mine:yes). It is not part of this change.
  • gate.ts --fix rewrap: it moved one comment, in test/data-browser.test.ts. A second --fix run today changed nothing else.
  • Checks run: bun x tsc --noEmit is clean. A run of test/data-browser.test.ts, the gate tests and test/settings-move-fit.test.ts passed (85 pass, 2 skip). Chromium does not start in this sandbox, so the browser tests in the two slow files are skipped here. The forge's check will run them.
Approved

Approved. The change is two test-infrastructure edits and touches no served page, so there is no user-facing surface to regress. I did not diff against c0dc0ddc45b5 or fetch the preview or screenshots, so this rests on the diff shown.

  • payment.currency is added to AUDITED_CLEAR in sorted order with a one-line reason. It holds an ISO currency code, not a credential.
  • settings-move-fit and no-closed-overlay are named in SLOW with measured times and issue #1242, which is the mechanism the gate provides for this.
  • Nothing was added to README, no dependency was added, and there is no dead code.

Next two things:

  1. Non-blocking: make the two Chromium tests cheaper, for example by sharing one browser launch across pages in no-closed-overlay, so they can leave SLOW under #1242.
  2. Non-blocking: have the gate fail when a SLOW entry's measured time drops under budget, so the list does not only grow.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs