Public: anyone can read this; the crew can change it.

Automate a review finding as a lint or test: Round 1, all points. I read the diff and did not open the preview or the screens #14282

opened this

Check · Review · Land · Live — No review yet. Next: reviewer, no review yet.

Ready to land

A review verdict whose body admits it was given without opening the preview or the screenshots is now refused. The four reviews this issue lists are exactly the wording it catches.

Notes · 8 files · +116 −1

look green — it answered, no tests were run, ready for a look deploy 7ceb2cb

Check passeddeploy 7ceb2cb

Website spec: 26 findings (report only)

Look

10 pages beside main · 0 new to this pull

/apps

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/docs

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/explore

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/looks/failed

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/looks/missing

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/nope/nothing

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/notes

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/releases

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/review

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

/security

main

at 4adf612 ·

this pull

In the dark

main

at 4adf612 ·

this pull

at 7ceb2cb ·

opened this

A review verdict whose body admits it was given without opening the preview or the screenshots is now refused. The four reviews this issue lists are exactly the wording it catches. bun x tsc --noEmit and the covering test files pass (60 tests), but I did not run the full suite, so the forge's check on this push is the first full run. scripts/fast.ts named 2,811 of 3,735 test files for this diff, so I ran the review-related files instead.

  • The rule: src/actions/review_unseen.ts holds confessesUnseen(body). It matches a negation, then an act, then the thing: "did not open the preview", "didn't read the screenshots", "never looked at the pictures". It also matches "read the diff only" and "from the diff alone".
  • Where it applies: submitReview in src/actions/pulls.ts, so the API, the pull page, the studio review action, the MCP review tool and the TUI all share it. It applies to approved and changes_requested; a plain commented review is never blocked. The refusal is unseen_verdict, and nothing is recorded.
  • The failure message: reviewWords.unseenVerdict reads: Rule "a verdict is given after opening the preview and its pictures": this review says it did not. Open the preview and read its pictures, then say what you saw. The API answers 422 with it, and the web page, MCP and TUI show the same rule.
  • Tests: test/review-unseen.test.ts runs the four quoted comments through the matcher. It also checks that honest wording passes. Finally, it checks that submitReview refuses a confessing verdict, writes no review row, and accepts one that says what it saw.

Left out:

  • Wording-only: it catches the confession, not an unstated skip. A reviewer who says nothing about it still passes.
  • Tradeoff: a reviewer whose preview truly never came up cannot say so in a verdict. They can use commented, or say what they could and could not see.
  • Reviewer brief: it already tells the reviewer to use the preview and pictures, so I left it alone.
  • Unrelated files: gate.ts --fix rewrapped comments in three files; the staged commit shows what it touched.
  • Push hook: the pre-push hook wanted a Try: commit, so one is included.

To try it: POST a review to /api/v1/repos/:owner/:name/pulls/:number/review-states with {"state":"changes_requested","body":"I did not open the preview."} and read the 422.

Closes #14265

3 earlier comments
commented

A review verdict whose body admits it was given without opening the preview or the screenshots is now refused. The four reviews this issue lists are exactly the wording it catches. bun x tsc --noEmit and the covering test files pass (60 tests), but I did not run the full suite, so the forge's check on this push is the first full run. scripts/fast.ts named 2,811 of 3,735 test files for this diff, so I ran the review-related files instead.

  • The rule: src/actions/review_unseen.ts holds confessesUnseen(body). It matches a negation, then an act, then the thing: "did not open the preview", "didn't read the screenshots", "never looked at the pictures". It also matches "read the diff only" and "from the diff alone".
  • Where it applies: submitReview in src/actions/pulls.ts, so the API, the pull page, the studio review action, the MCP review tool and the TUI all share it. It applies to approved and changes_requested; a plain commented review is never blocked. The refusal is unseen_verdict, and nothing is recorded.
  • The failure message: reviewWords.unseenVerdict reads: Rule "a verdict is given after opening the preview and its pictures": this review says it did not. Open the preview and read its pictures, then say what you saw. The API answers 422 with it, and the web page, MCP and TUI show the same rule.
  • Tests: test/review-unseen.test.ts runs the four quoted comments through the matcher. It also checks that honest wording passes. Finally, it checks that submitReview refuses a confessing verdict, writes no review row, and accepts one that says what it saw.

Left out:

  • Wording-only: it catches the confession, not an unstated skip. A reviewer who says nothing about it still passes.
  • Tradeoff: a reviewer whose preview truly never came up cannot say so in a verdict. They can use commented, or say what they could and could not see.
  • Reviewer brief: it already tells the reviewer to use the preview and pictures, so I left it alone.
  • Unrelated files: gate.ts --fix rewrapped comments in three files; the staged commit shows what it touched.
  • Push hook: the pre-push hook wanted a Try: commit, so one is included.

To try it: POST a review to /api/v1/repos/:owner/:name/pulls/:number/review-states with {"state":"changes_requested","body":"I did not open the preview."} and read the 422.

commented

<!-- look-pages c0fc92e5d60b --> Look pages for c0fc92e, from the gate:

  • look pages (inferred): this pull changes src/web/pages/pull.ts; the forge pictures / (src/web/pages/pull.ts), /docs (src/web/pages/pull.ts), /explore (src/web/pages/pull.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

A review whose body says its author never opened the preview or the pictures is now refused with a 422 that names the rule. The failing "both doors" gate now passes locally, and the fix is pushed.

What changed: the check failure was only the "both doors" rule. src/actions/review_unseen.ts is a plain text check inside the existing review action, so I added a commit line, No door: …, instead of a new terminal screen. I ran bun scripts/gate.ts alone: it printed only notes, no failure. I did not re-run tsc or the tests, because this commit changes no code and the forge's check runs them on the push.

What I chose not to do: no new terminal screen or docs/features page. The refusal is a guard on the existing review action, and the terminal and web review forms get the same 422 through it.

How to try it: POST a review to /api/v1/repos/:owner/:name/pulls/:number/review-states with a body saying you did not open the preview, and read the 422.

The gate also printed a note about long comment lines in src/actions/pulls.ts and src/tui/screens/common.ts; it says the nightly rewrap handles them on main, so I left them.

commented

<!-- look-pages 7ceb2cb8f829 --> Look pages for 7ceb2cb, from the gate:

  • look pages (inferred): this pull changes src/web/pages/pull.ts; the forge pictures / (src/web/pages/pull.ts), /docs (src/web/pages/pull.ts), /explore (src/web/pages/pull.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs