Public: anyone can read this; the crew can change it.

readKpis counts the day's landings without the correlated NOT EXISTS over every pull #7804

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`readKpis` now counts the day's landings in one pass. The old query ran a `NOT EXISTS` over every pull for each landing; that is gone. On 20,000 merged pulls the read takes about 14 ms, where the old query took about 8.7 s.

shipped production is serving it deploy fc82d1a

opened this

readKpis now counts the day's landings in one pass. The old query ran a NOT EXISTS over every pull for each landing; that is gone. On 20,000 merged pulls the read takes about 14 ms, where the old query took about 8.7 s.

How it works: a new firstLandings in src/flow/kpis.ts reads, in one query, every merged pull whose commit also appears among the window's landings. That includes pulls merged before the window. TypeScript then keeps the earliest per (repo, commit), ordered by merged_at then issue_id, and counts that pull only if it is a real landing on the default branch.

The counting rule is unchanged:

  • A commit that already landed before the day and is carried again today counts zero times.
  • The same commit merged twice in one instant counts once.
  • A pull merged into the base it found counts zero times.

Tests:

  • The existing tests in test/home-kpis.test.ts pass unchanged.
  • New test/kpis-landings-scale.test.ts seeds 20,000 merged pulls across two repos, with these cases planted:

- repeated commits - pulls merged into their own base - a commit carried from three days ago into the last hour - two pulls merging the same commit at the same moment

  • It checks that readKpis answers within 100 ms, judged against baselineMs. That timing runs on the production schema, with no extra index.
  • It then checks that the day's count and the last hour's count equal what the old query gives on the same data.

Worth a look: to keep that comparison from taking seconds, the test adds an index on merged_oid to its own throwaway database, after the timing test. Production gets no migration and no index.

When I broke the dedupe on purpose, the new test failed (663 counted, 648 expected). bun x tsc --noEmit is clean. scripts/fast.ts named 1,042 test files, too many to run, so I ran the nine files that cover the kpis instead, including home-200 and home-timing. All 75 tests pass.

🤖 Generated with Claude Code

Closes #7627

commented

readKpis now counts the day's landings in one pass. The old query ran a NOT EXISTS over every pull for each landing; that is gone. On 20,000 merged pulls the read takes about 14 ms, where the old query took about 8.7 s.

How it works: a new firstLandings in src/flow/kpis.ts reads, in one query, every merged pull whose commit also appears among the window's landings. That includes pulls merged before the window. TypeScript then keeps the earliest per (repo, commit), ordered by merged_at then issue_id, and counts that pull only if it is a real landing on the default branch.

The counting rule is unchanged:

  • A commit that already landed before the day and is carried again today counts zero times.
  • The same commit merged twice in one instant counts once.
  • A pull merged into the base it found counts zero times.

Tests:

  • The existing tests in test/home-kpis.test.ts pass unchanged.
  • New test/kpis-landings-scale.test.ts seeds 20,000 merged pulls across two repos, with these cases planted:

- repeated commits - pulls merged into their own base - a commit carried from three days ago into the last hour - two pulls merging the same commit at the same moment

  • It checks that readKpis answers within 100 ms, judged against baselineMs. That timing runs on the production schema, with no extra index.
  • It then checks that the day's count and the last hour's count equal what the old query gives on the same data.

Worth a look: to keep that comparison from taking seconds, the test adds an index on merged_oid to its own throwaway database, after the timing test. Production gets no migration and no index.

When I broke the dedupe on purpose, the new test failed (663 counted, 648 expected). bun x tsc --noEmit is clean. scripts/fast.ts named 1,042 test files, too many to run, so I ran the nine files that cover the kpis instead, including home-200 and home-timing. All 75 tests pass.

🤖 Generated with Claude Code

Approved

Reviewed every changed line. No security findings: the query is fully static with bound parameters, there is no new input or output surface, and nothing is written. The semantics match the old NOT EXISTS: same repo and commit, ordered by merged_at then issue_id, empty and NULL oids excluded, merges into the pull's own base excluded. The new test seeds duplicates, a commit carried from before the window, and same-instant twins, then compares against the old query. It would fail if the dedupe were removed. The timing test runs on the production schema. No dependencies were added and there is no dead code. I did not open the preview or the screenshots, because the change is internal to readKpis and has no UI.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs