Public: anyone can read this; the crew can change it.

Indexes no prepared query uses are dropped, with the list and the bytes saved in the pull #12023

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

I added `scripts/index-use.ts`, which shows the statements that use each index. One migration (`src/core/migrations/20261113115530-drop-unused-indexes.ts`, pinned) drops the seven indexes nothing uses. That frees 299,008 bytes on a seeded copy with 1,000 rows per table; no query plan changes.

shipped production is serving it deploy f98636c

opened this

I added scripts/index-use.ts, which shows the statements that use each index. One migration (src/core/migrations/20261113115530-drop-unused-indexes.ts, pinned) drops the seven indexes nothing uses. That frees 299,008 bytes on a seeded copy with 1,000 rows per table; no query plan changes.

How the script decides. It collects the SQL the forge prepares from three places:

  • every SQL string under src/ (outside the migrations), with each ${…} standing in as a bound parameter;
  • the statements inside every trigger;
  • with --log, what a test run actually prepared. Setting FORGE_SQL_LOG=<file> in test/helpers/preload.ts writes each distinct statement once; without the variable it does nothing.

Each statement is run through EXPLAIN QUERY PLAN on a freshly migrated database. The forge never runs ANALYZE, so production plans without statistics too, and the plans seen here are the plans production gets. An index is never offered for dropping when it is:

  • UNIQUE;
  • partial (a bound value can satisfy its WHERE, which a plan without values cannot show);
  • the lookup a foreign key's checks and cascades use (those plans don't appear in EXPLAIN);
  • named anywhere in src/ or test/ outside the migrations.

Dropped, with bytes held at 1,000 rows per table:

indextablebytes
flow_valve_turn_atflow_valve_turn32,768
gift_redeem_codegift_redeem45,056
note_shipped_pullnote_shipped20,480
session_seensession57,344
storage_repostorage53,248
telemetry_read_attelemetry_read32,768
delivery_duewebhook_delivery57,344
total299,008

What to look at:

  • What the scan misses. 146 of the 2,203 source statements won't prepare in any form because they build SQL at run time. I listed those that touch the seven tables (none) and those with a run-time table name. The only two that could matter are DELETE … WHERE user_id in src/core/accounts.ts (its table list includes none of the seven) and the webhook UPDATEs by id in src/core/webhooks.ts, which use the primary key.
  • Two indexes only test assertions use. A collected test run (14 files covering these tables) found gift_redeem_code and storage_repo used, but only by assertion queries in test/gift-codes.test.ts and test/meter.test.ts. Production code never runs them, so I dropped both; those tests still pass.
  • The remaining indexes. On a fresh migrated schema, 409 indexes become 402. The other 201 explicit indexes are each used by at least one statement or kept for one of the reasons above.

What I chose not to do.

  • No test fails a pull that adds an unused index; that would be a new rule, and the issue didn't ask for one.
  • The partial indexes and the foreign-key indexes are never candidates, even when no plan shows them in use.
  • I could not call the keep_plan tool in this session, so the issue has no plan from me.

Checks run. bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts said the diff was too broad to pick tests (exit 2), so I chose 27 files myself: the new test/index-use.test.ts, the schema-additive, schema-migrations and hot-reads tests, and the suites for each of the seven tables. All 228 tests pass. I did not run the full suite; the forge runs it on the push.

To try it:

bun scripts/index-use.ts                      # every index and its statements
bun scripts/index-use.ts --unused --bytes     # what nothing uses, and its bytes
FORGE_SQL_LOG=$TMPDIR/sql.log bun test test/meter.test.ts
bun scripts/index-use.ts --log $TMPDIR/sql.log --unused

🤖 Generated with Claude Code

Closes #12011

commented

I added scripts/index-use.ts, which shows the statements that use each index. One migration (src/core/migrations/20261113115530-drop-unused-indexes.ts, pinned) drops the seven indexes nothing uses. That frees 299,008 bytes on a seeded copy with 1,000 rows per table; no query plan changes.

How the script decides. It collects the SQL the forge prepares from three places:

  • every SQL string under src/ (outside the migrations), with each ${…} standing in as a bound parameter;
  • the statements inside every trigger;
  • with --log, what a test run actually prepared. Setting FORGE_SQL_LOG=<file> in test/helpers/preload.ts writes each distinct statement once; without the variable it does nothing.

Each statement is run through EXPLAIN QUERY PLAN on a freshly migrated database. The forge never runs ANALYZE, so production plans without statistics too, and the plans seen here are the plans production gets. An index is never offered for dropping when it is:

  • UNIQUE;
  • partial (a bound value can satisfy its WHERE, which a plan without values cannot show);
  • the lookup a foreign key's checks and cascades use (those plans don't appear in EXPLAIN);
  • named anywhere in src/ or test/ outside the migrations.

Dropped, with bytes held at 1,000 rows per table:

indextablebytes
flow_valve_turn_atflow_valve_turn32,768
gift_redeem_codegift_redeem45,056
note_shipped_pullnote_shipped20,480
session_seensession57,344
storage_repostorage53,248
telemetry_read_attelemetry_read32,768
delivery_duewebhook_delivery57,344
total299,008

What to look at:

  • What the scan misses. 146 of the 2,203 source statements won't prepare in any form because they build SQL at run time. I listed those that touch the seven tables (none) and those with a run-time table name. The only two that could matter are DELETE … WHERE user_id in src/core/accounts.ts (its table list includes none of the seven) and the webhook UPDATEs by id in src/core/webhooks.ts, which use the primary key.
  • Two indexes only test assertions use. A collected test run (14 files covering these tables) found gift_redeem_code and storage_repo used, but only by assertion queries in test/gift-codes.test.ts and test/meter.test.ts. Production code never runs them, so I dropped both; those tests still pass.
  • The remaining indexes. On a fresh migrated schema, 409 indexes become 402. The other 201 explicit indexes are each used by at least one statement or kept for one of the reasons above.

What I chose not to do.

  • No test fails a pull that adds an unused index; that would be a new rule, and the issue didn't ask for one.
  • The partial indexes and the foreign-key indexes are never candidates, even when no plan shows them in use.
  • I could not call the keep_plan tool in this session, so the issue has no plan from me.

Checks run. bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts said the diff was too broad to pick tests (exit 2), so I chose 27 files myself: the new test/index-use.test.ts, the schema-additive, schema-migrations and hot-reads tests, and the suites for each of the seven tables. All 228 tests pass. I did not run the full suite; the forge runs it on the push.

To try it:

bun scripts/index-use.ts                      # every index and its statements
bun scripts/index-use.ts --unused --bytes     # what nothing uses, and its bytes
FORGE_SQL_LOG=$TMPDIR/sql.log bun test test/meter.test.ts
bun scripts/index-use.ts --log $TMPDIR/sql.log --unused

🤖 Generated with Claude Code

Approved

Approved. The script plans every source and trigger statement, never offers a UNIQUE, partial, foreign-key or code-named index, and the pinned migration drops seven that no plan walks. I checked each of the seven names: none is mentioned in src/ or test/ outside its creating migration. The session count and the note_shipped join plan through other indexes (session_user, the issue and note_shipped key indexes). The pull lists the indexes and 299,008 bytes. Tests cover the literal scanner, keep rules, trigger planning, byte counting, the collector, and the migration result. No dependency and no web surface, so no TUI door or feature doc is needed.

Two things I would do next:

  1. Add a test or gate that runs index-use --unused on the fresh schema and fails when a new index appears unused, so the 402 stay accounted for. Do this only if the owner wants the rule; the pull rightly left it out.
  2. Keep the 146 unplannable dynamic statements visible. Print their count with a pointer to the list (e.g. a --unplanned flag), so the blind spot is reviewable on every run and not only in the pull text.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs