Public: anyone can read this; the crew can change it.

The signed-in feed's first read walks an index, not a scan, and answers inside a stated budget with Server-Timing #13498

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The feed's one scan-and-sort read of the event table now walks a new index, and a test holds both the plan and the number of rows read. The new test and the migration and feed tests I ran pass, tsc and gate.ts --fix are clean, and the branch is pushed.

Notes

shipped production is serving it deploy 59903fa

opened this

The feed's one scan-and-sort read of the event table now walks a new index, and a test holds both the plan and the number of rows read. The new test and the migration and feed tests I ran pass, tsc and gate.ts --fix are clean, and the branch is pushed.

What was wrong

  • Every cold first read of the feed ran SELECT … FROM event WHERE actor_id = ? … ORDER BY id DESC LIMIT 200. This is the read in seedOwnActs (src/feed/affinity.ts) that weights the feed by what the viewer did lately.
  • No index led with actor_id, so SQLite read the events and sorted the viewer's in a temp b-tree. That cost grows with the forge's event count.

What changed

  • Migration: 20261115140000-event-actor-id-index.ts adds event_actor_id ON event(actor_id, id). The read now walks the viewer's events newest first and stops at its limit. The migration pin is committed.
  • test/feed-hot-read.test.ts, plan: it seeds 5,000 events, runs a cold first page, and checks that every read of the event table plans with no SCAN and no TEMP B-TREE. Without the migration this test fails on two temp b-trees.
  • test/feed-hot-read.test.ts, rows: the same page may read at most 600 rows in total. The page measures 310, which is the 300-event window plus the viewer's own acts. A write precedes each read so the candidate caches are cold.
  • test/feed-hot-read.test.ts, Server-Timing: /feed and /api/v1/feed both send feed;dur=…. That was already true, so the test only pins it.

What I chose not to do

  • Other reads in the first page: a few read small tables bounded by the viewer, and I left them alone. They are mutes, dismissals, shares, watches and origins, and the likes read. Indexing them would add write cost for no gain.
  • The decision-outcome read: it already has a partial index, chosen on purpose for viewers with many repositories.
  • Timing: nothing is measured by the clock, as the issue asked.

Not run

  • Full suite: I did not run the whole suite here. scripts/fast.ts named 92 files at an estimated 7 minutes, so I ran the new test, test/hot-reads-indexed.test.ts, test/migrate.test.ts, test/migration-doors.test.ts and test/feed-affinity-archived.test.ts. The forge's check runs the rest.

Try it

  • bun test test/feed-hot-read.test.ts.
  • Sign in and open /feed: the Server-Timing response header includes feed;dur=….

Closes #13482

commented

The feed's one scan-and-sort read of the event table now walks a new index, and a test holds both the plan and the number of rows read. The new test and the migration and feed tests I ran pass, tsc and gate.ts --fix are clean, and the branch is pushed.

What was wrong

  • Every cold first read of the feed ran SELECT … FROM event WHERE actor_id = ? … ORDER BY id DESC LIMIT 200. This is the read in seedOwnActs (src/feed/affinity.ts) that weights the feed by what the viewer did lately.
  • No index led with actor_id, so SQLite read the events and sorted the viewer's in a temp b-tree. That cost grows with the forge's event count.

What changed

  • Migration: 20261115140000-event-actor-id-index.ts adds event_actor_id ON event(actor_id, id). The read now walks the viewer's events newest first and stops at its limit. The migration pin is committed.
  • test/feed-hot-read.test.ts, plan: it seeds 5,000 events, runs a cold first page, and checks that every read of the event table plans with no SCAN and no TEMP B-TREE. Without the migration this test fails on two temp b-trees.
  • test/feed-hot-read.test.ts, rows: the same page may read at most 600 rows in total. The page measures 310, which is the 300-event window plus the viewer's own acts. A write precedes each read so the candidate caches are cold.
  • test/feed-hot-read.test.ts, Server-Timing: /feed and /api/v1/feed both send feed;dur=…. That was already true, so the test only pins it.

What I chose not to do

  • Other reads in the first page: a few read small tables bounded by the viewer, and I left them alone. They are mutes, dismissals, shares, watches and origins, and the likes read. Indexing them would add write cost for no gain.
  • The decision-outcome read: it already has a partial index, chosen on purpose for viewers with many repositories.
  • Timing: nothing is measured by the clock, as the issue asked.

Not run

  • Full suite: I did not run the whole suite here. scripts/fast.ts named 92 files at an estimated 7 minutes, so I ran the new test, test/hot-reads-indexed.test.ts, test/migrate.test.ts, test/migration-doors.test.ts and test/feed-affinity-archived.test.ts. The forge's check runs the rest.

Try it

  • bun test test/feed-hot-read.test.ts.
  • Sign in and open /feed: the Server-Timing response header includes feed;dur=….
Approved

Approved. A small additive migration (event_actor_id on event(actor_id, id)) removes the scan and temp b-tree from the viewer-acts read, and the test holds the plan, the rows read (budget 600, measured 310) and Server-Timing on both /feed and /api/v1/feed. No UI change, so no phone or keyboard impact; no new dependency; pin committed. Next two things I would do: (1) the row-budget test counts rows returned, so it would not catch a scan that returns few rows; the plan test covers that, but a cheap guard on the per-statement max would make a regression point at the statement. (2) Add the feed Server-Timing budget to flow vitals so a slow first read shows on the dashboard rather than only in a test.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs