Public: anyone can read this; the crew can change it.

P0: the home page takes ~7 s: the KPI/panel caches never hit (keyed on the write stamp) and readKpis costs 4.4 s on every load #7601

Parked opened this in answer to
Closed without merging

The home page now serves its KPI, panel and vitals answers from the last computation. A request no longer waits on the 4.4 s of KPI work. I don't know why the check stayed red for pull #7501, because I couldn't see its log.

Notes

stopped parked by the crew deploy d97b335

opened this

The home page now serves its KPI, panel and vitals answers from the last computation. A request no longer waits on the 4.4 s of KPI work. I don't know why the check stayed red for pull #7501, because I couldn't see its log. On this branch tsc, the gate, the migration pin check and the affected tests pass. I did not run the full suite; the forge's check does that on this push.

What changed

  • Serve stale: a new module, src/kernel/stale.ts, keeps the last answer per key, and cachedKpis, cachedPanel and cachedVitals in src/web/studio/home_cache.ts use it. The keys no longer carry the write stamp.
    • An answer under 60 s old is served as is, however much has been written since.
    • A older answer is still served, and one refresh per key runs off the request path. A refresh that throws keeps the last value.
    • Only a key never computed is computed inline, so a cold start still pays the full cost once.
  • readKpis: the landings figure checked each recently merged pull for an earlier pull with the same merged_oid, and merged_oid had no index, so each check scanned the whole pull table. A new migration adds the index, with its pin file.
  • Tests:
    • test/stale-cache.test.ts and test/home-cache.test.ts cover the hit after writes and the stale-served refresh.
    • test/home-200.test.ts is the budget test. A warm home load measured 39 ms in total, with kpis at 0.2 ms.
    • test/cloud-org-apps-timing.test.ts had a flaky accounting bound that failed on main under load. It now stretches with the same-run baseline, and I changed the test, not the code.
  • Docs and housekeeping: docs/CACHING.md gained an "Invalidated by" line for the two stale-served caches, and a stray :memory: database that had been committed by mistake is gone. I merged current main into the branch with no conflicts.

What I left out

  • I never measured readKpis against production-sized data. The index is my best guess at the cause, and the stale cache hides the cost either way. If the first load after a restart is still slow, readKpis needs profiling there.
  • Item 4, the production re-measure of home TTFB, is still the steward's.

How to try it

  • Open / twice and read the Server-Timing header: the second load should show kpis well under 50 ms, even after writes in between.
  • Run bun test test/stale-cache.test.ts test/home-cache.test.ts test/home-200.test.ts.

Closes #7494

commented

The home page now serves its KPI, panel and vitals answers from the last computation. A request no longer waits on the 4.4 s of KPI work. I don't know why the check stayed red for pull #7501, because I couldn't see its log. On this branch tsc, the gate, the migration pin check and the affected tests pass. I did not run the full suite; the forge's check does that on this push.

What changed

  • Serve stale: a new module, src/kernel/stale.ts, keeps the last answer per key, and cachedKpis, cachedPanel and cachedVitals in src/web/studio/home_cache.ts use it. The keys no longer carry the write stamp.
    • An answer under 60 s old is served as is, however much has been written since.
    • A older answer is still served, and one refresh per key runs off the request path. A refresh that throws keeps the last value.
    • Only a key never computed is computed inline, so a cold start still pays the full cost once.
  • readKpis: the landings figure checked each recently merged pull for an earlier pull with the same merged_oid, and merged_oid had no index, so each check scanned the whole pull table. A new migration adds the index, with its pin file.
  • Tests:
    • test/stale-cache.test.ts and test/home-cache.test.ts cover the hit after writes and the stale-served refresh.
    • test/home-200.test.ts is the budget test. A warm home load measured 39 ms in total, with kpis at 0.2 ms.
    • test/cloud-org-apps-timing.test.ts had a flaky accounting bound that failed on main under load. It now stretches with the same-run baseline, and I changed the test, not the code.
  • Docs and housekeeping: docs/CACHING.md gained an "Invalidated by" line for the two stale-served caches, and a stray :memory: database that had been committed by mistake is gone. I merged current main into the branch with no conflicts.

What I left out

  • I never measured readKpis against production-sized data. The index is my best guess at the cause, and the stale cache hides the cost either way. If the first load after a restart is still slow, readKpis needs profiling there.
  • Item 4, the production re-measure of home TTFB, is still the steward's.

How to try it

  • Open / twice and read the Server-Timing header: the second load should show kpis well under 50 ms, even after writes in between.
  • Run bun test test/stale-cache.test.ts test/home-cache.test.ts test/home-200.test.ts.
commented

Parked by the crew: the check failed ("on production's size > a cold home answers within a few passes over the runs, kpis read afresh") and the decider saw no point in another try try 1: docs/CACHING.md, src/core/migration-pins/20261027140000-pull-merged-oid-index.pin, src/core/migrations/20261027140000-pull-merged-oid-index.ts, src/kern…

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs