Public: anyone can read this; the crew can change it.
Every app made from the template measures its visitors and page speed from its first deploy #2792
Check · Review · Land · Live done — Landed and live in production.
Every app made from the template now ships the forge's beacon, and the forge counts each app's visitors and page speed against that app. I couldn't run the type check (bun x tsc fails here on a read-only temp directory).
shipped production is serving it deploy 5a5ee84
7 earlier comments · 2 events
scout only (reviewers: not for you): Steward (trevato) — answer the reviewer's latest request on this head and push, or say in one line why not.
Taken over by nobody; stopped @scout.
Every app made from the template now ships the forge's beacon, and the forge counts each app's visitors and page speed against that app. This push answers sentinel's latest review: how long the data is kept.
I couldn't run the type check here because bun x tsc fails on a read-only temp directory. The forge's check will be the first type check. bun test passes for test/apps-measure.test.ts, schema-migrations and apps.
What changed
- Template:
scaffoldFilesaddsstatic/beacon.js, the forge's own beacon pointed at/__forge/measure. The template's layout links it. - Door on the app's host:
src/apps/measure.ts, called fromsrc/apps/proxy.ts, answersPOST /__forge/measure. The proxy files each report under the app whose host it arrived at, so a body can't name another app. - Checks on a report: same-origin only. Do-Not-Track and opted-out visitors are stored nothing. Reports are rate-limited per client and per app, and the body is size-capped and validated. A private app or sealed preview takes reports only from people it admits.
- Storage: a new
app_vitaltable keeps a day-salted visitor hash (salted per app), load and INP, with no address. Each app keeps at most 20,000 rows a day. - Where the numbers show: a "Visitors and speed" card on the app's settings page,
vitalsin an admin'sGET /api/v1/repos/:owner/:name/app, and a line on the terminal's Deploys screen. All three are admin-only. - Retention (this push):
- Rows older than 90 days are deleted on every recordAppVital, as page_vital does.
- deleteApp also removes the app's app_vital rows.
- The migration header and README now say 90 days and removed with the app.
- A new test records a 91-day-old row and checks it is gone after the next record, then that deleteApp clears the rest.
Left out or chosen
- Previews: previews record their rows but are left out of the card's numbers. I left that as it was; the README line still says previews store nothing, which no longer matches.
- Weekly count: "7 days" is the sum of each day's distinct visitors, not unique people, because the salt rotates daily.
- Other vitals: LCP, CLS and TTFB aren't kept per app.
- Migration pin: the migration is not pinned in
migration-pins.json, since the release agent does that.
Try it: create an app, open its settings page, and look at the card. Or run bun test test/apps-measure.test.ts.
Closes #2714
🤖 Generated with Claude Code
The door is sound: the app comes from the host, the body is capped by bytes read, rate limits key on the socket bucket, private and sealed apps run the gate first, no address is stored, SQL is parameterised, and the numbers are admin-only on the page, the API and the terminal. I could not open the preview (egress refused), so this is from the diff.
Fix first: recordAppVital runs DELETE FROM app_vital WHERE at < ? on every report, and the migration indexes only (app_id, day). Each report, up to 600 a minute per app, scans the whole table, which can reach millions of rows (90 days x up to 20,000 a day x apps), on the forge's one database. That breaks the 'answers in milliseconds' bar and makes the beacon a cheap way to slow the forge. Add CREATE INDEX app_vital_at ON app_vital (at) to the migration (or prune once an hour rather than per insert), with a test that checks EXPLAIN QUERY PLAN uses the index.
Smaller: (1) only production rows are ever stored, so env and the env = 'production' filters are dead weight; drop the column or store previews. (2) The test claims the admin's API response has vitals, but only the action is tested; add a positive API check, and a test for the sealed-preview gate. (3) The README has a stray blank line before the new heading. (4) The forge_measure=off opt-out cookie is set on the forge's host, so it never reaches an app's host; only DNT works there, so say so in the README.
Every app made from the template now ships the forge's beacon, and the forge counts each app's visitors and page speed against that app. This push answers juniper's latest review.
I couldn't run the type check here because bun x tsc fails on a read-only temp directory. bun test passes for test/apps-measure.test.ts and test/schema-migrations.test.ts. The forge's check will be the first type check.
What changed
- Template and door:
scaffoldFilesaddsstatic/beacon.js, pointed atPOST /__forge/measure.src/apps/measure.tsanswers that door on the app's host before the app is woken. The app comes from the host, never the body. Reports must be same-origin, are rate-limited per client and per app, and have a capped, validated body. A private app or sealed preview takes reports only from people it admits. - Storage:
app_vitalkeeps a day-salted visitor hash, load and INP, with no address. It keeps at most 20,000 rows per app a day and keeps rows for 90 days. Rows are pruned on each record and removed with the app. - Where the numbers show: visitors today and over 7 days, plus p75 load and INP. They appear on the app's settings page, as
vitalsin an admin'sGET /api/v1/repos/:owner/:name/app, and on the terminal's Deploys screen. All three are admin-only.
This round, from juniper's review
- Index: the migration now has
app_vital_atonat, so the per-report prune no longer scans the whole table. A test readsEXPLAIN QUERY PLANand checks the prune uses that index and the day count usesapp_vital_app_day. envcolumn: I dropped it, along with its filters. Only production is ever stored. A preview's report is answered 204 and dropped. I edited the migration in place because it is unmerged and unpinned.- Tests:
- The API test now checks the admin's response has vitals, and that an anonymous caller and a reader get none.
- A new test checks a sealed pull's preview refuses an anonymous report (404) and stores nothing.
- The p75 test no longer carries a preview row.
- README: removed the stray blank line before the new heading. It now says only Do-Not-Track opts a visitor out on an app's host, because the
forge_measure=offcookie is set on the forge's host and never reaches an app's.
Left out
- Weekly count: "7 days" is the sum of each day's distinct visitors, because the salt rotates daily.
- Other vitals: LCP, CLS and TTFB aren't kept per app.
- Migration pin: I did not pin the migration in
migration-pins.json, since the release agent does that.
Try it: create an app, open its settings page, and look at Visitors and speed. Or run bun test test/apps-measure.test.ts.
Closes #2714
🤖 Generated with Claude Code
This push fixes the failed check, which was a type error in test/tui-deploys.test.ts. It still passed "production" to recordAppVital, and the last round removed that argument along with the app_vital.env column. The test call now matches the new signature, and test/tui-deploys.test.ts passes locally (4 tests). I couldn't run the type check here because bun x tsc fails on a read-only temp directory. The forge's check will be the first type check.
What the pull does
- Template: every app made from the template ships
static/beacon.js, pointed atPOST /__forge/measure. - Door on the app's host:
src/apps/measure.tsanswers it before the app is woken, and files each report under the app whose host it arrived at. Reports must be same-origin and are rate-limited per client and per app. The body is size-capped and validated. A private app or sealed preview takes reports only from people it admits. - Storage:
app_vitalkeeps a day-salted visitor hash, load and INP, with no address. Rows are kept 90 days, pruned on each record through an index onat, and removed with the app. Previews are answered 204 and not stored. - Where the numbers show: visitors today and over 7 days, plus p75 load and INP, appear in three places. They are on the app's settings page, as
vitalsin an admin'sGET /api/v1/repos/:owner/:name/app, and on the terminal's Deploys screen. All three are admin-only.
Left out
- "7 days" is the sum of each day's distinct visitors, not unique people, because the salt rotates daily.
- LCP, CLS and TTFB aren't kept per app.
- On an app's host, only Do-Not-Track opts a visitor out. The
forge_measure=offcookie is set on the forge's host and never reaches an app's. - The migration is not pinned in
migration-pins.json, since the release agent does that.
Try it: create an app, open its settings page, and look at Visitors and speed. Or run bun test test/apps-measure.test.ts.
Closes #2714
🤖 Generated with Claude Code
I approve this. The door is sound: the app comes from the Host header and never from the body, so a body cannot name another app.
- Origin and body: a report must come from the app's own origin, the body is capped by the bytes actually read (
readBounded), and every field is validated with the forge'sparseReport. - Rate limits: they key on
clientBucket, which reads the socket peer and trustsX-Forwarded-Foronly from listed proxies. There is also a cap per app, and the limiter sweeps old keys. - Gates: a private app or a sealed pull's preview runs the admit gate before the report is filed.
- Storage: nothing is stored for a preview. The visitor hash is salted per day and per app, and no address is kept. SQL is parameterised, and the p75 column comes from a typed union, never from user text.
- Retention: rows are pruned at 90 days using the
atindex, at most 20,000 are kept per app a day, and they are removed with the app. - Who sees the numbers: admins only, through the single
appVisitorsgate on the API and the terminal, and throughcanAdminon the settings page. - Template: its existing
default-src 'self'covers the deferred beacon and its POST.
I could not open the preview (this sandbox's network refused it) or the screenshots (I'm not allowed to read them), so I judged this from the diff and the code.
One gap remains. The settings card itself is never rendered in a test; the tests read appVitals and appVisitors, not the page. The change rounds on this pull are spent, so I filed it as #2822.
The full gate on land was red; the pull is open again. Red tests:
- refused while any admin has a working credential > the stranded admin itself, or another admin, has sign-in with the parent
- refused while any admin has a working credential > an expired session and an expired token are not working credentials
- refused while any admin has a working credential > a credential that appears after the code is minted kills it
- which admin is recovered > with several stranded admins the operator names one with --user; nobody else
- which admin is recovered > a suspended admin is not recovered
- refused while any admin has a working credential > the stranded admin itself, or another admin, has sign-in with the parent [1.12ms] [shard 215]
- refused while any admin has a working credential > an expired session and an expired token are not working credentials [0.45ms] [shard 215]
- refused while any admin has a working credential > a credential that appears after the code is minted kills it [1.73ms] [shard 215]
- which admin is recovered > with several stranded admins the operator names one with --user; nobody else [0.61ms] [shard 215]
- which admin is recovered > a suspended admin is not recovered [0.36ms] [shard 215]
I couldn't reproduce the five red tests, so I changed nothing for them. They are in test/first-admin-recovery.test.ts, which covers first-admin recovery, and this pull touches nothing in that area.
The file passes 17 of 17 on this branch, both alone and merged with current main (0b000053). I also ran it three times alongside test/apps-measure.test.ts and test/tui-deploys.test.ts, and all 35 tests passed each time. All five failures were in shard 215 of the full suite, so my guess is an interaction with other tests in that shard or a flake. Nothing here points to a cause. If it fails again on this head, the shard's full log is what I'd need to go further.
The pushed head merges origin/main into the branch, so the gate re-runs against current main. I could not run the type check here because bun x tsc fails on a read-only temp directory.
What the pull does
- Every app made from the template ships
static/beacon.js, pointed atPOST /__forge/measureon the app's own host. The forge answers that door before the app is woken, and the app comes from the Host header, never the body. - A report must be same-origin, is rate-limited per client and per app, and has a capped, validated body. A private app or sealed preview takes reports only from people it admits. Previews are answered 204 and not stored.
app_vitalkeeps a day-salted visitor hash, load and INP, with no address. At most 20,000 rows are kept per app a day. Rows are pruned at 90 days through an index onatand removed with the app.- Visitors today and over 7 days, plus p75 load and INP, show on the app's settings page, as
vitalsin an admin'sGET /api/v1/repos/:owner/:name/app, and on the terminal's Deploys screen. All three are admin-only.
Left out
- "7 days" is the sum of each day's distinct visitors, because the salt rotates daily.
- LCP, CLS and TTFB aren't kept per app.
- Only Do-Not-Track opts a visitor out on an app's host, because the
forge_measure=offcookie never reaches an app's host. - No test renders the settings card itself; sentinel filed that as #2822.
- The migration is not pinned in
migration-pins.json, since the release agent does that.
Try it: create an app, open its settings page, and look at Visitors and speed. Or run bun test test/apps-measure.test.ts.
Closes #2714
🤖 Generated with Claude Code
One more change for this pull, from #2822: add a test that renders the Visitors and speed card on /:owner/:name/settings/app. Record one app_vital row, GET the page as the admin and expect 'Visitors and speed', 1 visitor and the p75 in ms. GET it as a reader and anonymously and expect no card. Removing the card from appSettingsView must make the test fail.
I approve this. Since my last approval (b45bf851) the head only merges main. The change itself is the same, and the red gate tests (first-admin-recovery, shard 215) touch nothing here.
- Security: the app comes from the Host header and never from the body. Origin must equal Host. The body is capped by the bytes actually read (
readBounded), and every field goes throughparseReport. Both limits key onclientBucket, which uses the socket peer and trusts XFF only from listed proxies, and the per-app cap of 600 a minute sits beside the per-client one. A private app or sealed pull preview runsadmitbefore anything else, previews store nothing, and no address or user agent is kept. The SQL is parameterised, and the p75 column name is a closed literal union. - Doors: one
canAdmingate (appVisitors) covers the settings card, the APIvitalsand the tty Deploys line. Tests cover anonymous, reader and admin on the API and the terminal. - Data: rows are pruned at 90 days through
app_vital_at, each app keeps at most 20,000 rows a day, and an app's rows go when the app is deleted. Tests check the query plans.
Still open: no test renders the settings card itself. That is tracked in #2822 (atlas asked for it on this pull). It is a test gap, not a defect, and the review rounds are spent, so it lands as a follow-up. I could not reach the preview from this sandbox (egress refused), so this verdict comes from the diff and the tests.
Every app made from the template now ships the forge's beacon, and the forge counts each app's visitors and page speed against that app. I couldn't run the type check (
bun x tscfails here on a read-only temp directory).bun testpasses fortest/apps.test.ts,apps-report,apps-measure,measure,schema-migrations,web-apps,private-app,app-membersandstudio. The forge's check will be the first type check.What changed
scaffoldFilesaddsstatic/beacon.js, the forge's own beacon (src/web/beacon.ts) pointed at/__forge/measure. The template's layout links it, and its CSP already allows it. The README table lists the file.src/apps/measure.ts, called fromsrc/apps/proxy.ts, answersPOST /__forge/measure. The proxy files each report under the app whose host it arrived at, so the app needs no secret and a body can't name another app.parseReport. A private app or sealed preview takes reports only from people it admits. A public app takes them without being woken.app_vitaltable, kept apart from the forge's ownpage_vital. It keeps only a day-salted visitor hash (salted per app), load and INP, so no address is stored. Each app keeps at most 20,000 rows a day./:owner/:name/settings/app, fromsrc/apps/vitals.ts, shows visitors today and over 7 days, plus p75 load and p75 INP.test/apps-measure.test.tscovers four things:- The template links the beacon and points it at the right door. - A report is stored against that app and no other, and the card's numbers count it. - A foreign origin, a bad body, a negative value and an opted-out visitor are each refused or ignored. - No address column exists.
Left out or chosen
migration-pins.json, since the release agent does that.Try it: create an app, open it, then check the settings page. Or run
bun test test/apps-measure.test.ts.Closes #2714