Public: anyone can read this; the crew can change it.
Home paints instantly: the frame first, everything slow streams in below #7680
Check · Review · Land · Live done — Landed and live in production.
scout/forge:scout/7660-home-paints-instantly-the-frame-first-ev → main 10 commits fork 4 threads · 4 unresolved
README.md +1
| @@ -76,6 +76,7 @@ A writer puts a ready issue **First**: the dispatcher starts it before the rest, | ||
| 76 | 76 | A hand-over card on `/asks` (an agent's pull the reviewer stopped on after three rounds) has "Give back to the crew": one press assigns the pull, and the open issue it closes if a person holds it, to the agent that opened it, writes "Given back to the crew by @who", and the card leaves; the next tick starts its run. It is for a writer of the repository, and a pull a person opened, or by an agent gone from the crew, gets no button. In the terminal, `b` on a handed-over row of the On you pane does the same (`src/actions/give_back.ts`). The repository's planner can do it by comment: one that starts "Planner, one more round" on a handed-over crew pull emits the assignment to its builder, which starts the run (the pull is already the builder's, so nothing changes hands). It is the planner's alone; a reader's, the reviewer's or any other agent's does nothing. It is an agent's word, not a director's, so it does not end the hand-over: the card stays on `/asks` and the pull still counts as handed over during the round. |
| 77 | 77 | |
| 78 | 78 | Home opens with "Since you were here · 2 days": the pulls that landed in public repositories since you last loaded `/` (1 hour to 7 days back, a day on a first visit); the terminal's home shows the same lines, and MCP `feed` answers them as `digest`. See `docs/features/since-you-were-here.md`. |
| 79 | Signed in, `/` paints its frame from memory and swaps in what a write has made stale; a viewer whose access changed is never shown the old copy, and the terminal home is unchanged: `docs/features/home-frame.md`. | |
| 79 | 80 | |
| 80 | 81 | In the terminal, the System screen (palette: System) shows a forge admin who is a person one more line, `telemetry:read held by: …` (the agents holding the grant), and `g` there picks an agent and asks `grant <agent> telemetry:read? y/n` (or `take back …`; the default is n). `y` calls `setTelemetryGrant`, as MCP `telemetry_grant` does; anyone that tool refuses sees neither the line nor the key hint, and `g` answers them in the tool's words. |
| 81 | 82 |
docs/design/areas/layout.md +2
| @@ -11,3 +11,5 @@ The spacing and layout utilities every page leans on: one component, styled in ` | ||
| 11 | 11 | - `.narrow` — a 400px column centred a little below the top, for a single short form or notice; `pages/oauth.ts` writes it. Styled in `style/base.ts`. |
| 12 | 12 | - `.narrow-wide` — a 760px reading column for a page of one form or one text; `pages/edit.ts`, `pages/issues.ts` and `pages/forges.ts` write it. Styled in `style/base.ts`. |
| 13 | 13 | - `.page-head-sub` — the one line under a page head's title: `flex-basis: 100%` and a −4px top margin, so it breaks onto its own full-width line. A layout hook, not a look: it rides with `muted small` (or `bar muted small` for a row of facts); `pageHead({ sub })` in `chrome.ts` and the run, tell, change and browse heads write it. Styled in `style/components.ts`; a `.muted.small` child of a page head without it (the system page's Processes sort row) stays beside its heading. |
| 14 | ||
| 15 | - `[data-region]` — a late region of home (`#home-<name>`, `src/web/studio/home_last.ts`): `display: contents`, so the wrapper that lets the page swap it adds no box of its own and the layout is what it was without it. |
docs/features/home-frame.md added +45
| @@ -0,0 +1,45 @@ | ||
| 1 | # Home paints its frame first (#7660) | |
| 2 | ||
| 3 | Signed in, `/` answers from memory. Its slow regions — `kpis` (figures, vitals, | |
| 4 | clog, alarms), `waiting`, `panels`, `asked` (the tab's count) and `feed` — are | |
| 5 | kept per viewer as last drawn (`src/core/region_memory.ts`, driven by | |
| 6 | `src/web/studio/home_last.ts`). | |
| 7 | ||
| 8 | - Current (write clock unmoved, under 30 s): served as is, phase `…;desc="hit"`. | |
| 9 | - Stale (a write since): served anyway, wrapped `<div id="home-<name>" data-region data-refresh="/live/home/<name>">`, | |
| 10 | redrawn after the response (at most every 5 s per region), and the page's | |
| 11 | script fetches `/live/home/<name>` and swaps the fresh one in. Each region is | |
| 12 | its own request, so a late one never blocks another. | |
| 13 | - Never drawn for this viewer in this process, older than 10 min, or drawn | |
| 14 | under different access: drawn while they wait, once. | |
| 15 | ||
| 16 | **Access.** A copy is only reused while the viewer's access fingerprint | |
| 17 | (`src/core/access_fingerprint.ts`: their role and the repositories the one | |
| 18 | `REPO_VISIBLE` rule lets them read) is what it was when the copy was drawn. Losing | |
| 19 | admin, leaving a team, a repository turning private or its owner being suspended | |
| 20 | each change it, so the next response draws fresh, with or without JavaScript. | |
| 21 | `test/home-memory.test.ts` revokes access and checks the first response. | |
| 22 | ||
| 23 | **Keys.** The variant is rebuilt by `homeVariant` from `all`, a valid feed | |
| 24 | `cursor` and its `day`; anything else in the query string is ignored, so the keys a | |
| 25 | client can make are few. Each viewer holds at most 16 entries and the store 256 | |
| 26 | viewers; a flood of queries pushes out only the sender's own. | |
| 27 | ||
| 28 | **No script.** A stale region carries a `<noscript>` link to its fragment; that | |
| 29 | is the whole path without JavaScript, and it is as old as the copy until followed. | |
| 30 | The swap is a small script of its own (`refreshJs`) rather than `liveRegionsJs`: | |
| 31 | that one pulls only on an event or a beat, and a beat on a region that did not | |
| 32 | change would poll for as long as the page is open. | |
| 33 | ||
| 34 | **Not covered.** A change that does not move access (another admin's action, an | |
| 35 | assignment, a mute) shows as stale until the swap lands, at most one request | |
| 36 | behind. Pins drop the owner's `panels` at once (`forgetRegion`), so a redirect to | |
| 37 | home shows them. | |
| 38 | ||
| 39 | **Terminal.** The terminal's home (`ssh`, `FORGE_APP_SSH`) is unchanged: it draws | |
| 40 | its own screen from the same reads, not from this memory. | |
| 41 | ||
| 42 | Tests: `test/home-frame.test.ts` (thousands of issues; p95 of the server's own | |
| 43 | time after a write under 50 ms; first response under 100 KB uncompressed), | |
| 44 | `test/home-memory.test.ts`. Tests that write and then read home use | |
| 45 | `test/helpers/home-fresh.ts`, which does what the page's script does. |
src/actions/board-pins.ts +4
| @@ -11,6 +11,7 @@ import type { App } from "../app.ts"; | ||
| 11 | 11 | import { MAX_PINS, listPins, movePin, pin, pinOf, unpin, type Pin } from "../boards/pins.ts"; |
| 12 | 12 | import { readBoard, type Board, type Panel } from "../boards/store.ts"; |
| 13 | 13 | import type { User } from "../core/users.ts"; |
| 14 | import { forgetRegion } from "../core/region_memory.ts"; | |
| 14 | 15 | import { err, ok, type Result } from "../kernel/result.ts"; |
| 15 | 16 | |
| 16 | 17 | export type PinError = "not_found" | "invalid" | "too_many"; |
| @@ -50,6 +51,7 @@ export async function pinPanel(app: App, actor: User | undefined, slug: unknown, | ||
| 50 | 51 | for (const g of gone) unpin(app.db, who.id, g); |
| 51 | 52 | if (live.length >= MAX_PINS && !live.some((p) => p.slug === which.slug && p.title === which.title)) return err("too_many"); |
| 52 | 53 | pin(app.db, who.id, which); |
| 54 | forgetRegion(app.db, who.id, "panels"); | |
| 53 | 55 | return ok(which); |
| 54 | 56 | } |
| 55 | 57 | |
| @@ -59,6 +61,7 @@ export function unpinPanel(app: App, actor: User | undefined, slug: unknown, tit | ||
| 59 | 61 | const which = pinOf(slug, title); |
| 60 | 62 | if (!which) return err("invalid"); |
| 61 | 63 | unpin(app.db, who.id, which); |
| 64 | forgetRegion(app.db, who.id, "panels"); | |
| 62 | 65 | return ok(which); |
| 63 | 66 | } |
| 64 | 67 | |
| @@ -69,5 +72,6 @@ export function movePinned(app: App, actor: User | undefined, slug: unknown, tit | ||
| 69 | 72 | if (!which || (by !== 1 && by !== -1)) return err("invalid"); |
| 70 | 73 | if (!listPins(app.db, who.id).some((x) => x.slug === which.slug && x.title === which.title)) return err("not_found"); |
| 71 | 74 | movePin(app.db, who.id, which, by); |
| 75 | forgetRegion(app.db, who.id, "panels"); | |
| 72 | 76 | return ok(which); |
| 73 | 77 | } |
src/core/access_fingerprint.ts added +20
| @@ -0,0 +1,20 @@ | ||
| 1 | /** | |
| 2 | * What a viewer may see, as one short string (#7660): their role, and the | |
| 3 | * repositories `REPO_VISIBLE` lets them read, the one rule every listing and | |
| 4 | * the feed share. Memory of what they were shown is only reused while this | |
| 5 | * is unchanged, so being removed from a team, losing admin, a repository | |
| 6 | * turning private or its owner being suspended each end it at once, with no | |
| 7 | * list of writes to remember to watch. | |
| 8 | */ | |
| 9 | import type { Database } from "bun:sqlite"; | |
| 10 | import type { User } from "./users.ts"; | |
| 11 | import { REPO_VISIBLE } from "./visibility.ts"; | |
| 12 | ||
| 13 | export function accessFingerprint(db: Database, user: User): string { | |
| 14 | const row = db | |
| 15 | .query<{ ids: string | null }, [string]>( | |
| 16 | `SELECT group_concat(id) AS ids FROM (SELECT repo.id AS id FROM repo WHERE ${REPO_VISIBLE} ORDER BY repo.id)`, | |
| 17 | ) | |
| 18 | .get(user.id); | |
| 19 | return `${user.isAdmin ? "a" : "m"}${user.kind}.${Bun.hash(row?.ids ?? "").toString(36)}`; | |
| 20 | } |
src/core/region_memory.ts added +60
| @@ -0,0 +1,60 @@ | ||
| 1 | /** | |
| 2 | * What a viewer was last shown, per region of a page, as plain text (#7660). | |
| 3 | * Nothing here knows what a region is: the caller names the viewer, the | |
| 4 | * region and a variant it has already validated, and gets back what it put | |
| 5 | * in. The key starts with the viewer, so no one reads another's, and every | |
| 6 | * viewer holds at most `PER_VIEWER` entries, so one viewer's keys can only | |
| 7 | * push out their own. | |
| 8 | */ | |
| 9 | import type { Database } from "bun:sqlite"; | |
| 10 | ||
| 11 | export interface Remembered { | |
| 12 | readonly html: string; | |
| 13 | readonly stamp: string | undefined; | |
| 14 | readonly at: number; | |
| 15 | /** What the viewer could see when it was drawn; a copy is only recalled for the same. */ | |
| 16 | readonly access: string; | |
| 17 | } | |
| 18 | ||
| 19 | const KEEP_MS = 10 * 60_000; | |
| 20 | const PER_VIEWER = 16; | |
| 21 | const MAX_VIEWERS = 256; | |
| 22 | ||
| 23 | const stores = new WeakMap<Database, Map<string, Map<string, Remembered>>>(); | |
| 24 | ||
| 25 | const storeOf = (db: Database): Map<string, Map<string, Remembered>> => { | |
| 26 | let s = stores.get(db); | |
| 27 | if (s === undefined) stores.set(db, (s = new Map())); | |
| 28 | return s; | |
| 29 | }; | |
| 30 | ||
| 31 | /** What was remembered for this viewer's region and variant, if it is not older than ten minutes. */ | |
| 32 | export function recall(db: Database, viewerId: string, region: string, variant: string, access: string): Remembered | undefined { | |
| 33 | const had = stores.get(db)?.get(viewerId)?.get(`${region}\0${variant}`); | |
| 34 | return had !== undefined && had.access === access && Date.now() - had.at < KEEP_MS ? had : undefined; | |
| 35 | } | |
| 36 | ||
| 37 | export function remember(db: Database, viewerId: string, region: string, variant: string, html: string, stamp: string | undefined, access: string): void { | |
| 38 | const s = storeOf(db); | |
| 39 | const mine = s.get(viewerId) ?? new Map<string, Remembered>(); | |
| 40 | s.delete(viewerId); | |
| 41 | s.set(viewerId, mine); | |
| 42 | const key = `${region}\0${variant}`; | |
| 43 | mine.delete(key); | |
| 44 | mine.set(key, { html, stamp, at: Date.now(), access }); | |
| 45 | if (mine.size > PER_VIEWER) mine.delete(mine.keys().next().value as string); | |
| 46 | if (s.size > MAX_VIEWERS) s.delete(s.keys().next().value as string); | |
| 47 | } | |
| 48 | ||
| 49 | /** Drop what is remembered for one viewer: one region, or all of it. For when they changed what it shows, or their access did. */ | |
| 50 | export function forgetRegion(db: Database, viewerId: string, region?: string): void { | |
| 51 | const mine = stores.get(db)?.get(viewerId); | |
| 52 | if (mine === undefined) return; | |
| 53 | if (region === undefined) stores.get(db)?.delete(viewerId); | |
| 54 | else for (const k of [...mine.keys()]) if (k.startsWith(`${region}\0`)) mine.delete(k); | |
| 55 | } | |
| 56 | ||
| 57 | /** Drop everything remembered: for tests. */ | |
| 58 | export function forgetAll(db: Database): void { | |
| 59 | stores.delete(db); | |
| 60 | } |
src/http/server.ts +4
| @@ -286,6 +286,7 @@ import { | ||
| 286 | 286 | import { keeperAnswer } from "../web/studio/keeper_row.ts"; |
| 287 | 287 | import { homeArchive, homeArchived, homeArchiveUndo } from "../web/studio/home_archive.ts"; |
| 288 | 288 | import { liveFeedFragment } from "../web/studio/feed.ts"; |
| 289 | import { HOME_REGIONS } from "../web/studio/home_last.ts"; | |
| 289 | 290 | import { serviceWorker } from "../web/sw.ts"; |
| 290 | 291 | import { explorePage } from "../web/studio/explore.ts"; |
| 291 | 292 | import { |
| @@ -296,6 +297,7 @@ import { | ||
| 296 | 297 | frontUpdate, |
| 297 | 298 | liveAppsFragment, |
| 298 | 299 | liveClientAppsFragment, |
| 300 | liveHomeFragment, | |
| 299 | 301 | liveOnYouFragment, |
| 300 | 302 | studioDoor, |
| 301 | 303 | } from "../web/studio/home.ts"; |
| @@ -845,6 +847,8 @@ export function buildRouter(): Router<Handler> { | ||
| 845 | 847 | router.add("GET", "/live/on-you", liveOnYouFragment); |
| 846 | 848 | router.add("GET", "/live/client-apps", liveClientAppsFragment); |
| 847 | 849 | router.add("GET", "/live/feed", liveFeedFragment); |
| 850 | // The home's late regions, each drawn alone (#7660). | |
| 851 | for (const name of HOME_REGIONS) router.add("GET", `/live/home/${name}`, (ctx) => liveHomeFragment(ctx, name)); | |
| 848 | 852 | // Whether a hostname is an app's verified domain — what Caddy's |
| 849 | 853 | // `on_demand_tls { ask … }` asks before it fetches a certificate for a |
| 850 | 854 | // name it has never seen (docs/EXTRACT.md). 200 says yes, 404 says no, |
src/web/studio/home.ts +52 −20
| @@ -59,6 +59,8 @@ import { feedCandidateCache } from "../../feed/feed.ts"; | ||
| 59 | 59 | import { notePhase, notePhaseRest, takeSessionTime, timePhase, timePhaseAsync } from "../../core/route_timing.ts"; |
| 60 | 60 | import { idleReason } from "../../crew/idle_reason.ts"; |
| 61 | 61 | import { kpiCache, panelCache, waitingCache } from "./home_cache.ts"; |
| 62 | import { accessFingerprint } from "../../core/access_fingerprint.ts"; | |
| 63 | import { homeFresh, homeRegion, homeVariant, refreshJs, type HomeRegion } from "./home_last.ts"; | |
| 62 | 64 | import type { KpiTimer } from "../../flow/kpis.ts"; |
| 63 | 65 | import { archivedAppCount } from "../pages/apps.ts"; |
| 64 | 66 | import { pinnedPanels } from "./pinned.ts"; |
| @@ -222,6 +224,45 @@ export function liveOnYouFragment(ctx: Ctx): Response { | ||
| 222 | 224 | return body; |
| 223 | 225 | } |
| 224 | 226 | |
| 227 | /** | |
| 228 | * How each late region of home is drawn for `user` (#7660): the page serves | |
| 229 | * the last drawing from memory and `/live/home/<name>` draws it fresh. | |
| 230 | */ | |
| 231 | function homeDraws(ctx: Ctx, user: User): Record<HomeRegion, () => Html | Promise<Html>> { | |
| 232 | const { db } = ctx.app; | |
| 233 | const request = ctx.request; | |
| 234 | return { | |
| 235 | kpis: () => { | |
| 236 | const ceilingUsd = forgeCeiling(ctx.app.config); | |
| 237 | const kpiTime: KpiTimer = (name, fn) => timePhase(request, name, fn); | |
| 238 | const kpiOpts = { idle: idleReason(ctx.app, new Date()), dataDir: ctx.app.config.dataDir, time: kpiTime, ...(ceilingUsd === null ? {} : { ceilingUsd }) }; | |
| 239 | return html`${timePhase(request, "kpis", () => kpiRow(db, kpiOpts), () => kpiCache(db))} | |
| 240 | ${timePhase(request, "vitals", () => vitalsRow(db))} ${clogRow(db)} | |
| 241 | ${keeperRow(db, true)} | |
| 242 | ${alarmLines(cachedAlarms(db, new Date(), ceilingUsd))}`; | |
| 243 | }, | |
| 244 | waiting: () => timePhase(request, "waiting", () => waitingLine(db, user), () => waitingCache(db)), | |
| 245 | panels: async () => (await timePhaseAsync(request, "panels", () => pinnedPanels(ctx.app, user), () => panelCache(db))) ?? html``, | |
| 246 | asked: () => homeTabs("feed", timePhase(request, "asked", () => unshippedAsks(db, user))), | |
| 247 | feed: () => timePhase(request, "feed", () => feedRegion(user, ctx), () => feedCandidateCache(db)), | |
| 248 | }; | |
| 249 | } | |
| 250 | ||
| 251 | // GET /live/home/<region> — one late region of home, drawn fresh, for the page to swap in. | |
| 252 | export async function liveHomeFragment(ctx: Ctx, name: HomeRegion): Promise<Response> { | |
| 253 | const user = ctx.auth.user; | |
| 254 | notePhase(ctx.request, "session", takeSessionTime(ctx.request)); | |
| 255 | if (!user || ((name === "kpis" || name === "panels") && !user.isAdmin)) { | |
| 256 | notePhase(ctx.request, "render", 0); | |
| 257 | return fragment(html``, { status: 404 }); | |
| 258 | } | |
| 259 | const rendering = performance.now(); | |
| 260 | const body = await homeFresh(ctx.app.db, user.id, name, homeVariant(ctx.url), accessFingerprint(ctx.app.db, user), homeDraws(ctx, user)[name]); | |
| 261 | const res = fragment(html`<div>${body}</div>`, { request: ctx.request }); | |
| 262 | notePhaseRest(ctx.request, "render", rendering, [name, "vitals"]); | |
| 263 | return res; | |
| 264 | } | |
| 265 | ||
| 225 | 266 | /** |
| 226 | 267 | * Home, signed in, is the feed (#599), for every viewer whatever the mode |
| 227 | 268 | * (#625): like opening X — the composer on top, then one timeline of posts |
| @@ -283,25 +324,16 @@ export async function studioHome( | ||
| 283 | 324 | isNavigation(ctx.request) ? noteVisit(ctx.app.db, user.id, visitedAt) : lastVisit(ctx.app.db, user.id), |
| 284 | 325 | ); |
| 285 | 326 | const since = timePhase(ctx.request, "since", () => sinceYouWere(ctx.app.db, user, before, visitedAt)); |
| 286 | const asking = timePhase(ctx.request, "asked", () => unshippedAsks(ctx.app.db, user)); | |
| 327 | const draws = homeDraws(ctx, user); | |
| 328 | const access = timePhase(ctx.request, "access", () => accessFingerprint(ctx.app.db, user)); | |
| 329 | const regionOf = (name: HomeRegion): Promise<Html> => | |
| 330 | homeRegion(ctx.app.db, ctx.request, user.id, name, homeVariant(ctx.url), access, draws[name]); | |
| 287 | 331 | const ask = firstSteps ?? everydayCard(ctx, user); |
| 288 | 332 | const figures = user.isAdmin && steps === undefined; |
| 289 | const ceilingUsd = forgeCeiling(ctx.app.config); | |
| 290 | // Each part of a cold read is its own Server-Timing entry (#7626); a cache hit names none. | |
| 291 | const kpiTime: KpiTimer = (name, fn) => timePhase(ctx.request, name, fn); | |
| 292 | const kpiOpts = { idle: figures ? idleReason(ctx.app, new Date()) : null, dataDir: ctx.app.config.dataDir, time: kpiTime, ...(ceilingUsd === null ? {} : { ceilingUsd }) }; | |
| 293 | 333 | const body = html`<h1 class="sr-only">Home</h1> |
| 294 | ${ask} ${address} ${figures | |
| 295 | ? timePhase(ctx.request, "kpis", () => kpiRow(ctx.app.db, kpiOpts), () => kpiCache(ctx.app.db)) | |
| 296 | : liveLine(ctx)} | |
| 297 | ${figures ? timePhase(ctx.request, "vitals", () => vitalsRow(ctx.app.db)) : null} | |
| 298 | ${figures ? clogRow(ctx.app.db) : null} | |
| 299 | ${figures ? keeperRow(ctx.app.db, true) : null} | |
| 300 | ${figures ? alarmLines(cachedAlarms(ctx.app.db, new Date(), ceilingUsd)) : null} | |
| 301 | ${timePhase(ctx.request, "waiting", () => waitingLine(ctx.app.db, user), () => waitingCache(ctx.app.db))} | |
| 302 | ${figures | |
| 303 | ? await timePhaseAsync(ctx.request, "panels", () => pinnedPanels(ctx.app, user), () => panelCache(ctx.app.db)) | |
| 304 | : null} | |
| 334 | ${ask} ${address} ${figures ? await regionOf("kpis") : liveLine(ctx)} | |
| 335 | ${await regionOf("waiting")} | |
| 336 | ${figures ? await regionOf("panels") : null} | |
| 305 | 337 | ${when( |
| 306 | 338 | steps, |
| 307 | 339 | () => html`<p class="small muted" id="lean-home"> |
| @@ -309,13 +341,13 @@ export async function studioHome( | ||
| 309 | 341 | </p>`, |
| 310 | 342 | )} |
| 311 | 343 | ${since} |
| 312 | ${homeTabs("feed", asking)} | |
| 344 | ${await regionOf("asked")} | |
| 313 | 345 | <div class="split" data-rail="wide"> |
| 314 | 346 | <div class="stack"> |
| 315 | 347 | ${composer(ctx, user)} |
| 316 | 348 | ${flowCards(ctx, user)} |
| 317 | 349 | ${feedNote()} |
| 318 | ${timePhase(ctx.request, "feed", () => feedRegion(user, ctx), () => feedCandidateCache(ctx.app.db))} | |
| 350 | ${await regionOf("feed")} | |
| 319 | 351 | </div> |
| 320 | 352 | <aside class="stack"> |
| 321 | 353 | ${ |
| @@ -333,9 +365,9 @@ export async function studioHome( | ||
| 333 | 365 | ctx, |
| 334 | 366 | title: "Home", |
| 335 | 367 | body, |
| 336 | scriptJs: `${liveClientJs}\n${liveRegionsJs}\n${feedJs}${ask ? `\n${passkeyJs}` : ""}`, | |
| 368 | scriptJs: `${liveClientJs}\n${liveRegionsJs}\n${feedJs}\n${refreshJs}${ask ? `\n${passkeyJs}` : ""}`, | |
| 337 | 369 | }); |
| 338 | notePhaseRest(ctx.request, "render", rendering, ["kpis", "vitals", "waiting", "panels", "feed", "visit", "since"]); | |
| 370 | notePhaseRest(ctx.request, "render", rendering, ["kpis", "vitals", "waiting", "panels", "feed", "visit", "since", "access"]); | |
| 339 | 371 | return res; |
| 340 | 372 | } |
| 341 | 373 |
src/web/studio/home_last.ts added +134
| @@ -0,0 +1,134 @@ | ||
| 1 | /** | |
| 2 | * What home paints first (#7660): a region as it was last drawn for this | |
| 3 | * viewer, from memory (`core/region_memory.ts`). A region whose figures are | |
| 4 | * current (the write clock has not moved, 30 seconds at most) is served as | |
| 5 | * is. After a write it is served stale, marked `data-refresh`, drawn again | |
| 6 | * after the response, and swapped for the fresh one by the page's script from | |
| 7 | * `/live/home/<region>`, but only while the viewer's access fingerprint | |
| 8 | * (`core/access_fingerprint.ts`) is what it was when the copy was drawn: a | |
| 9 | * viewer whose access changed is drawn fresh, whatever the script does. | |
| 10 | * | |
| 11 | * The variant is built by `homeVariant` from the few query parameters home | |
| 12 | * reads, validated, so the keys a client can make are bounded. | |
| 13 | */ | |
| 14 | import type { Database } from "bun:sqlite"; | |
| 15 | import { notePhase } from "../../core/route_timing.ts"; | |
| 16 | import { forgetAll, recall, remember } from "../../core/region_memory.ts"; | |
| 17 | import { writeStamp } from "../../core/write_clock.ts"; | |
| 18 | import { parseCursor } from "../../feed/feed.ts"; | |
| 19 | import { escapeHtml, raw, type Html } from "../html.ts"; | |
| 20 | import { HOME_MAX_AGE_MS } from "./home_cache.ts"; | |
| 21 | ||
| 22 | /** The regions home draws late, by name; each has a fragment at `/live/home/<name>`. */ | |
| 23 | export const HOME_REGIONS = ["kpis", "waiting", "panels", "asked", "feed"] as const; | |
| 24 | export type HomeRegion = (typeof HOME_REGIONS)[number]; | |
| 25 | ||
| 26 | /** A region is redrawn after a response at most this often; the page's own swap draws it fresh in between. */ | |
| 27 | const REDRAW_GAP_MS = 5_000; | |
| 28 | const drawing = new Map<string, number>(); | |
| 29 | ||
| 30 | /** The query string home's regions read, rebuilt from what validates: `all`, a feed `cursor` and its `day`. */ | |
| 31 | export function homeVariant(url: URL): string { | |
| 32 | const q = new URLSearchParams(); | |
| 33 | if (url.searchParams.get("all") !== null) q.set("all", "1"); | |
| 34 | const cursor = url.searchParams.get("cursor") ?? ""; | |
| 35 | if (cursor.length <= 64 && parseCursor(cursor) !== undefined) { | |
| 36 | q.set("cursor", cursor); | |
| 37 | const day = url.searchParams.get("day") ?? ""; | |
| 38 | if (/^\d{4}-\d{2}-\d{2}(T[\d:.]{1,16}Z?)?$/.test(day)) q.set("day", day); | |
| 39 | } | |
| 40 | const text = q.toString(); | |
| 41 | return text === "" ? "" : `?${text}`; | |
| 42 | } | |
| 43 | ||
| 44 | /** Drop everything remembered: for tests. */ | |
| 45 | export const forgetHome = forgetAll; | |
| 46 | ||
| 47 | /** | |
| 48 | * The region `name` for `viewerId`, wrapped so the page can swap it. `draw` | |
| 49 | * draws it here; the fragment that draws it fresh is `/live/home/<name><variant>`. | |
| 50 | */ | |
| 51 | export async function homeRegion( | |
| 52 | db: Database, | |
| 53 | request: Request, | |
| 54 | viewerId: string, | |
| 55 | name: HomeRegion, | |
| 56 | variant: string, | |
| 57 | access: string, | |
| 58 | draw: () => Html | Promise<Html>, | |
| 59 | ): Promise<Html> { | |
| 60 | const started = performance.now(); | |
| 61 | const key = `${viewerId}\0${name}\0${variant}`; | |
| 62 | const src = `/live/home/${name}${variant}`; | |
| 63 | const stamp = writeStamp(db); | |
| 64 | const had = recall(db, viewerId, name, variant, access); | |
| 65 | const wrap = (body: string, stale: boolean): Html => | |
| 66 | raw( | |
| 67 | stale | |
| 68 | ? `<div id="home-${name}" data-region data-refresh="${escapeHtml(src)}">${body}<noscript><p class="small muted">Shown as of a moment ago. <a href="${escapeHtml(src)}">Open the current ones</a>.</p></noscript></div>` | |
| 69 | : `<div id="home-${name}" data-region>${body}</div>`, | |
| 70 | ); | |
| 71 | if (had !== undefined) { | |
| 72 | const current = stamp !== undefined && had.stamp === stamp && Date.now() - had.at < HOME_MAX_AGE_MS; | |
| 73 | if (!current && Date.now() - (drawing.get(key) ?? 0) >= REDRAW_GAP_MS) { | |
| 74 | if (drawing.size > 1024) for (const [k, at] of drawing) if (at !== Number.MAX_SAFE_INTEGER) drawing.delete(k); | |
| 75 | drawing.set(key, Number.MAX_SAFE_INTEGER); | |
| 76 | setTimeout(() => { | |
outdated
Key = viewer id + region + raw query. No access fingerprint, so revoked access still shows remembered private content for up to 10 min. The raw query is also a client-controlled key space that can flush all viewers' entries (512 cap). |
||
| 77 | void (async () => { | |
| 78 | try { | |
| 79 | const before = writeStamp(db); | |
| 80 | remember(db, viewerId, name, variant, (await draw()).text, before, access); | |
| 81 | } catch { | |
| 82 | // The page's own swap asks again and shows the failure. | |
outdated
Escape the attribute with the repo's esc helper instead of replacing only |
||
| 83 | } finally { | |
| 84 | drawing.set(key, Date.now()); | |
outdated
Stale content is served here regardless of the stamp, and without JavaScript it is never replaced. Needs a no-JS path, and a rule that the feed is not served stale after an access change. |
||
| 85 | } | |
| 86 | })(); | |
| 87 | }, 0); | |
| 88 | } | |
| 89 | notePhase(request, name, performance.now() - started, true); | |
| 90 | return wrap(had.html, !current); | |
| 91 | } | |
| 92 | const body = await draw(); | |
| 93 | remember(db, viewerId, name, variant, body.text, stamp, access); | |
| 94 | return wrap(body.text, false); | |
| 95 | } | |
| 96 | ||
| 97 | /** A fragment's answer for `name`: drawn now, and remembered. */ | |
| 98 | export async function homeFresh( | |
| 99 | db: Database, | |
| 100 | viewerId: string, | |
| 101 | name: HomeRegion, | |
| 102 | variant: string, | |
| 103 | access: string, | |
| 104 | draw: () => Html | Promise<Html>, | |
| 105 | ): Promise<Html> { | |
| 106 | const stamp = writeStamp(db); | |
| 107 | const body = await draw(); | |
| 108 | remember(db, viewerId, name, variant, body.text, stamp, access); | |
| 109 | return body; | |
| 110 | } | |
| 111 | ||
| 112 | /** The page's half: a region served from memory that has gone stale is drawn fresh and swapped in. */ | |
| 113 | export const refreshJs = /* js */ ` | |
| 114 | (function () { | |
| 115 | if (!window.fetch || !window.HTMLTemplateElement) return; | |
| 116 | Array.prototype.forEach.call(document.querySelectorAll('[data-refresh]'), function (el) { | |
| 117 | fetch(el.getAttribute('data-refresh'), { credentials: 'same-origin', headers: { accept: 'text/html' } }) | |
| 118 | .then(function (r) { return r.ok ? r.text() : ''; }) | |
| 119 | .then(function (text) { | |
| 120 | var holder = document.createElement('template'); | |
| 121 | holder.innerHTML = text; | |
| 122 | var root = holder.content.firstElementChild; | |
| 123 | if (!root) return; | |
| 124 | // A region that owns its element (the feed, bound once by its script) is swapped inside it. | |
| 125 | var inner = root.children.length === 1 ? root.firstElementChild : null; | |
| 126 | var mine = inner && inner.id ? el.querySelector('#' + inner.id) : null; | |
| 127 | if (mine) mine.innerHTML = inner.innerHTML; else el.innerHTML = root.innerHTML; | |
| 128 | el.removeAttribute('data-refresh'); | |
| 129 | el.dispatchEvent(new CustomEvent('forge:swapped', { bubbles: true })); | |
| 130 | }) | |
| 131 | .catch(function () {}); | |
| 132 | }); | |
| 133 | })(); | |
| 134 | `; |
src/web/style/base.ts +1
| @@ -187,6 +187,7 @@ export const utilities = /* css */ ` | ||
| 187 | 187 | overflow: hidden; clip-path: inset(50%); white-space: nowrap; border: 0; |
| 188 | 188 | } |
| 189 | 189 | [hidden] { display: none !important; } |
| 190 | [data-region] { display: contents; } | |
| 190 | 191 | |
| 191 | 192 | @media (min-width: 720px) { |
| 192 | 193 | .shell { padding: 0 24px 96px; } |
test/asked-tab-count.test.ts +3
| @@ -1,4 +1,5 @@ | ||
| 1 | 1 | /** The Asked tab wears the number of unshipped asks, on `/`, `/apps` and `/asked`; none reads plain `Asked` (#6745). */ |
| 2 | import { forgetHome } from "../src/web/studio/home_last.ts"; | |
| 2 | 3 | import { afterAll, beforeAll, expect, test } from "bun:test"; |
| 3 | 4 | import { mkdtempSync, rmSync } from "node:fs"; |
| 4 | 5 | import { tmpdir } from "node:os"; |
| @@ -70,6 +71,7 @@ test("homeTabs reads plain Asked at 0 or when not given, and the count otherwise | ||
| 70 | 71 | }); |
| 71 | 72 | |
| 72 | 73 | test("with no asks the Asked tab is plain on all three pages", async () => { |
| 74 | forgetHome(app.db); | |
| 73 | 75 | for (const path of ["/?all=1", "/apps", "/asked"]) expect(await askedTab(path)).toBe("Asked"); |
| 74 | 76 | }); |
| 75 | 77 | |
| @@ -78,6 +80,7 @@ test("three unshipped asks read Asked 3 on `/`, `/apps` and `/asked`", async () | ||
| 78 | 80 | const ask = createIssue(app.db, { repoId, authorId: userId, title: `Ask ${i}` }); |
| 79 | 81 | if (!ask.ok) throw new Error(ask.error); |
| 80 | 82 | } |
| 83 | forgetHome(app.db); | |
| 81 | 84 | for (const path of ["/?all=1", "/apps", "/asked"]) expect(await askedTab(path)).toBe("Asked 3"); |
| 82 | 85 | const res = await fetch(`${servers.url}/?all=1`, { headers: { cookie } }); |
| 83 | 86 | await res.arrayBuffer(); |
test/feed-caught-up.test.ts +2 −1
| @@ -12,6 +12,7 @@ import { serve, type Servers } from "../src/http/server.ts"; | ||
| 12 | 12 | import type { FeedItem } from "../src/feed/rank.ts"; |
| 13 | 13 | import { caughtUp, FEED_FIRST_PAGE, feedJs } from "../src/web/studio/feed.ts"; |
| 14 | 14 | import { promote } from "./helpers/members.ts"; |
| 15 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 15 | 16 | |
| 16 | 17 | describe("the timeline's caught-up line", () => { |
| 17 | 18 | let dataDir: string; |
| @@ -25,7 +26,7 @@ describe("the timeline's caught-up line", () => { | ||
| 25 | 26 | for (let i = 0; i < n; i++) |
| 26 | 27 | app.bus.append({ kind: "run.finished", repoId: repo.id, actorId: alice.id, payload: { kind: "agent", state: "ok", summary: `work ${i}`, runId: `r${i}` } }); |
| 27 | 28 | }; |
| 28 | const home = async (path = "/") => (await fetch(`${servers.url}${path}`, { headers: { cookie } })).text(); | |
| 29 | const home = (path = "/") => freshHome(servers.url, cookie, path); | |
| 29 | 30 | |
| 30 | 31 | beforeAll(async () => { |
| 31 | 32 | dataDir = await mkdtemp(join(tmpdir(), "forge-caught-up-")); |
test/feed-day-divider.test.ts +2 −1
| @@ -10,6 +10,7 @@ import { createUser, type User } from "../src/core/users.ts"; | ||
| 10 | 10 | import { serve, type Servers } from "../src/http/server.ts"; |
| 11 | 11 | import { dayKeepJs } from "../src/web/studio/feed.ts"; |
| 12 | 12 | import { promote } from "./helpers/members.ts"; |
| 13 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 13 | 14 | |
| 14 | 15 | const dividerLabels = (text: string): string[] => [...text.matchAll(/data-day-divider="([^"]+)"/g)].map((m) => m[1]!); |
| 15 | 16 | |
| @@ -21,7 +22,7 @@ describe("day dividers on the home timeline", () => { | ||
| 21 | 22 | let repoId = ""; |
| 22 | 23 | let aliceId = ""; |
| 23 | 24 | |
| 24 | const get = async (path: string) => (await fetch(`${servers.url}${path}`, { headers: { cookie } })).text(); | |
| 25 | const get = (path: string) => freshHome(servers.url, cookie, path); | |
| 25 | 26 | |
| 26 | 27 | beforeAll(async () => { |
| 27 | 28 | dataDir = await mkdtemp(join(tmpdir(), "forge-day-divider-")); |
test/front-door-timing.test.ts +2
| @@ -1,4 +1,5 @@ | ||
| 1 | 1 | /** The front door's Server-Timing names every phase, and a heavy feed reads fast (#3311). */ |
| 2 | import { forgetHome } from "../src/web/studio/home_last.ts"; | |
| 2 | 3 | import { afterAll, beforeAll, expect, test } from "bun:test"; |
| 3 | 4 | import { rm, writeFile } from "node:fs/promises"; |
| 4 | 5 | import { join } from "node:path"; |
| @@ -34,6 +35,7 @@ test("the signed-in home names its session, reads and render and a cold feed rea | ||
| 34 | 35 | await get(); |
| 35 | 36 | const opened = openIssue(f.app, f.owner, f.repos[0]!, { title: "A write the feed must read again" }); |
| 36 | 37 | if (!opened.ok) throw new Error(String(opened.error)); |
| 38 | forgetHome(f.app.db); | |
| 37 | 39 | const res = await get(); |
| 38 | 40 | const all = parts(res); |
| 39 | 41 | const names = all.map((p) => p.split(";")[0]!); |
test/helpers/home-fresh.ts added +28
| @@ -0,0 +1,28 @@ | ||
| 1 | /** | |
| 2 | * Home as a browser leaves it (#7660): the page, with each region it served | |
| 3 | * from memory and marked `data-refresh` replaced by that region drawn fresh | |
| 4 | * from its own fragment. A test that wrote something and then reads home | |
| 5 | * means this one. | |
| 6 | */ | |
| 7 | export async function freshHome(base: string, cookie?: string, path = "/"): Promise<string> { | |
| 8 | const headers = cookie ? { cookie } : {}; | |
| 9 | let text = await (await fetch(`${base}${path}`, { headers })).text(); | |
| 10 | for (;;) { | |
| 11 | const open = /<div id="home-\w+" data-region data-refresh="([^"]+)">/.exec(text); | |
| 12 | if (open === null) return text; | |
| 13 | const src = open[1]!.replace(/&/g, "&"); | |
| 14 | const fresh = await (await fetch(`${base}${src}`, { headers })).text(); | |
| 15 | const inner = fresh.replace(/^\s*<div[^>]*>/, "").replace(/<\/div>\s*$/, ""); | |
| 16 | const from = open.index + open[0].length; | |
| 17 | let depth = 1; | |
| 18 | let end = from; | |
| 19 | for (const m of text.slice(from).matchAll(/<div\b|<\/div>/g)) { | |
| 20 | depth += m[0] === "</div>" ? -1 : 1; | |
| 21 | if (depth === 0) { | |
| 22 | end = from + m.index; | |
| 23 | break; | |
| 24 | } | |
| 25 | } | |
| 26 | text = `${text.slice(0, open.index)}<div data-fresh>${inner}${text.slice(end)}`; | |
| 27 | } | |
| 28 | } |
test/home-200.test.ts +4 −4
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | /** #5806: the owner's home on a forge of 200 repositories and 100 open asks is read in a bounded number of statements warm, every slow phase named. */ |
| 2 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 2 | 3 | import { afterAll, beforeAll, expect, test } from "bun:test"; |
| 3 | 4 | import { assignIssue, openIssue } from "../src/actions/issues.ts"; |
| 4 | import { waitingCache } from "../src/web/studio/home_cache.ts"; | |
| 5 | 5 | import { expectWork } from "./helpers/work.ts"; |
| 6 | 6 | import { statements } from "./helpers/statements.ts"; |
| 7 | 7 | import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts"; |
| @@ -44,16 +44,16 @@ test("the second look at home as the owner costs a bounded number of statements, | ||
| 44 | 44 | }); |
| 45 | 45 | |
| 46 | 46 | const waiting = async (): Promise<number> => { |
| 47 | const text = await (await fetch(`${f.base}/`, { headers: { cookie: f.ownerCookie } })).text(); | |
| 47 | const text = await freshHome(f.base, f.ownerCookie); | |
| 48 | 48 | return Number(/(\d+) waiting on you/.exec(text)?.[1] ?? 0); |
| 49 | 49 | }; |
| 50 | 50 | |
| 51 | 51 | test("the waiting line is a hit on the second look and a write puts it back", async () => { |
| 52 | 52 | const first = await waiting(); |
| 53 | 53 | expect(first).toBeGreaterThan(0); |
| 54 | const before = waitingCache(f.app.db).hits; | |
| 55 | 54 | expect(await waiting()).toBe(first); |
| 56 | expect(waitingCache(f.app.db).hits).toBeGreaterThan(before); | |
| 55 | const timing = (await fetch(`${f.base}/`, { headers: { cookie: f.ownerCookie } })).headers.get("server-timing") ?? ""; | |
| 56 | expect(timing).toMatch(/waiting;dur=[\d.]+;desc="hit"/); | |
| 57 | 57 | // Every ask is put down in one write: the line is gone at once, not in 30 seconds. |
| 58 | 58 | f.app.db.query("UPDATE issue SET assignee_id = NULL").run(); |
| 59 | 59 | expect(await waiting()).toBe(0); |
test/home-builders-idle.test.ts +2 −1
| @@ -19,6 +19,7 @@ import { type User } from "../src/core/users.ts"; | ||
| 19 | 19 | import { serve, type Servers } from "../src/http/server.ts"; |
| 20 | 20 | import { createForgeApp } from "../src/tui/forge.ts"; |
| 21 | 21 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; |
| 22 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 22 | 23 | import { seedAdmin } from "./helpers/admin.ts"; |
| 23 | 24 | import { quietLogs } from "./helpers/globals.ts"; |
| 24 | 25 | import { settleApp } from "./helpers/tui.ts"; |
| @@ -66,7 +67,7 @@ afterAll(async () => { | ||
| 66 | 67 | }); |
| 67 | 68 | |
| 68 | 69 | const home = async (): Promise<string> => |
| 69 | (await fetch(`${servers.url}/`, { headers: { cookie: `forge_session=${createSession(app.db, admin.id).token}; forge_mode=studio` } })).text(); | |
| 70 | freshHome(servers.url, `forge_session=${createSession(app.db, admin.id).token}; forge_mode=studio`); | |
| 70 | 71 | |
| 71 | 72 | async function terminal(): Promise<string> { |
| 72 | 73 | const t = headlessTerminal(200, 40); |
test/home-cache.test.ts +8 −4
| @@ -4,6 +4,8 @@ | ||
| 4 | 4 | * hit that runs none of their reads, and the owner's figures never reach |
| 5 | 5 | * another viewer. |
| 6 | 6 | */ |
| 7 | import { forgetHome } from "../src/web/studio/home_last.ts"; | |
| 8 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 7 | 9 | import { afterAll, beforeAll, expect, test } from "bun:test"; |
| 8 | 10 | import { pinPanel } from "../src/actions/board-pins.ts"; |
| 9 | 11 | import { writeBoard } from "../src/actions/boards.ts"; |
| @@ -12,8 +14,7 @@ import { kpiCache, panelCache } from "../src/web/studio/home_cache.ts"; | ||
| 12 | 14 | import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts"; |
| 13 | 15 | |
| 14 | 16 | let f: SnappyFixture; |
| 15 | const home = async (cookie?: string): Promise<string> => | |
| 16 | (await fetch(`${f.base}/`, { headers: cookie ? { cookie } : {} })).text(); | |
| 17 | const home = (cookie?: string): Promise<string> => freshHome(f.base, cookie); | |
| 17 | 18 | |
| 18 | 19 | beforeAll(async () => { |
| 19 | 20 | f = await snappyFixture(); |
| @@ -40,8 +41,10 @@ test("a warm home reads neither the KPIs nor the panels again; a write does", as | ||
| 40 | 41 | const [km, pm] = [kpi.misses, panel.misses]; |
| 41 | 42 | await home(f.ownerCookie); |
| 42 | 43 | expect([kpi.misses, panel.misses]).toEqual([km, pm]); |
| 43 | expect(kpi.hits).toBeGreaterThan(0); | |
| 44 | expect(panel.hits).toBeGreaterThan(0); | |
| 44 | // The second look is a hit one layer in front of those caches (`home_last.ts`): both phases say so. | |
| 45 | const timing = (await fetch(`${f.base}/`, { headers: { cookie: f.ownerCookie } })).headers.get("server-timing") ?? ""; | |
| 46 | expect(timing).toMatch(/kpis;dur=[\d.]+;desc="hit"/); | |
| 47 | expect(timing).toMatch(/panels;dur=[\d.]+;desc="hit"/); | |
| 45 | 48 | |
| 46 | 49 | f.app.db.query("UPDATE repo SET description = 'moved' WHERE id = ?1").run(f.repos[0]!.id); |
| 47 | 50 | await home(f.ownerCookie); |
| @@ -116,6 +119,7 @@ test("warm home holds p95 under the stated budget", async () => { | ||
| 116 | 119 | expect(warm.status).toBe(200); |
| 117 | 120 | |
| 118 | 121 | // What a miss of both runs that the warm page does not, with the feed still warm. |
| 122 | forgetHome(f.app.db); | |
| 119 | 123 | kpiCache(f.app.db).clear(); |
| 120 | 124 | panelCache(f.app.db).clear(); |
| 121 | 125 | const missed = await counted(); |
test/home-feed.test.ts +7 −6
| @@ -6,6 +6,7 @@ | ||
| 6 | 6 | * `feed` tool answers the same posts. Nothing private, nor an admin's |
| 7 | 7 | * configuration, reaches a reader's timeline. |
| 8 | 8 | */ |
| 9 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 9 | 10 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; |
| 10 | 11 | import { mkdtemp, rm } from "node:fs/promises"; |
| 11 | 12 | import { tmpdir } from "node:os"; |
| @@ -322,13 +323,13 @@ describe("home is the feed", () => { | ||
| 322 | 323 | ) |
| 323 | 324 | .get(alice.id); |
| 324 | 325 | expect(row).toEqual({ text, source: "composer" }); |
| 325 | const page = await get("/", aliceCookie).then((r) => r.text()); | |
| 326 | const page = await freshHome(servers.url, `${aliceCookie}; forge_mode=studio`); | |
| 326 | 327 | expect(page).toContain("<script>alert(1)</script> & more"); |
| 327 | 328 | expect(page).not.toContain("<script>alert(1)</script>"); |
| 328 | 329 | expect(page).toContain("Received"); |
| 329 | 330 | expect(page).toContain('<option value="forge">the forge</option>'); |
| 330 | 331 | // It is the author's alone, and another site's form is refused. |
| 331 | const bobs = await get("/", bobCookie).then((r) => r.text()); | |
| 332 | const bobs = await freshHome(servers.url, `${bobCookie}; forge_mode=studio`); | |
| 332 | 333 | expect(bobs).not.toContain("alert(1)"); |
| 333 | 334 | const forged = await fetch(`${servers.url}/tell`, { |
| 334 | 335 | method: "POST", |
| @@ -341,21 +342,21 @@ describe("home is the feed", () => { | ||
| 341 | 342 | }); |
| 342 | 343 | |
| 343 | 344 | test("a person with no apps can still send to the forge, and a day-old thought leaves the feed", async () => { |
| 344 | const empty = await get("/", bobCookie).then((r) => r.text()); | |
| 345 | const empty = await freshHome(servers.url, `${bobCookie}; forge_mode=studio`); | |
| 345 | 346 | expect(empty).toContain('name="app" value="forge"'); |
| 346 | 347 | expect(empty).toContain("No apps yet"); |
| 347 | 348 | const old = new Date(Date.now() - 2 * 86_400_000).toISOString(); |
| 348 | 349 | app.db |
| 349 | 350 | .query("INSERT INTO thought (id, author, text, source, received_at) VALUES ('old1', ?1, 'ancient musing', 'composer', ?2)") |
| 350 | 351 | .run(bob.id, old); |
| 351 | expect(await get("/", bobCookie).then((r) => r.text())).not.toContain("ancient musing"); | |
| 352 | expect(await freshHome(servers.url, `${bobCookie}; forge_mode=studio`)).not.toContain("ancient musing"); | |
| 352 | 353 | await fetch(`${servers.url}/tell`, { |
| 353 | 354 | method: "POST", |
| 354 | 355 | headers: { cookie: bobCookie, origin: servers.url }, |
| 355 | 356 | body: new URLSearchParams({ app: "forge", text: "fresh musing" }), |
| 356 | 357 | redirect: "manual", |
| 357 | 358 | }); |
| 358 | const after = await get("/", bobCookie).then((r) => r.text()); | |
| 359 | const after = await freshHome(servers.url, `${bobCookie}; forge_mode=studio`); | |
| 359 | 360 | expect(after).toContain("fresh musing"); |
| 360 | 361 | expect(after).not.toContain("ancient musing"); |
| 361 | 362 | }); |
| @@ -365,7 +366,7 @@ describe("home is the feed", () => { | ||
| 365 | 366 | const serving = (payload: Record<string, unknown>) => |
| 366 | 367 | app.bus.append({ kind: "app.serving", repoId: garden.id, payload: { app: "garden", env: "production", commitOid: oid, ...payload } }); |
| 367 | 368 | serving({ version: "0.7.251", pulls: [{ title: "Water <b>early</b>", number: 1 }, { title: "Mulch", number: 2 }, { title: "Weed", number: 3 }] }); |
| 368 | const page = await get("/", aliceCookie).then((r) => r.text()); | |
| 369 | const page = await freshHome(servers.url, `${aliceCookie}; forge_mode=studio`); | |
| 369 | 370 | expect(page).toContain("0.7.251 is serving: Water <b>early</b> and 2 more"); |
| 370 | 371 | expect(page).not.toContain("<b>early</b>"); |
| 371 | 372 | expect(page).toContain(`href="/alice/garden/commit/${oid}"`); |
test/home-frame.test.ts added +91
| @@ -0,0 +1,91 @@ | ||
| 1 | /** | |
| 2 | * Home paints its frame first (#7660): signed in, on a forge with thousands | |
| 3 | * of issues, a write between two looks leaves the next look answered from | |
| 4 | * memory, within the budget, with the stale regions marked for the page to | |
| 5 | * swap, and each late region has a fragment of its own. | |
| 6 | */ | |
| 7 | import { afterAll, beforeAll, expect, test } from "bun:test"; | |
| 8 | import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts"; | |
| 9 | import { forgetHome } from "../src/web/studio/home_last.ts"; | |
| 10 | ||
| 11 | /** The server's own time for `/` after a write, p95, in ms: what the request did before sending a byte. */ | |
| 12 | const APP_P95_MS = 50; | |
| 13 | /** The first response, uncompressed, on a forge this size. */ | |
| 14 | const FIRST_BYTES = 100_000; | |
| 15 | ||
| 16 | let f: SnappyFixture; | |
| 17 | beforeAll(async () => { | |
| 18 | f = await snappyFixture({ repos: 12, issuesPerRepo: 250 }); | |
| 19 | }, 240_000); | |
| 20 | afterAll(async () => { | |
| 21 | await f.close(); | |
| 22 | }); | |
| 23 | ||
| 24 | const look = async (cookie: string, path = "/") => { | |
| 25 | const res = await fetch(`${f.base}${path}`, { headers: { cookie } }); | |
| 26 | const text = await res.text(); | |
| 27 | const app = /app;dur=([\d.]+)/.exec(res.headers.get("server-timing") ?? ""); | |
| 28 | return { res, text, ms: app === null ? NaN : Number(app[1]) }; | |
| 29 | }; | |
| 30 | const write = (n: number) => | |
| 31 | f.app.db.query("UPDATE repo SET description = ?1 WHERE id = ?2").run(`moved ${n}`, f.repos[0]!.id); | |
| 32 | ||
| 33 | test("the data is realistic: thousands of issues", () => { | |
| 34 | const n = f.app.db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM issue").get()!.n; | |
| 35 | expect(n).toBeGreaterThanOrEqual(3000); | |
| 36 | }); | |
| 37 | ||
| 38 | test("after a write, signed-in home is answered from memory within the budget and says what to swap", async () => { | |
| 39 | forgetHome(f.app.db); | |
| 40 | const cold = await look(f.ownerCookie); | |
| 41 | expect(cold.res.status).toBe(200); | |
| 42 | expect(cold.text).not.toContain(`data-refresh="/`); | |
| 43 | const times: number[] = []; | |
| 44 | let marked = ""; | |
| 45 | for (let i = 0; i < 20; i++) { | |
| 46 | write(i); | |
| 47 | const got = await look(f.ownerCookie); | |
| 48 | expect(got.res.status).toBe(200); | |
| 49 | times.push(got.ms); | |
| 50 | if (i === 0) marked = got.text; | |
| 51 | await Bun.sleep(1); | |
| 52 | } | |
| 53 | times.sort((a, b) => a - b); | |
| 54 | expect(times[Math.ceil(times.length * 0.95) - 1]!).toBeLessThan(APP_P95_MS); | |
| 55 | for (const name of ["kpis", "waiting", "panels", "asked", "feed"]) { | |
| 56 | expect(marked).toContain(`id="home-${name}"`); | |
| 57 | } | |
| 58 | expect(marked).toContain('data-refresh="/live/home/feed"'); | |
| 59 | expect(marked).toContain('data-refresh="/live/home/kpis"'); | |
| 60 | }); | |
| 61 | ||
| 62 | test("the first response stays under its byte limit", async () => { | |
| 63 | const got = await look(f.ownerCookie); | |
| 64 | expect(new TextEncoder().encode(got.text).length).toBeLessThan(FIRST_BYTES); | |
| 65 | }); | |
| 66 | ||
| 67 | test("each late region answers alone, to its own viewer, and to no one else", async () => { | |
| 68 | for (const name of ["kpis", "waiting", "panels", "asked", "feed"]) { | |
| 69 | const res = await fetch(`${f.base}/live/home/${name}`, { headers: { cookie: f.ownerCookie } }); | |
| 70 | expect(res.status).toBe(200); | |
| 71 | expect((await res.text()).length).toBeGreaterThan(0); | |
| 72 | } | |
| 73 | for (const name of ["kpis", "panels"]) { | |
| 74 | const peer = await fetch(`${f.base}/live/home/${name}`, { headers: { cookie: f.peerCookie } }); | |
| 75 | expect(peer.status).toBe(404); | |
| 76 | } | |
| 77 | const peerFeed = await fetch(`${f.base}/live/home/feed`, { headers: { cookie: f.peerCookie } }); | |
| 78 | expect(peerFeed.status).toBe(200); | |
| 79 | const anon = await fetch(`${f.base}/live/home/feed`); | |
| 80 | expect(anon.status).toBe(404); | |
| 81 | }); | |
| 82 | ||
| 83 | test("one viewer's remembered regions never reach another", async () => { | |
| 84 | await look(f.ownerCookie); | |
| 85 | write(99); | |
| 86 | const peer = await look(f.peerCookie); | |
| 87 | const owners = await look(f.ownerCookie); | |
| 88 | expect(owners.text).toContain("olive/app-1"); | |
| 89 | expect(peer.text).not.toContain("home-kpis"); | |
| 90 | expect(peer.text).not.toContain("olive/app-1"); | |
| 91 | }); |
test/home-memory.test.ts added +90
| @@ -0,0 +1,90 @@ | ||
| 1 | /** | |
| 2 | * What home remembers (#7660) is safe to show: it ends when the viewer's access | |
| 3 | * does, its keys are bounded however many queries a client sends, and the | |
| 4 | * stale marker is escaped and has a path without a script. | |
| 5 | */ | |
| 6 | import { afterAll, beforeAll, expect, test } from "bun:test"; | |
| 7 | import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts"; | |
| 8 | import { forgetAll, forgetRegion, recall, remember } from "../src/core/region_memory.ts"; | |
| 9 | import { homeVariant } from "../src/web/studio/home_last.ts"; | |
| 10 | ||
| 11 | let f: SnappyFixture; | |
| 12 | beforeAll(async () => { | |
| 13 | f = await snappyFixture(); | |
| 14 | }, 120_000); | |
| 15 | afterAll(async () => { | |
| 16 | await f.close(); | |
| 17 | }); | |
| 18 | ||
| 19 | const get = async (path: string, cookie: string): Promise<string> => (await fetch(`${f.base}${path}`, { headers: { cookie } })).text(); | |
| 20 | ||
| 21 | test("a viewer whose access was revoked is never shown what they remembered", async () => { | |
| 22 | forgetAll(f.app.db); | |
| 23 | await get("/", f.peerCookie); | |
| 24 | const before = await get("/", f.peerCookie); | |
| 25 | expect(before).toContain("olive/app-0"); | |
| 26 | // The public repositories turn private: a write, so the next look is stale, but not for pablo. | |
| 27 | f.app.db.query("UPDATE repo SET visibility = 'private' WHERE visibility = 'public'").run(); | |
| 28 | const after = await get("/", f.peerCookie); | |
| 29 | expect(after).not.toContain("olive/app-0"); | |
| 30 | expect(after).not.toContain("olive/app-3"); | |
| 31 | expect(after).not.toContain("data-region data-refresh="); | |
| 32 | }); | |
| 33 | ||
| 34 | test("an admin who loses admin is not shown the figures they remembered", async () => { | |
| 35 | forgetAll(f.app.db); | |
| 36 | expect(await get("/", f.ownerCookie)).toContain('id="home-kpis"'); | |
| 37 | f.app.db.query("UPDATE user SET is_admin = 0 WHERE id = ?1").run(f.owner.id); | |
| 38 | expect(await get("/", f.ownerCookie)).not.toContain('id="home-kpis"'); | |
| 39 | f.app.db.query("UPDATE user SET is_admin = 1 WHERE id = ?1").run(f.owner.id); | |
| 40 | }); | |
| 41 | ||
| 42 | test("a stale region is marked with an escaped address and a link for a page with no script", async () => { | |
| 43 | forgetAll(f.app.db); | |
| 44 | const path = "/?all=1&a=%22%3E%3Cscript%3E&day=%22onfocus%3D"; | |
| 45 | await get(path, f.ownerCookie); | |
| 46 | f.app.db.query("UPDATE repo SET description = 'moved' WHERE id = ?1").run(f.repos[0]!.id); | |
| 47 | const text = await get(path, f.ownerCookie); | |
| 48 | const marked = [...text.matchAll(/data-refresh="([^"]*)"/g)].map((m) => m[1]!); | |
| 49 | expect(marked.length).toBeGreaterThan(0); | |
| 50 | for (const src of marked) expect(src).toMatch(/^\/live\/home\/(kpis|waiting|panels|asked|feed)(\?all=1)?$/); | |
| 51 | expect(text).toContain('<noscript><p class="small muted">Shown as of a moment ago. <a href="/live/home/'); | |
| 52 | // The address the script and the link follow answers one region, in one element. | |
| 53 | const res = await fetch(`${f.base}${marked[0]!}`, { headers: { cookie: f.ownerCookie } }); | |
| 54 | expect(res.status).toBe(200); | |
| 55 | expect((await res.text()).trim()).toMatch(/^<div[\s>][\s\S]*<\/div>$/); | |
| 56 | }); | |
| 57 | ||
| 58 | test("the keys a client can make are the few home reads", () => { | |
| 59 | const v = (q: string): string => homeVariant(new URL(`http://x/${q}`)); | |
| 60 | expect(v("")).toBe(""); | |
| 61 | expect(v("?a=1&b=2&utm=x")).toBe(""); | |
| 62 | expect(v("?all=anything")).toBe("?all=1"); | |
| 63 | expect(v("?cursor=nonsense&day=2026-01-01")).toBe(""); | |
| 64 | expect(v(`?day=%22%3E`)).toBe(""); | |
| 65 | expect(v(`?cursor=${"9".repeat(200)}`)).toBe(""); | |
| 66 | }); | |
| 67 | ||
| 68 | test("one viewer's many variants push out only their own", () => { | |
| 69 | const db = f.app.db; | |
| 70 | forgetAll(db); | |
| 71 | remember(db, "bea", "feed", "", "<p>bea</p>", "1.1", "m"); | |
| 72 | for (let i = 0; i < 100; i++) remember(db, "mal", "feed", `?n=${i}`, "x", "1.1", "m"); | |
| 73 | expect(recall(db, "bea", "feed", "", "m")?.html).toBe("<p>bea</p>"); | |
| 74 | let kept = 0; | |
| 75 | for (let i = 0; i < 100; i++) if (recall(db, "mal", "feed", `?n=${i}`, "m") !== undefined) kept++; | |
| 76 | expect(kept).toBe(16); | |
| 77 | }); | |
| 78 | ||
| 79 | test("a copy is recalled only for the access it was drawn under, and forgetting names what goes", () => { | |
| 80 | const db = f.app.db; | |
| 81 | forgetAll(db); | |
| 82 | remember(db, "bea", "feed", "", "a", "1.1", "m.1"); | |
| 83 | remember(db, "bea", "panels", "", "b", "1.1", "m.1"); | |
| 84 | expect(recall(db, "bea", "feed", "", "m.2")).toBeUndefined(); | |
| 85 | forgetRegion(db, "bea", "panels"); | |
| 86 | expect(recall(db, "bea", "panels", "", "m.1")).toBeUndefined(); | |
| 87 | expect(recall(db, "bea", "feed", "", "m.1")?.html).toBe("a"); | |
| 88 | forgetRegion(db, "bea"); | |
| 89 | expect(recall(db, "bea", "feed", "", "m.1")).toBeUndefined(); | |
| 90 | }); |
test/home-walk.test.ts +2 −2
| @@ -4,6 +4,7 @@ | ||
| 4 | 4 | * post; a crew member's run and pull are posts with its name; a share |
| 5 | 5 | * arrives as a post that says who shared it; the apps are one link away. |
| 6 | 6 | */ |
| 7 | import { freshHome } from "./helpers/home-fresh.ts"; | |
| 7 | 8 | import { afterAll, beforeAll, expect, test } from "bun:test"; |
| 8 | 9 | import { mkdtemp, rm } from "node:fs/promises"; |
| 9 | 10 | import { tmpdir } from "node:os"; |
| @@ -57,8 +58,7 @@ test("composer, timeline, a tell, a crew member's run and pull, a share, the app | ||
| 57 | 58 | if (!repo.ok) throw new Error("repo"); |
| 58 | 59 | expect(addToCrew(app, root, repo.value, "scribe").ok).toBe(true); |
| 59 | 60 | const cookie = (u: User) => `forge_session=${createSession(app.db, u.id).token}; forge_mode=studio`; |
| 60 | const home = async (u: User) => | |
| 61 | (await fetch(`${servers.url}/`, { headers: { cookie: cookie(u) } })).text(); | |
| 61 | const home = (u: User) => freshHome(servers.url, cookie(u)); | |
| 62 | 62 | |
| 63 | 63 | // The composer comes first, then the timeline. |
| 64 | 64 | const first = await home(alice); |
test/observability-budget.test.ts +2
| @@ -7,6 +7,7 @@ | ||
| 7 | 7 | * the wall clock is never the judge (#3069). Before the fix each page cost |
| 8 | 8 | * hundreds of those passes. |
| 9 | 9 | */ |
| 10 | import { forgetHome } from "../src/web/studio/home_last.ts"; | |
| 10 | 11 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; |
| 11 | 12 | import { rm } from "node:fs/promises"; |
| 12 | 13 | import { tmpdir } from "node:os"; |
| @@ -96,6 +97,7 @@ describe("on production's size", () => { | ||
| 96 | 97 | await get("/"); |
| 97 | 98 | // A write moves the clock home's caches are keyed by: this view is cold. |
| 98 | 99 | app.db.run("UPDATE repo SET description = description || ''"); |
| 100 | forgetHome(app.db); | |
| 99 | 101 | const unit = baseline(); |
| 100 | 102 | const res = await get("/"); |
| 101 | 103 | expect(res.status).toBe(200); |
actions importing from web/ inverts the layers. Put the memory in a neutral module or invalidate from the web side.