Public: anyone can read this; the crew can change it.

Land "Since you were here" from pull #6684's branch, with its duplicate `pull_merged` index taken out #6721

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/6711-land-since-you-were-here-from-pull-6684 → main 5 commits fork 2 threads · 2 unresolved

  • Merged. scout/6711-land-since-you-were-here-from-pull-6684 was merged into main as fed63db.

Review

Sign in to review this change.
10 files changed +303 −4
README.md +2
@@ -40,6 +40,8 @@ when). The terminal's palette has the same as "Crew hand-overs", a read-only
4040 list of the same rows (repo#number, title, reasons; it does not open the
4141 issue — use the link on the page). Both are a forge admin's only.
4242
43 Home opens with "Since you were here · 2 days": the pulls that landed in public repositories since you last loaded `/` (1 hour to 7 days back, a day on a first visit); the terminal's home shows the same lines. See `docs/features/since-you-were-here.md`.
44
4345 A hand-over card on `/asks` (an agent's pull the reviewer stopped on after three rounds) has "Give back to the crew": one press assigns the pull, and the open issue it closes if a person holds it, to the agent that opened it, writes "Given back to the crew by @who", and the card leaves; the next tick starts its run. It is for a writer of the repository, and a pull a person opened, or by an agent gone from the crew, gets no button. In the terminal, `b` on a handed-over row of the On you pane does the same (`src/actions/give_back.ts`).
4446
4547 A pull the merge queue holds as red alone (red on its own merge at its current head; cleared by a new head or a moved main) says so under its checks line: "red alone at <time>, waiting for a new head or a new main", on the pull page, its live checks line and the studio change step, and in the checks row of the pull's terminal screen (`redMark` in `src/checks/queue.ts`). Nothing is drawn when it is not held.
docs/features/since-you-were-here.md added +15
@@ -0,0 +1,15 @@
1 # Since you were here
2
3 Home shows a digest headed `Since you were here · 2 days`: the pulls that landed in public repositories since you last loaded `/`.
4
5 - The forge records your last load in `home_visit` (sqlite), at most once a minute per user.
6 - The window starts at that time, at least 1 hour and at most 7 days. A first visit looks back 24 hours.
7 - Nothing landed in the window: no section. Private repositories are never read.
8
9 ## The terminal
10
11 The terminal's home (`ssh`, the dashboard) shows the same lines under the same heading, read from the same `home_visit` row. The terminal only reads it: opening home on the web is what moves "since you were here".
12
13 ## A read that writes
14
15 `GET /` writes the viewer's `home_visit` row, but only for a real navigation: a request marked `Sec-Purpose`/`Purpose` prefetch, prerender or preview, or whose `Sec-Fetch-Dest` is not `document`, reads the row and leaves it alone. That is a deliberate exception to "a GET changes nothing": it is one upsert per user per minute at most, it holds no content, and the response does not depend on the write (the digest uses the visit *before* this one).
src/core/home_visit.ts added +35
@@ -0,0 +1,35 @@
1 /**
2 * When a person last loaded `/`, in epoch milliseconds. `noteVisit` answers
3 * the visit before this one and writes the new time at most once a minute,
4 * so a reload storm is one cheap indexed read.
5 */
6
7 import type { Database } from "bun:sqlite";
8
9 export const VISIT_WRITE_GAP_MS = 60 * 1000;
10
11 export function lastVisit(db: Database, userId: string): number | undefined {
12 return db.query<{ at: number }, [string]>("SELECT at FROM home_visit WHERE user_id = ?1").get(userId)?.at;
13 }
14
15 /** The time of the visit before this one, if any; records this one unless it is within a minute of it. */
16 export function noteVisit(db: Database, userId: string, now: number): number | undefined {
17 const before = lastVisit(db, userId);
18 if (before === undefined || now - before >= VISIT_WRITE_GAP_MS)
19 db.query(
20 "INSERT INTO home_visit (user_id, at) VALUES (?1, ?2) ON CONFLICT(user_id) DO UPDATE SET at = excluded.at",
21 ).run(userId, now);
22 return before;
23 }
24
25 /**
26 * Whether a request is a person opening the page: not a prefetch or preview
27 * (`Sec-Purpose` / `Purpose`), and, when the browser says what it is fetching,
28 * a document. A client that sends neither header (curl, the terminal) counts.
29 */
30 export function isNavigation(req: Request): boolean {
31 const purpose = `${req.headers.get("sec-purpose") ?? ""} ${req.headers.get("purpose") ?? ""}`.toLowerCase();
32 if (purpose.includes("prefetch") || purpose.includes("prerender") || purpose.includes("preview")) return false;
33 const dest = req.headers.get("sec-fetch-dest");
34 return dest === null || dest === "document";
35 }
src/core/migrations/20261027090000-home-visit.ts added +8
@@ -0,0 +1,8 @@
1 // When a person last loaded home (#5804): the "Since you were here" digest
2 // starts there. One row per user, written at most once a minute.
3 export default `
4 CREATE TABLE home_visit (
5 user_id TEXT PRIMARY KEY REFERENCES user(id) ON DELETE CASCADE,
6 at INTEGER NOT NULL
7 ) WITHOUT ROWID;
8 `;
src/feed/digest.ts +18 −2
@@ -5,6 +5,22 @@
55
66 export const DIGEST_WINDOW_MS = 24 * 3600 * 1000;
77 export const DIGEST_LINES = 3;
8 export const DIGEST_MIN_MS = 3600 * 1000;
9 export const DIGEST_MAX_MS = 7 * 24 * 3600 * 1000;
10
11 /** The window since the last visit: a day for a first visit, else at least an hour and at most a week. */
12 export function windowOf(lastVisit: number | undefined, now: number): number {
13 if (lastVisit === undefined || !Number.isFinite(lastVisit)) return DIGEST_WINDOW_MS;
14 return Math.min(DIGEST_MAX_MS, Math.max(DIGEST_MIN_MS, now - lastVisit));
15 }
16
17 /** The heading that names the window, e.g. `Since you were here · 2 days`. */
18 export function headingOf(windowMs: number): string {
19 const hours = Math.round(windowMs / 3600000);
20 if (hours < 24) return `Since you were here · ${hours} ${hours === 1 ? "hour" : "hours"}`;
21 const days = Math.round(hours / 24);
22 return `Since you were here · ${days} ${days === 1 ? "day" : "days"}`;
23 }
824
925 export type DigestKind = "landed" | "released" | "failed" | "decided";
1026 const KINDS: readonly DigestKind[] = ["landed", "released", "failed", "decided"];
@@ -32,8 +48,8 @@ interface Group {
3248 counts: Record<DigestKind, number>;
3349 }
3450
35 export function digest(events: readonly DigestEvent[], now: number): DigestLine[] {
36 const since = now - DIGEST_WINDOW_MS;
51 export function digest(events: readonly DigestEvent[], now: number, windowMs: number = DIGEST_WINDOW_MS): DigestLine[] {
52 const since = now - windowMs;
3753 const groups = new Map<string, Group>();
3854 for (const e of events) {
3955 if (!(e.at >= since && e.at <= now)) continue;
src/tui/screens/dashboard.ts +19
@@ -23,6 +23,8 @@ import { coveredByVerdict, forgeVerdict, verdictLine } from "../../kernel/guard_
2323 import { clientAppRows, clientDoings, clientFinished, doingLine, finishedLine } from "../../core/client-apps.ts";
2424 import { nextStepsFor } from "../../core/next-steps.ts";
2525 import { agentsLine } from "../../web/studio/client_home.ts";
26 import { lastVisit } from "../../core/home_visit.ts";
27 import { sinceDigest } from "../../web/studio/since.ts";
2628 import { askForSomething } from "./tell.ts";
2729 import { mayView } from "../../core/sealed.ts";
2830 import { applyFleetIds, previewFleetIds, undoFleetOp } from "../../actions/fleet.ts";
@@ -183,6 +185,8 @@ export function dashboardScreen(f: Session, ctx: Ctx): Screen {
183185 let waits: string[] = [];
184186 /** A client's apps, one sentence each on whether its agents work or wait (#1812). */
185187 let clientLines: string[] = [];
188 /** "Since you were here" and its lines: read from the web home's last-visit row, never written here. */
189 let sinceLines: string[] = [];
186190 /** The ids of those apps, so the key that asks for something stays on them. */
187191 let clientRepoIds = new Set<string>();
188192 /** The live address of each of those apps that is running, by repo id; `o` shows it. */
@@ -235,6 +239,12 @@ export function dashboardScreen(f: Session, ctx: Ctx): Screen {
235239 ctx.invalidate();
236240 };
237241
242 const loadSinceLines = (): void => {
243 const d = f.user ? sinceDigest(f.app.db, lastVisit(f.app.db, f.user.id), Date.now()) : undefined;
244 sinceLines = d === undefined ? [] : [d.heading, ...d.lines.map((l) => ` ${l.says}`)];
245 ctx.invalidate();
246 };
247
238248 const loadPanelLines = async (): Promise<void> => {
239249 const listed = await listPinned(f.app, f.user);
240250 const titles = listed.ok ? listed.value.map((p) => `${plain(p.panel.title)} (${plain(p.board.title)})`) : [];
@@ -881,6 +891,7 @@ export function dashboardScreen(f: Session, ctx: Ctx): Screen {
881891 // functions of the outbox, so one subscription refreshes all of them.
882892 live(f, signal, {}, reload);
883893 reload();
894 loadSinceLines();
884895 return Promise.all([loadClientLines(), loadPanelLines()]).then(() => undefined);
885896 },
886897 render(area, s) {
@@ -910,6 +921,14 @@ export function dashboardScreen(f: Session, ctx: Ctx): Screen {
910921 );
911922 body = rest!;
912923 }
924 if (sinceLines.length > 0) {
925 const lines = sinceLines.slice(0, 4);
926 const [strip, rest] = split(body, "v", [len(lines.length), fill(1)]);
927 lines.forEach((line, i) =>
928 s.text(strip!.x, strip!.y + i, truncate(line, strip!.w), style(i === 0 ? c.mutedFg : c.fg, c.bg), strip!.x + strip!.w),
929 );
930 body = rest!;
931 }
913932 // A short terminal drops the last figures before it drops the list.
914933 const shown = figures.slice(0, Math.max(0, body.h - FIGURES_ROOM));
915934 if (shown.length > 0) {
src/web/studio/home.ts +9 −1
@@ -58,6 +58,8 @@ import { feedCandidateCache } from "../../feed/feed.ts";
5858 import { notePhase, notePhaseRest, takeSessionTime, timePhase, timePhaseAsync } from "../../core/route_timing.ts";
5959 import { kpiCache, panelCache, waitingCache } from "./home_cache.ts";
6060 import { pinnedPanels } from "./pinned.ts";
61 import { isNavigation, lastVisit, noteVisit } from "../../core/home_visit.ts";
62 import { sinceYouWere } from "./since.ts";
6163 import { homeTabs } from "./home_tabs.ts";
6264 import { crewKeyNote } from "./crew.ts";
6365 import { everydayCard } from "../pages/everyday.ts";
@@ -266,6 +268,11 @@ export async function studioHome(
266268 const { repos: appRepos } = timePhase(ctx.request, "repos", () => writableAppsOf(ctx, user));
267269 // Render is what is left of the page once the named reads are taken out.
268270 const rendering = performance.now();
271 const visitedAt = Date.now();
272 const before = timePhase(ctx.request, "visit", () =>
273 isNavigation(ctx.request) ? noteVisit(ctx.app.db, user.id, visitedAt) : lastVisit(ctx.app.db, user.id),

GET / mutates here. Bounded and documented, but it is the one write on a GET; a same-origin POST would be cleaner.

274 );
275 const since = timePhase(ctx.request, "since", () => sinceYouWere(ctx.app.db, before, visitedAt));
269276 const ask = firstSteps ?? everydayCard(ctx, user);
270277 const figures = user.isAdmin && steps === undefined;
271278 const ceilingUsd = forgeCeiling(ctx.app.config);
@@ -286,6 +293,7 @@ export async function studioHome(
286293 <a href="/">Just my apps</a>
287294 </p>`,
288295 )}
296 ${since}
289297 ${homeTabs("feed")}
290298 <div class="split" data-rail="wide">
291299 <div class="stack">
@@ -316,7 +324,7 @@ export async function studioHome(
316324 body,
317325 scriptJs: `${liveClientJs}\n${liveRegionsJs}\n${feedJs}${ask ? `\n${passkeyJs}` : ""}`,
318326 });
319 notePhaseRest(ctx.request, "render", rendering, ["kpis", "vitals", "waiting", "panels", "feed"]);
327 notePhaseRest(ctx.request, "render", rendering, ["kpis", "vitals", "waiting", "panels", "feed", "visit", "since"]);
320328 return res;
321329 }
322330
src/web/studio/since.ts added +55
@@ -0,0 +1,55 @@
1 /**
2 * "Since you were here · 2 days": the pulls that landed in public repositories
3 * since the viewer last loaded home (`core/home_visit.ts`), as the digest's
4 * lines. Nothing landed, no section. Private repositories are never read here.
5 */
6
7 import type { Database } from "bun:sqlite";
8 import { digest, headingOf, windowOf, type DigestEvent, type DigestLine } from "../../feed/digest.ts";
9 import { html, type Html } from "../html.ts";
10 import { listCard, row } from "../ui.ts";
11
12 interface Row {
13 owner: string;
14 name: string;
15 number: number;
16 merged_at: string;
17 }
18
19 export function landedSince(db: Database, since: number, now: number): DigestEvent[] {
20 const rows = db
21 .query<Row, [string, string]>(
22 `SELECT u.username AS owner, r.name AS name, i.number AS number, p.merged_at AS merged_at
23 FROM pull p JOIN issue i ON i.id = p.issue_id JOIN repo r ON r.id = i.repo_id JOIN user u ON u.id = r.owner_id
24 WHERE p.merged_at >= ?1 AND p.merged_at <= ?2 AND r.visibility = 'public'
25 ORDER BY p.merged_at DESC LIMIT 200`,
26 )
27 .all(new Date(since).toISOString(), new Date(now).toISOString());
28 return rows.map((r) => ({
29 repo: `${r.owner}/${r.name}`,
30 kind: "landed",
31 at: Date.parse(r.merged_at),
32 href: `/${r.owner}/${r.name}/pulls/${r.number}`,
33 weight: 1,
34 }));
35 }
36
37 /** The heading and lines of the digest since `lastVisit`; no lines, no digest. */
38 export function sinceDigest(
39 db: Database,
40 lastVisit: number | undefined,
41 now: number,
42 ): { heading: string; lines: DigestLine[] } | undefined {
43 const window = windowOf(lastVisit, now);
44 const lines = digest(landedSince(db, now - window, now), now, window);
45 return lines.length === 0 ? undefined : { heading: headingOf(window), lines };
46 }
47
48 export function sinceYouWere(db: Database, lastVisit: number | undefined, now: number): Html {
49 const d = sinceDigest(db, lastVisit, now);
50 if (d === undefined) return html``;
51 return listCard(
52 d.lines.map((l) => row({ title: html`${l.says}`, href: l.href, nav: true })),
53 { id: "since-you-were-here", head: html`<h2>${d.heading}</h2>` },

The row title is the digest line text only. A short meta with the repo name would help a screen-reader user tell lines apart.

54 );
55 }
test/home-200.test.ts +1 −1
@@ -39,7 +39,7 @@ test("the second look at home as the owner is within its budget, and every phase
3939 console.log(`home warm ${warm.ms.toFixed(0)} ms:`, warm.phases.map((p) => `${p.name} ${p.ms}`).join(", "));
4040 expectBudget(warm.ms, { ms: 300, against: await baselineMs(f.base) });
4141 // The page's own total is the sum of named phases: nothing over 20 ms hides in "app" alone.
42 const known = new Set(["app", "spawned", "steps", "session", "repos", "kpis", "waiting", "panels", "feed", "render"]);
42 const known = new Set(["app", "spawned", "steps", "session", "repos", "kpis", "waiting", "panels", "feed", "visit", "since", "render"]);
4343 for (const p of slow) expect(known.has(p.name)).toBe(true);
4444 const total = warm.phases.find((p) => p.name === "app")!.ms;
4545 const named = warm.phases.filter((p) => p.name !== "app").reduce((s, p) => s + p.ms, 0);
test/home-since.test.ts added +141
@@ -0,0 +1,141 @@
1 /**
2 * #5804: "Since you were here" starts at the viewer's last load of home, at
3 * least an hour and at most a week back, a day on a first visit; recording
4 * the visit writes at most once a minute and shows as a `visit` phase on `/`.
5 */
6 import { afterAll, beforeAll, expect, test } from "bun:test";
7 import { lastVisit, noteVisit } from "../src/core/home_visit.ts";
8 import { DIGEST_WINDOW_MS, headingOf, windowOf } from "../src/feed/digest.ts";
9 import { landedSince } from "../src/web/studio/since.ts";
10 import { createForgeApp } from "../src/tui/forge.ts";
11 import { dashboardScreen } from "../src/tui/screens/dashboard.ts";
12 import { headlessTerminal } from "../src/tui/term/terminal.ts";
13 import { settleApp } from "./helpers/tui.ts";
14 import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts";
15
16 const H = 3600 * 1000;
17 const now = Date.parse("2026-10-27T12:00:00Z");
18
19 test("the window: a first visit is a day, 3 hours is 3 hours, 10 days is capped at 7, under an hour is an hour", () => {
20 expect(windowOf(undefined, now)).toBe(DIGEST_WINDOW_MS);
21 expect(windowOf(now - 3 * H, now)).toBe(3 * H);
22 expect(windowOf(now - 10 * 24 * H, now)).toBe(7 * 24 * H);
23 expect(windowOf(now - 5 * 60 * 1000, now)).toBe(H);
24 expect(windowOf(now + H, now)).toBe(H);
25 });
26
27 test("the heading names the window", () => {
28 expect(headingOf(3 * H)).toBe("Since you were here · 3 hours");
29 expect(headingOf(H)).toBe("Since you were here · 1 hour");
30 expect(headingOf(48 * H)).toBe("Since you were here · 2 days");
31 expect(headingOf(7 * 24 * H)).toBe("Since you were here · 7 days");
32 });
33
34 let f: SnappyFixture;
35 beforeAll(async () => {
36 f = await snappyFixture();
37 }, 120_000);
38 afterAll(async () => {
39 await f.close();
40 });
41
42 test("a visit is written at most once a minute and answers the one before", () => {
43 const id = f.peer.id;
44 expect(noteVisit(f.app.db, id, now)).toBeUndefined();
45 expect(lastVisit(f.app.db, id)).toBe(now);
46 expect(noteVisit(f.app.db, id, now + 30_000)).toBe(now);
47 expect(lastVisit(f.app.db, id)).toBe(now);
48 expect(noteVisit(f.app.db, id, now + 61_000)).toBe(now);
49 expect(lastVisit(f.app.db, id)).toBe(now + 61_000);
50 });
51
52 test("home heads the digest with the window, lists only public landed pulls, and records the visit", async () => {
53 const db = f.app.db;
54 const pub = f.repos.find((r) => r.visibility === "public")!;
55 const priv = f.repos.find((r) => r.visibility === "private")!;
56 const land = (repoId: string, number: number, ago: number): void => {
57 const id = `since-${repoId}-${number}`;
58 db.query("INSERT INTO issue (id, repo_id, number, kind, title, author_id) VALUES (?1, ?2, ?3, 'pull', 't', ?4)").run(id, repoId, 9000 + number, f.owner.id);
59 db.query("INSERT INTO pull (issue_id, base_branch, head_branch, base_oid, head_oid, merged_at) VALUES (?1, 'main', 'h', 'a', 'b', ?2)").run(id, new Date(Date.now() - ago).toISOString());
60 };
61 land(pub.id, 1, 2 * H);
62 land(pub.id, 2, 5 * 24 * H);
63 land(priv.id, 3, H);
64 expect(landedSince(db, Date.now() - 3 * H, Date.now())).toHaveLength(1);
65 expect(landedSince(db, Date.now() - 10 * 24 * H, Date.now())).toHaveLength(2);
66
67 db.query("INSERT INTO home_visit (user_id, at) VALUES (?1, ?2) ON CONFLICT(user_id) DO UPDATE SET at = excluded.at").run(f.owner.id, Date.now() - 3 * H);
68 const res = await fetch(`${f.base}/?all=1`, { headers: { cookie: f.ownerCookie } });
69 const text = await res.text();
70 expect(text).toContain("Since you were here · 3 hours");
71 expect(text).toContain("1 landed");
72 expect(text).not.toContain(`/${f.owner.username}/${priv.name}/pulls/9003`);
73 const visit = /visit;dur=([\d.]+)/.exec(res.headers.get("server-timing") ?? "");
74 expect(visit).not.toBeNull();
75 expect(Date.now() - lastVisit(db, f.owner.id)!).toBeLessThan(5000);
76 });
77
78 test("a first-time user, with no home_visit row, gets the 24 hour heading", async () => {
79 const db = f.app.db;
80 db.query("DELETE FROM home_visit WHERE user_id = ?1").run(f.owner.id);
81 const text = await (await fetch(`${f.base}/?all=1`, { headers: { cookie: f.ownerCookie } })).text();
82 expect(text).toContain("Since you were here · 1 day<");
83 expect(text).toContain("1 landed");
84 expect(lastVisit(db, f.owner.id)).toBeDefined();
85 });
86
87 test("the terminal home shows the same digest from the same row and writes nothing", async () => {
88 const db = f.app.db;
89 const at = Date.now() - 3 * H;
90 db.query("INSERT INTO home_visit (user_id, at) VALUES (?1, ?2) ON CONFLICT(user_id) DO UPDATE SET at = excluded.at").run(f.owner.id, at);
91 const terminal = headlessTerminal(120, 30);
92 const { app: tui, session } = createForgeApp(
93 f.app,
94 { sshUsername: f.owner.username, remoteAddress: "127.0.0.1", fingerprint: "SHA256:test", user: f.owner },
95 { terminal, exitHooks: false },
96 );
97 tui.start();
98 tui.ctx.push(dashboardScreen(session, tui.ctx));
99 await settleApp(tui);
100 const text = tui.screen.toText();
101 tui.stop?.();
102 expect(text).toContain("Since you were here · 3 hours");
103 expect(text).toContain("1 landed");
104 expect(lastVisit(db, f.owner.id)).toBe(at);
105 });
106
107 test("a prefetch, a preview or a non-document fetch leaves home_visit unchanged; a navigation moves it", async () => {
108 const db = f.app.db;
109 const at = Date.now() - 3 * H;
110 const set = () => db.query("INSERT INTO home_visit (user_id, at) VALUES (?1, ?2) ON CONFLICT(user_id) DO UPDATE SET at = excluded.at").run(f.owner.id, at);
111 for (const headers of [
112 { "sec-purpose": "prefetch" },
113 { purpose: "prefetch" },
114 { "sec-purpose": "prefetch;prerender" },
115 { "sec-fetch-dest": "empty" },
116 { "sec-fetch-dest": "iframe" },
117 ]) {
118 set();
119 const text = await (await fetch(`${f.base}/?all=1`, { headers: { cookie: f.ownerCookie, ...headers } })).text();
120 expect(text).toContain("Since you were here · 3 hours");
121 expect(lastVisit(db, f.owner.id)).toBe(at);
122 }
123 set();
124 await (await fetch(`${f.base}/?all=1`, { headers: { cookie: f.ownerCookie, "sec-fetch-dest": "document" } })).text();
125 expect(lastVisit(db, f.owner.id)).toBeGreaterThan(at);
126 });
127
128 test("the digest query reads pull through the existing pull_merged index, and the home-visit migration adds no second one", () => {
129 const plan = f.app.db
130 .query<{ detail: string }, [string, string]>(
131 `EXPLAIN QUERY PLAN SELECT p.merged_at FROM pull p WHERE p.merged_at >= ?1 AND p.merged_at <= ?2 ORDER BY p.merged_at DESC LIMIT 200`,
132 )
133 .all("2026-10-26T00:00:00.000Z", "2026-10-27T00:00:00.000Z")
134 .map((r) => r.detail)
135 .join("\n");
136 expect(plan).toContain("pull_merged");
137 const onMerged = f.app.db
138 .query<{ name: string }, []>(`SELECT name FROM sqlite_master WHERE type = 'index' AND tbl_name = 'pull' AND sql LIKE '%merged_at%'`)
139 .all();
140 expect(onMerged.map((r) => r.name)).toEqual(["pull_merged"]);
141 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs