Public: anyone can read this; the crew can change it.

An issue with hundreds of comments shows its newest 50 and one link for the earlier ones: issue #334's page under 250 KB #6134

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/6124-an-issue-with-hundreds-of-comments-shows → main 3 commits fork 4 threads · 4 unresolved

  • Merged. scout/6124-an-issue-with-hundreds-of-comments-shows was merged into main as cd5af9b.

Review

Sign in to review this change.
8 files changed +210 −6
README.md +18
@@ -296,6 +296,24 @@ carries the attention dot.
296296
297297 A fork ahead of its parent offers its commits upstream: on its home a signed-in reader of the parent sees "Offer N commits upstream" (into the new-pull form with both branches chosen) or "Offered in #M" once a pull is open; a writer who has the Following card gets its "Propose upstream" instead, one route not two. In the terminal the fork's About card (`I`) has an `offer` row and `o` opens the same pull form (or the open pull). Both read `upstreamOffer` (`src/actions/offer_upstream.ts`), the open pull by one SQL lookup.
298298
299 ## Long threads
300
301 An issue or pull page renders the opening post and its newest 50 comments
302 (`THREAD_WINDOW`, `src/core/issues.ts`) with the state-change lines between
303 them; above them a plain link, `Show N earlier items`, goes to the same page
304 with `?all=1` (other query parts kept), which renders everything, with no
305 JavaScript. N counts state-change lines as well as comments. A comment's own
306 anchor is `#c-<id>`; links the forge writes to a comment (the feed's replies)
307 go through `commentAnchor()`, which counts comments the way the page cuts
308 them, and carry `?all=1` when the comment is folded away, so they always
309 land. A hand-typed `#c-…` into the folded part cannot be seen by the server
310 (the fragment never leaves the browser), so it needs `?all=1` too. The live
311 region still appends new comments at the bottom.
312
313 The terminal's issue and pull screens stay whole on purpose: they are a
314 scrolling text view of one thread already in memory, not a page a phone
315 downloads, so there is no weight to cut and no second key to learn.
316
299317 ## Live in the terminal
300318
301319 `ssh` into the forge and you get the whole thing — repositories, code, issues,
docs/design/areas/comments.md +1
@@ -5,3 +5,4 @@ A comment on an issue or pull, and the form that writes one. Both are a `card()`
55 - `.comment` — one comment as a card: its `.card-head` (byline and time) on the muted ground, the body under it. Written by `commentCard()` in `src/web/comments.ts` and by `reviewRows()` in `src/web/diff-view.ts` (an inline review comment); styled in `src/web/style/components.ts`, and spaced inside a review thread by `.dc-review .comment` in `src/web/style/content.ts`.
66 - `.comment-body` — a comment's rendered markdown, always beside `.prose`, padded inside its card. Written by `commentCard()` in `src/web/comments.ts`, `reviewRows()` in `src/web/diff-view.ts` and `reviewEntry()` in `src/web/studio/change.ts`; styled in `src/web/style/components.ts` (tighter in a timeline event's card).
77 - `.comment-form` — the card that writes a comment: a textarea and its `.form-actions`. Written by `composer()` and `editForm()` in `src/web/pages/thread.ts`, `reviewRows()` in `src/web/diff-view.ts` (a reply in a review thread) and `studioApp()` in `src/web/studio/app.ts` (the "Tell the crew" box); styled in `src/web/style/components.ts`, with its narrow-screen action line and its review-thread margin in `src/web/style/content.ts`.
8 - `.tl-earlier` — the line above a long thread's newest items, `Show N earlier items`, a plain link to `?all=1`. Written by `conversation()` in `src/web/pages/thread.ts`; styled in `src/web/style/components.ts` beside `.tl-item`, indented to the timeline's item column.
src/core/issues.ts +20
@@ -639,6 +639,26 @@ export interface ListCommentsOptions {
639639 limit?: number;
640640 }
641641
642 /** How many timeline items a thread page shows before it folds the earlier ones behind `?all=1`. */
643 export const THREAD_WINDOW = 50;
644
645 /**
646 * Where a comment's own anchor lands on its thread page: `#c-<id>` when it
647 * is among the newest comments the page shows, `?all=1#c-<id>` when it is
648 * folded away. One COUNT, not the whole thread. The page cuts its window on
649 * comments too (`conversation()`, web/pages/thread.ts): the newest
650 * THREAD_WINDOW comments, with the state lines between them riding along.
651 * Both count the same thing, so the anchor always lands.
652 */
653 export function commentAnchor(db: Database, issueId: string, commentId: string): string {
654 const after = db
655 .query<{ n: number }, [string, string]>(
656 "SELECT COUNT(*) AS n FROM comment WHERE issue_id = ?1 AND kind = 'comment' AND id > ?2",
657 )
658 .get(issueId, commentId)!.n;
659 return `${after >= THREAD_WINDOW ? "?all=1" : ""}#c-${commentId}`;
outdated

Comments-only COUNT disagrees with the page window, which counts all entries. Make them count the same thing so the anchor always lands.

660 }
661
642662 export function listComments(
643663 db: Database,
644664 issueId: string,
src/feed/items.ts +10 −3
@@ -12,7 +12,7 @@ import type { Chips } from "../core/feed_chips.ts";
1212 import { listDecidedIn } from "../core/decisions.ts";
1313 import { audienceFilter } from "../core/event_audience.ts";
1414 import { listEventsVisibleTo, type ForgeEvent } from "../core/events.ts";
15 import { getIssuesByNumbers, numberKey, type Issue } from "../core/issues.ts";
15 import { commentAnchor, getIssuesByNumbers, numberKey, type Issue } from "../core/issues.ts";
1616 import {
1717 listDetachedMirrors,
1818 listOrigins,
@@ -597,13 +597,20 @@ function fromEvent(
597597 `#${number}`,
598598 60,
599599 );
600 // The comment itself: past the newest window it is behind `?all=1`.
601 const thread = subjects.issue(repo.id, number);
602 const commentId = p["commentId"];
603 const anchor =
604 thread !== undefined && typeof commentId === "string"
605 ? commentAnchor(db, thread.id, commentId)
606 : "";
600607 return {
601608 ...common,
602609 kind: "reply",
603610 link:
604 e.kind === "issue.commented"
611 (e.kind === "issue.commented"
605612 ? `${base(repo)}/issues/${number}`
606 : change,
613 : change) + anchor,
607614 what: said === "" ? `replied on “${on}”` : `on “${on}”: ${clip(said)}`,
608615 };
609616 }
src/web/pages/thread.ts +31 −2
@@ -40,6 +40,7 @@ import {
4040 getIssue,
4141 labelsFor,
4242 listComments,
43 THREAD_WINDOW,
4344 MAX_BODY,
4445 MAX_TITLE,
4546 } from "../../core/issues.ts";
@@ -130,6 +131,8 @@ export function threadContext(
130131 export interface Entry {
131132 at: string;
132133 node: Html;
134 /** A conversation comment: the window is cut on these (`commentAnchor`, core/issues.ts, counts the same). */
135 comment?: true;
133136 }
134137
135138 const EVENT_LOOK: Record<string, { icon: IconName; tone: Tone; verb: string }> =
@@ -305,6 +308,7 @@ export function conversation(
305308 ]);
306309 const entries: Entry[] = comments.map((c) => ({
307310 at: c.createdAt,
311 comment: true as const,
308312 node: threadComment(c.id, c.authorName, c.createdAt, c.body, {
309313 originAuthor: c.originAuthor,
310314 originKind: glyph,
@@ -315,6 +319,31 @@ export function conversation(
315319 entries.push(...interruptionEntries(ctx, issue));
316320 entries.push(...(options.extra ?? []));
317321 entries.sort((a, b) => (a.at < b.at ? -1 : a.at > b.at ? 1 : 0));
322 // The opening post and the newest THREAD_WINDOW comments with the state
323 // changes between them; everything before the oldest of those waits behind
324 // `?all=1` (a plain link, so it works without JavaScript). The cut is on
325 // comments so that `commentAnchor` counts the very same thing and every
326 // link the forge writes to a comment lands.
327 const everything = ctx.url.searchParams.get("all") === "1";
328 let hidden = 0;
329 if (!everything) {
outdated

Link is built from ctx.url.pathname; fine, but keep any other query the page needs and cover the pull page in a test too.

330 let seen = 0;
331 for (let i = entries.length - 1; i >= 0; i--) {
332 if (entries[i]!.comment && ++seen === THREAD_WINDOW) {
333 hidden = i;
334 break;
335 }
336 }
337 }
outdated

tl-earlier has no CSS and is not documented in docs/design/areas/comments.md. Style it with tokens and name it there. Reword the link as 'Show N earlier items', since N includes events.

338 const shown = hidden === 0 ? entries : entries.slice(hidden);
339 const more = new URLSearchParams(ctx.url.searchParams);
340 more.set("all", "1");
341 const earlier =
342 hidden === 0
343 ? null
344 : html`<p class="tl-earlier small">
345 <a href="${ctx.url.pathname}?${more.toString()}">Show ${String(hidden)} earlier items</a>
346 </p>`;
318347
319348 const opening = tlComment(
320349 commentCard({
@@ -334,7 +363,7 @@ export function conversation(
334363 node: html`<div class="timeline" id="conversation">
335364 ${opening}${when(options.editAction, (action) =>
336365 editForm(issue, action),
337 )}${entries.map((e) => e.node)}
366 )}${earlier}${shown.map((e) => e.node)}
338367 </div>`,
339368 lastCommentId: comments.length > 0 ? comments[comments.length - 1]!.id : "",
340369 };
@@ -353,7 +382,7 @@ export function threadComment(
353382 mark?: Mark | undefined;
354383 } = {},
355384 ): Html {
356 return html`<div class="tl-item" data-kind="comment" data-comment-id="${id}">
385 return html`<div class="tl-item" id="c-${id}" data-kind="comment" data-comment-id="${id}">
357386 <span class="tl-dot">${icon("comment")}</span>${commentCard({
358387 id,
359388 author,
src/web/style/components.ts +1
@@ -808,6 +808,7 @@ details[open] > summary.row > .row-lead { transform: rotate(90deg); }
808808 display: flex; align-items: center; gap: 6px; flex-wrap: wrap;
809809 min-height: 32px;
810810 }
811 .tl-earlier { margin: 0; padding-left: 36px; color: var(--muted-fg); font-size: 13px; }
811812 .tl-item[data-kind="comment"] .card { margin: 0; }
812813 .tl-item[data-kind="event"] > .card { flex-basis: 100%; margin: 4px 0 0; color: var(--fg); font-size: 14px; }
813814 .tl-item[data-kind="event"] > .card .comment-body { padding: 8px 12px; }
test/thread-render-timing.test.ts +1 −1
@@ -29,7 +29,7 @@ afterAll(async () => {
2929 });
3030
3131 async function view(): Promise<{ ms: number; html: string }> {
32 const res = await fetch(`${f.base}${path}`, { headers: { cookie: f.ownerCookie } });
32 const res = await fetch(`${f.base}${path}?all=1`, { headers: { cookie: f.ownerCookie } });
3333 const text = await res.text();
3434 expect(res.status).toBe(200);
3535 expect(text.match(/data-comment-id="/g)?.length).toBeGreaterThanOrEqual(200);
test/thread-window.test.ts added +128
@@ -0,0 +1,128 @@
1 /** A long thread shows its opening post and newest 50 items; `?all=1` shows everything (#6124). */
2 import { afterAll, beforeAll, expect, test } from "bun:test";
3 import { snappyFixture, type SnappyFixture } from "../bench/snappy.ts";
4 import { addComment, openIssue, setIssueState } from "../src/actions/issues.ts";
5 import { openPull } from "../src/actions/pulls.ts";
6 import { commitToBranch } from "../src/actions/repos.ts";
7
8 import { commentAnchor, getIssue, listComments } from "../src/core/issues.ts";
9 import { candidatesFor } from "../src/feed/items.ts";
10
11 let f: SnappyFixture;
12 let path = "";
13 let ids: string[] = [];
14 let issue: NonNullable<ReturnType<typeof getIssue>>;
15
16 beforeAll(async () => {
17 f = await snappyFixture({ repos: 1, issuesPerRepo: 1 });
18 const repo = f.repos[0]!;
19 const opened = openIssue(f.app, f.owner, repo, { title: "Long", body: "opening post" });
20 if (!opened.ok) throw new Error(String(opened.error));
21 issue = opened.value;
22 for (let i = 0; i < 300; i++) {
23 const added = addComment(f.app, f.owner, repo, opened.value, `comment number ${i}`);
24 if (!added.ok) throw new Error(String(added.error));
25 }
26 ids = listComments(f.app.db, issue.id, { kind: "comment" }).map((c) => c.id);
27 path = `/${repo.ownerName}/${repo.name}/issues/${opened.value.number}`;
28 }, 120_000);
29 afterAll(async () => {
30 await f.close();
31 });
32
33 const get = async (suffix = ""): Promise<string> =>
34 (await fetch(`${f.base}${path}${suffix}`, { headers: { cookie: f.ownerCookie } })).text();
35 const count = (html: string): number => html.match(/class="tl-item" id="c-/g)?.length ?? 0;
36
37 test("the default page has the newest 50, a link naming the 250 earlier, and stays small", async () => {
38 const html = await get();
39 expect(count(html)).toBe(50);
40 expect(html).toContain("comment number 299");
41 expect(html).not.toContain("comment number 249<");
42 expect(html).toContain(`href="${path}?all=1"`);
43 expect(html).toContain("Show 250 earlier items");
44 expect(html).toContain("opening post");
45 expect(new TextEncoder().encode(html).length).toBeLessThan(250 * 1024);
46 });
47
48 test("?all=1 has all 300 and no link", async () => {
49 const html = await get("?all=1");
50 expect(count(html)).toBe(300);
51 expect(html).not.toContain("earlier</a>");
52 });
53
54 test("a comment's anchor lands: hidden ones go through ?all=1", () => {
55 const db = f.app.db;
56 expect(commentAnchor(db, issue.id, ids[0]!)).toBe(`?all=1#c-${ids[0]}`);
57 expect(commentAnchor(db, issue.id, ids[249]!)).toBe(`?all=1#c-${ids[249]}`);
58 expect(commentAnchor(db, issue.id, ids[250]!)).toBe(`#c-${ids[250]}`);
59 expect(commentAnchor(db, issue.id, ids[299]!)).toBe(`#c-${ids[299]}`);
60 });
61
62 test("the feed links a reply to its own comment, through ?all=1 when it is folded away", () => {
63 const links = candidatesFor(f.app.db, f.owner, Date.now())
64 .filter((c) => c.link?.startsWith(path))
65 .map((c) => c.link);
66 expect(links).toContain(`${path}?all=1#c-${ids[0]}`);
67 expect(links).toContain(`${path}#c-${ids[299]}`);
68 });
69
70 test("the anchor id is in the page", async () => {
71 expect(await get()).toContain(`id="c-${ids[299]}"`);
72 });
73
74 test("state lines among the newest comments do not move the edge: the anchor still lands", async () => {
75 const repo = f.repos[0]!;
76 const opened = openIssue(f.app, f.owner, repo, { title: "Edge", body: "edge" });
77 if (!opened.ok) throw new Error(String(opened.error));
78 const edge = opened.value;
79 const said: string[] = [];
80 for (let i = 0; i < 60; i++) {
81 const added = addComment(f.app, f.owner, repo, edge, `edge comment ${i}`);
82 if (!added.ok) throw new Error(String(added.error));
83 said.push(added.value.id);
84 // A close and a reopen after every fifth comment, so state lines sit between and after the edge.
85 if (i % 5 === 4) {
86 setIssueState(f.app, f.owner, repo, edge, "closed");
87 setIssueState(f.app, f.owner, repo, edge, "open");
88 }
89 }
90 const url = `${f.base}/${repo.ownerName}/${repo.name}/issues/${edge.number}`;
91 const page = await (await fetch(url, { headers: { cookie: f.ownerCookie } })).text();
92 const edgeId = said[10]!; // the 50th newest comment
93 expect(commentAnchor(f.app.db, edge.id, edgeId)).toBe(`#c-${edgeId}`);
94 expect(page).toContain(`id="c-${edgeId}"`);
95 const folded = said[9]!;
96 expect(commentAnchor(f.app.db, edge.id, folded)).toBe(`?all=1#c-${folded}`);
97 expect(page).not.toContain(`id="c-${folded}"`);
98 const all = await (await fetch(`${url}?all=1`, { headers: { cookie: f.ownerCookie } })).text();
99 expect(all).toContain(`id="c-${folded}"`);
100 });
101
102 test("a pull thread windows too, and the link keeps the page's other query", async () => {
103 const repo = f.repos[0]!;
104 const seeded = await commitToBranch(f.app, f.owner, repo, {
105 message: "seed",
106 files: [{ path: "README.md", content: "hello\n" }],
107 });
108 expect(seeded.ok).toBe(true);
109 const branched = await commitToBranch(f.app, f.owner, repo, {
110 branch: "feature",
111 message: "feature",
112 files: [{ path: "feature.txt", content: "yes\n" }],
113 });
114 expect(branched.ok).toBe(true);
115 const pull = await openPull(f.app, f.owner, repo, { title: "Feature", baseBranch: "main", headBranch: "feature" });
116 if (!pull.ok) throw new Error(String(pull.error));
117 for (let i = 0; i < 70; i++) {
118 const added = addComment(f.app, f.owner, repo, pull.value, `pull comment ${i}`);
119 if (!added.ok) throw new Error(String(added.error));
120 }
121 const url = `${f.base}/${repo.ownerName}/${repo.name}/pulls/${pull.value.number}`;
122 const page = await (await fetch(`${url}?diff=0`, { headers: { cookie: f.ownerCookie } })).text();
123 expect(page).toContain("Show 20 earlier items");
124 expect(page).toContain(`href="/${repo.ownerName}/${repo.name}/pulls/${pull.value.number}?diff=0&amp;all=1"`);
125 expect(page.match(/class="tl-item" id="c-/g)?.length).toBe(50);
126 const all = await (await fetch(`${url}?diff=0&all=1`, { headers: { cookie: f.ownerCookie } })).text();
127 expect(all.match(/class="tl-item" id="c-/g)?.length).toBe(70);
128 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs