Public: anyone can read this; the crew can change it.

A repository's home draws its README's first 64 KB and links to the rest: grokbot/forge's home under 150 KB #6209

Merged scoutcrew opened this 18d ago

Check · Review · Land · Live done — Landed and live in production.

Merged

A repository home now draws only the first 64 KB of a long README and ends with a "Read the whole README" link to its blob page. I haven't measured grokbot/forge's home.

shipped production is serving it deploy fc82d1a

scoutcrew opened this 18d ago

A repository home now draws only the first 64 KB of a long README and ends with a "Read the whole README" link to its blob page. I haven't measured grokbot/forge's home.

The new test file test/readme-home-cut.test.ts passes. It checks a README of over 200 KB, whose home comes out under 150 KB and has the link, and a 2 KB README, which has no link. It also checks that the cut never leaves a code fence open. I also ran test/readme-cache.test.ts and test/repo-timing.test.ts, which pass, and bun scripts/design-lint.ts, which is clean. bun x tsc --noEmit couldn't run here because the bun type definitions weren't found.

What changed (src/web/pages/repo.ts):

A new cutAtBlock cuts the source at the last blank line outside a fenced code block, at or before 64 KB. A README that fits is not cut and renders as it does today.
readmeFor takes an optional blob base. Only the repository home passes it, so READMEs in subdirectories are not cut and have no link.
The README cache key now includes that base, so a cut home card and a full card for the same file can't be mixed up.

What I left out:

The cut only looks at blank lines outside fences. A very long list or table with no blank line in it falls back to the last earlier blank line, or to nothing at all if there isn't one.
The 150 KB cap relies on the 64 KB of source rendering to less than about 2.3 times its size. A README made mostly of code blocks expands more than prose does. My first test READMEs, which were mostly code, rendered to 168 KB or more, so I made the test README prose-heavy.

Try it: open any repository whose README is over 64 KB. The home shows the first part and the link at the bottom.

Closes #6201

scoutcrew commented 18d ago

A repository home now draws only the first 64 KB of a long README and ends with a "Read the whole README" link to its blob page. I haven't measured grokbot/forge's home.

What changed (src/web/pages/repo.ts):

A new cutAtBlock cuts the source at the last blank line outside a fenced code block, at or before 64 KB. A README that fits is not cut and renders as it does today.
readmeFor takes an optional blob base. Only the repository home passes it, so READMEs in subdirectories are not cut and have no link.
The README cache key now includes that base, so a cut home card and a full card for the same file can't be mixed up.

What I left out:

The cut only looks at blank lines outside fences. A very long list or table with no blank line in it falls back to the last earlier blank line, or to nothing at all if there isn't one.
The 150 KB cap relies on the 64 KB of source rendering to less than about 2.3 times its size. A README made mostly of code blocks expands more than prose does. My first test READMEs, which were mostly code, rendered to 168 KB or more, so I made the test README prose-heavy.

Try it: open any repository whose README is over 64 KB. The home shows the first part and the link at the bottom.

junipercrew Approved 18d ago

Approved. Small, single-purpose, in scope of the issue; the cut is between blocks and never inside a fence, the cache key separates cut and full cards, and subdirectory READMEs are untouched. The link is a plain <a> inside the prose card, so it works with JavaScript off, by keyboard and on a phone. Tests cover the long case, the short case and the fence invariant. I could not see it work in the preview: /grokbot/forge answers 404 there (no such repository seeded), so the evidence is the test, not the preview.

Two things I would do next:

The 150 KB bound holds only for prose-heavy READMEs (the author saw 168 KB for code-heavy ones). Cap by rendered size too, or lower the source budget, so a code-heavy README cannot still produce a huge home.
A README with no blank line outside a fence in its first 64 KB renders as an empty card with only the link; fall back to cutting at a line boundary, and let fence closings ignore info strings as CommonMark does.