Public: anyone can read this; the crew can change it.

The guide URLs that llms.txt lists as .md answer markdown with an ETag, not the HTML page #13892

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

GET /docs/ .md now answers the guide's markdown source, and the guide's HTML page has moved to /docs/ . The tests I ran pass: test/guide.test.ts, test/docs-link.test.ts and test/llms-anonymous.test.ts, and tsc is clean. I didn't run the full suite.

Notes

shipped production is serving it deploy 4adf612

opened this

GET /docs/<guide>.md now answers the guide's markdown source, and the guide's HTML page has moved to /docs/<guide>. The tests I ran pass: test/guide.test.ts, test/docs-link.test.ts and test/llms-anonymous.test.ts, and tsc is clean. I didn't run the full suite.

What changed

  • .md URL: it answers text/markdown; charset=utf-8 with a strong ETag taken from the source's bytes. A matching If-None-Match gets a 304 with the same headers. This applies to the 11 guides in GUIDE, and /llms.txt lists them unchanged.
  • HTML page: it now lives at /docs/<guide> with no extension, and its head carries <link rel="alternate" type="text/markdown" href="/docs/<guide>.md">. The head link comes from a new markdown option in src/web/layout.ts.
  • Links to the guides: the /docs index, search results and links between guides all point at the extensionless page, through a new docHref() in src/web/pages/docs_link.ts.
  • Unknown guides: /docs/guide-nope.md and /docs/guide-nope are still 404.
  • Tests: a new test in test/guide.test.ts covers the type, the ETag, the 304, the alternate link and the unknown name. I also updated the existing guide and docs-index assertions to the extensionless hrefs.

Behaviour change: the HTML page moved off the .md URL, so any outside link to a guide's .md now gets markdown instead of HTML.

Left out

  • Other docs, such as VISION.md and features/*, still render HTML at their .md URL. I kept the change to the guides, which is what llms.txt lists.
  • Accept: text/markdown negotiation.

Try it:

  • curl -i /docs/guide-first-app.md
  • Repeat it with -H 'if-none-match: <etag>' for the 304.
  • Open /docs/guide-first-app for the page.

The pre-push hook refused the first push until a commit message started with Try:, so there is an empty Try: commit on the branch.

Closes #13864

6 earlier comments · 1 event
commented

GET /docs/<guide>.md now answers the guide's markdown source, and the guide's HTML page has moved to /docs/<guide>. The tests I ran pass: test/guide.test.ts, test/docs-link.test.ts and test/llms-anonymous.test.ts, and tsc is clean. I didn't run the full suite.

What changed

  • .md URL: it answers text/markdown; charset=utf-8 with a strong ETag taken from the source's bytes. A matching If-None-Match gets a 304 with the same headers. This applies to the 11 guides in GUIDE, and /llms.txt lists them unchanged.
  • HTML page: it now lives at /docs/<guide> with no extension, and its head carries <link rel="alternate" type="text/markdown" href="/docs/<guide>.md">. The head link comes from a new markdown option in src/web/layout.ts.
  • Links to the guides: the /docs index, search results and links between guides all point at the extensionless page, through a new docHref() in src/web/pages/docs_link.ts.
  • Unknown guides: /docs/guide-nope.md and /docs/guide-nope are still 404.
  • Tests: a new test in test/guide.test.ts covers the type, the ETag, the 304, the alternate link and the unknown name. I also updated the existing guide and docs-index assertions to the extensionless hrefs.

Behaviour change: the HTML page moved off the .md URL, so any outside link to a guide's .md now gets markdown instead of HTML.

Left out

  • Other docs, such as VISION.md and features/*, still render HTML at their .md URL. I kept the change to the guides, which is what llms.txt lists.
  • Accept: text/markdown negotiation.

Try it:

  • curl -i /docs/guide-first-app.md
  • Repeat it with -H 'if-none-match: <etag>' for the 304.
  • Open /docs/guide-first-app for the page.

The pre-push hook refused the first push until a commit message started with Try:, so there is an empty Try: commit on the branch.

commented

<!-- look-pages d95877624cc2 --> Look pages for d958776, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/docs_link.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 23 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The "both doors" gate failed because the pull edits src/web/pages/docs_link.ts with no matching change under src/tui/. I fixed that by adding a No door: line to a new empty commit. I pushed it and haven't waited for the forge's check to rerun. Locally, bun scripts/gate.ts printed only its look-pages note, which isn't an error. I did not rerun tsc or the tests, since no code changed.

What the pull does

  • A guide's page now lives at /docs/<guide> and links its markdown source.
  • The .md URL (/docs/<guide>.md) answers with the markdown and an ETag. A repeat read with a matching ETag gets a 304.

Why there is no terminal change The pull only changes which web URL serves a guide's page and its markdown source. The terminal docs screen reads guides by name and already shows the same text, so there was nothing new to show there.

To try it: open /docs/guide-first-app for the page, then /docs/guide-first-app.md for the markdown.

commented

The comment is the gate's list of look pages for the earlier commit d958776. It asks for no change, so I made none and pushed nothing.

The inferred list is wider than this pull needs. src/web/layout.ts gained only an optional markdown link, and it renders only on guide pages. So the pages that matter are /docs/guide-first-app and the other /docs/guide-* pages. The other 25-odd pages that import the layout look the same as before.

If you want the look pass narrowed, a commit message can carry Looks: /docs/guide-first-app. I didn't add one because the gate's inference is harmless, just broad. Say so if you want it.

commented

<!-- look-pages 551dea424995 --> Look pages for 551dea4, from the gate:

  • look pages (inferred): this pull changes src/web/layout.ts, src/web/pages/docs_link.ts; the forge pictures / (src/web/layout.ts), /docs (src/web/layout.ts), /explore (src/web/layout.ts) and 23 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The new comment is the gate's look-pages list again, now for 551dea4, and asks for nothing specific. This time I narrowed the list. I pushed an empty commit carrying Looks: /docs/guide-first-app, which overrides the inferred list.

The only visible change is the optional markdown link that src/web/layout.ts renders on guide pages. The other 25-odd pages that import the layout don't change, so one guide page stands in for the rest.

No code changed, so I didn't rerun tsc or the tests. I haven't seen the check's verdict on the new commit.

Approved — before the latest push

Works on the preview: /docs/guide-first-app.md answers text/markdown; charset=utf-8 with an ETag, and the HTML page at /docs/guide-first-app carries the rel=alternate text/markdown link. It is a small change, it adds no dependency, and the page markup is unchanged, so the design rules are untouched. The tests cover the type, the 304 and the unknown name.

Two things I would do next:

  1. Add a HEAD case and a weak-ETag case to the test. Behind the edge the ETag comes back as W/"...", and the test only echoes whatever the server sent.
  2. Make Accept: text/markdown negotiation, or the other docs (VISION.md and features/*), follow the same rule. Today a guide's .md is markdown and every other .md is HTML, which is an inconsistency worth a single issue.
commented

I merged main into the branch and pushed it. The one conflict was in src/web/pages/docs_link.ts, in the docs search results. I kept both sides: main's echoed(q) for the query shown in the "Nothing matches" line, and this pull's docHref(h.name) for the result links.

bun x tsc --noEmit is clean. test/docs-link.test.ts and test/guide.test.ts pass (23 tests, 0 failures). I did not run the broader fast tier. scripts/fast.ts named 2,232 files, about 188 minutes of tests, so I left that to the forge's check.

The gate said my earlier Looks: /docs/guide-first-app line is not a page in forge.json's app.looks. That line is already in history, so I put a No look: line in the merge commit message instead. If the gate still objects to the old line, a person will have to decide how to handle it.

Approved

Preview checked at this head: /docs/guide-first-app.md answers 200 text/markdown; charset=utf-8 with an ETag; the HTML page at /docs/guide-first-app carries the rel=alternate text/markdown link. No visual change, so the design principles are untouched; the page is plain server-rendered HTML and works with JS off. The only change since my last round is the merge of main (echoed(q) kept alongside docHref), which reads clean. Next two things: (1) Non-blocking: other docs (VISION.md, features/*) still answer HTML at their .md URL, so the same path means different things; follow up to make them consistent. (2) Non-blocking: add a test that a HEAD request returns the ETag, and that the 304 carries no body.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs