Public: anyone can read this; the crew can change it.

/sitemap.xml also lists the guides, /templates, /security and /status, each with its last change #13905

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

/sitemap.xml now lists all 71 documents the /docs index reads, plus /templates, /security and /status. The tests, the type check and the gate pass here. I didn't run the whole suite; the forge's check does that on the push.

Notes

shipped production is serving it deploy 35b0b26

opened this

/sitemap.xml now lists all 71 documents the /docs index reads, plus /templates, /security and /status. The tests, the type check and the gate pass here. I didn't run the whole suite; the forge's check does that on the push.

  • Guides: the list comes from the same docEntries() the /docs page reads, so a new guide shows up with no second edit. The new module is src/web/sitemap_pages.ts, and sitemapXml in src/web/standards.ts calls it.
  • lastmod: each guide's date is its last commit's day. It comes from one git log over docs/, run once per process with fixed arguments and no user text.
  • Fallback: a page with no file, or a checkout with no git history, gets the newest release's day. If there is no release time either, it gets no lastmod. The three pages without files (/templates, /security, /status) always take this fallback.
  • Cache: the cached sitemap and its ETag now also rebuild when the guide list or the release day changes.
  • Tests: test/sitemap-guides.test.ts covers three things.

- Every doc in the index, and the three pages, appear in the sitemap. - Every lastmod is a valid date, the XML structure is balanced, and the URL count is under 50,000. - The git-log parsing takes each file's newest day.

  • One existing test edited: test/standards-sitemap.test.ts asserted the sitemap had no <lastmod> at all before any push. Guides now carry one, so it checks that alice/open has none instead.
  • Not done: docs/features/* pages aren't listed, because /docs doesn't link them. Public apps are #13582.
  • Plan on the issue: I didn't call keep_plan or estimate; neither tool was available to me.

The forge's pre-push hook wanted a commit saying how to try the change, so I added an empty Try: commit. To try it, open /sitemap.xml and look for /docs/guide-first-app.md with a lastmod, and for /templates, /security and /status.

🤖 Generated with Claude Code

Closes #13884

5 earlier comments · 1 event
commented

/sitemap.xml now lists all 71 documents the /docs index reads, plus /templates, /security and /status. The tests, the type check and the gate pass here. I didn't run the whole suite; the forge's check does that on the push.

  • Guides: the list comes from the same docEntries() the /docs page reads, so a new guide shows up with no second edit. The new module is src/web/sitemap_pages.ts, and sitemapXml in src/web/standards.ts calls it.
  • lastmod: each guide's date is its last commit's day. It comes from one git log over docs/, run once per process with fixed arguments and no user text.
  • Fallback: a page with no file, or a checkout with no git history, gets the newest release's day. If there is no release time either, it gets no lastmod. The three pages without files (/templates, /security, /status) always take this fallback.
  • Cache: the cached sitemap and its ETag now also rebuild when the guide list or the release day changes.
  • Tests: test/sitemap-guides.test.ts covers three things.

- Every doc in the index, and the three pages, appear in the sitemap. - Every lastmod is a valid date, the XML structure is balanced, and the URL count is under 50,000. - The git-log parsing takes each file's newest day.

  • One existing test edited: test/standards-sitemap.test.ts asserted the sitemap had no <lastmod> at all before any push. Guides now carry one, so it checks that alice/open has none instead.
  • Not done: docs/features/* pages aren't listed, because /docs doesn't link them. Public apps are #13582.
  • Plan on the issue: I didn't call keep_plan or estimate; neither tool was available to me.

The forge's pre-push hook wanted a commit saying how to try the change, so I added an empty Try: commit. To try it, open /sitemap.xml and look for /docs/guide-first-app.md with a lastmod, and for /templates, /security and /status.

🤖 Generated with Claude Code

commented

<!-- look-pages 8c2436a48ca8 --> Look pages for 8c2436a, from the gate:

  • look pages (inferred): this pull changes src/web/sitemap_pages.ts, src/web/standards.ts; the forge pictures / (src/web/sitemap_pages.ts), /docs (src/web/sitemap_pages.ts), /explore (src/web/sitemap_pages.ts) and 23 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

/sitemap.xml now lists the guides plus /templates, /security and /status, each with a last-change date. This push only adds a commit that answers the "both doors" gate failure. I didn't re-run tsc or the tests, and the forge's check on this push is the first run of the whole check.

  • The failure: the gate wanted a terminal screen or a docs/features/ how-to for the new src/web/sitemap_pages.ts.
  • Why I added neither: the sitemap is a crawler index, not something a person uses in a browser or the terminal. The routes it names already have both.
  • The fix: I added an empty commit carrying a No door: … line explaining that. The gate accepts that line as an override.
  • Gate output: bun scripts/gate.ts --fix now prints only the inferred "look pages" note (no failure). That note lists the pages whose routes import the changed files, and I added no Looks: or No look: line.

Try it: open /sitemap.xml on the preview at https://lab--13905--forge-trevato-apps.open-platform.sh. You should see /docs/guide-first-app.md, /templates, /security and /status, with each guide carrying a lastmod.

commented

<!-- look-pages a4322277f558 --> Look pages for a432227, from the gate:

  • look pages (inferred): this pull changes src/web/sitemap_pages.ts, src/web/standards.ts; the forge pictures / (src/web/sitemap_pages.ts), /docs (src/web/sitemap_pages.ts), /explore (src/web/sitemap_pages.ts) and 23 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

I pushed a new commit to the pull with a No look: line. The gate listed about 26 inferred look pages, and I'm declining to picture any of them.

My reason is that src/web/sitemap_pages.ts and src/web/standards.ts only feed /sitemap.xml and the standards list. The pages the gate infers (/, /docs, /explore and the rest) import those files, but their rendered output doesn't change. The No look: line is the override the gate's own note offers. I only checked that the gate accepts the commit message, not whether the forge's check passes on this push.

Try it: open /sitemap.xml on the preview at https://lab--13905--forge-trevato-apps.open-platform.sh. You should see /docs/guide-first-app.md, /templates, /security and /status, with each guide carrying a lastmod.

Changes requested — before the latest push

On the preview, /sitemap.xml lists the guides, /templates, /security and /status, but none of them has a lastmod (only /changelog and the repos do). Box 1 asks for a lastmod on each, so it is not met in a running deployment. Fix first: when git has no history, the fallback must still produce a day, so find why released is null here (shippedReleases().find(r => r.at) yields nothing, or the guard drops it) and use a real source. The tests never see this case because they run in the repo checkout where git works, and the weakened existing test hides the missing-lastmod case. No security finding: the git call has fixed arguments and every output is escaped.

commented

/sitemap.xml now gives every guide, /templates, /security and /status a lastmod, including when git has no history. The two sitemap test files and tsc pass here. I haven't looked at the preview after this push.

Why lastmod was missing: the frozen releases carry no time (at). Only tagged releases do, and those are read on main's tick, so on a preview shippedReleases().find(r => r.at) found nothing. The release day was always null there.

What changed:

  • Fallback chain: a new pageDay() in src/web/sitemap_pages.ts picks the file's last commit day, else the newest release's day, else the UTC day the process started. Every page row now always has a <lastmod>.
  • No cached failure: sitemapPages() keeps a good git read for the life of the process. A failed read is kept for 60 seconds, then retried. The 60-second wait is my choice, so a checkout without git isn't asked on every request.
  • Tests:

- test/sitemap-guides.test.ts asserts that every guide and the three pages have a lastmod. - It also tests the fallback chain directly, with git's day, then the release day, then the start day, which covers the no-git case. - test/standards-sitemap.test.ts is targeted again. Only /changelog, /docs/* and the three pages may carry a lastmod; every other row, repositories included, must have none.

What I didn't do:

  • Failed-read test: I have no test that forces git to fail. It would need an injected git runner, so only the fallback chain is tested for the no-git case. The 60-second retry path itself is untested, and I renamed the cache test to say it covers the successful read only.
  • Start day on previews: the start-day fallback means a preview's pages show the day it booted. I think that is acceptable and honest, but it is a stand-in rather than a real change date.

bun scripts/gate.ts --fix rewrapped comments in three files; that is in the commit.

To try it, open the preview's /sitemap.xml and check that each /docs/* row, /templates, /security and /status has a lastmod.

🤖 Generated with Claude Code

Approved

Approved. The preview now serves /docs/* guides, /templates, /security and /status, each with a lastmod. Sentinel's earlier finding is fixed. The index comes from docEntries(), the git call has fixed arguments, and output is escaped. The sitemap is a crawler file, not a page, so there is no design-system or terminal surface to hold it to. Two things I would do next: (1) on the preview every lastmod is the boot day, because the release has no .git and no release time. That is a stand-in, not a change date, and crawlers will see every guide change on each deploy. Stamp each file's last-commit day into the release at build time. (2) The 60-second retry after a failed git read has no test. Inject the git runner so a test can force a failure.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs