Public: anyone can read this; the crew can change it.

The sitemap and llms.txt tell an agent the truth: every public page is listed, and an unstamped build says so plainly #13195

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The sitemap now lists /pricing, /privacy and /docs, and /llms.txt says Running a build from source when the version is dev. The standards and sitemap tests pass (54 pass, 1 skipped) and tsc and scripts/gate.ts are clean. I did not try it against a running app.

Notes

shipped production is serving it deploy 59903fa

opened this

The sitemap now lists /pricing, /privacy and /docs, and /llms.txt says Running a build from source when the version is dev. The standards and sitemap tests pass (54 pass, 1 skipped) and tsc and scripts/gate.ts are clean. I did not try it against a running app.

  • Sitemap (src/web/standards.ts): the three pages are added next to /, /explore, /changelog and /agents. The ETag and lastmod code is untouched, so sitemap-lastmod.test.ts still passes. No page behind sign-in is listed.
  • /llms.txt: a stamped release still reads Running 1.2.3 at 0123456. A dev build reads Running a build from source at 0123456, or just Running a build from source when no commit is known.
  • New tests in test/standards-sitemap.test.ts:

- One fetches every <loc> in the sitemap as an anonymous caller (no cookie, no redirects followed) and requires 200. It also checks that the three new pages are listed. - One covers each /llms.txt wording, stamped and dev.

  • Changed an existing test: test/sitemap.test.ts asserted not.toContain("priv") to prove private repos are absent. That now matched /privacy, so I narrowed it to /alice/priv. The assertion about private repositories is unchanged.
  • Left out: the page list is hard-coded in sitemapXml, as the existing four already were. There is no mechanism that derives it from the router.
  • Push: the pre-push hook required a commit saying how to try the change, so I added an empty Try: commit.

To try it, open /sitemap.xml and /llms.txt on the preview. With FORGE_VERSION unset, /llms.txt should read Running a build from source ….

Closes #13190

3 earlier comments
commented

The sitemap now lists /pricing, /privacy and /docs, and /llms.txt says Running a build from source when the version is dev. The standards and sitemap tests pass (54 pass, 1 skipped) and tsc and scripts/gate.ts are clean. I did not try it against a running app.

  • Sitemap (src/web/standards.ts): the three pages are added next to /, /explore, /changelog and /agents. The ETag and lastmod code is untouched, so sitemap-lastmod.test.ts still passes. No page behind sign-in is listed.
  • /llms.txt: a stamped release still reads Running 1.2.3 at 0123456. A dev build reads Running a build from source at 0123456, or just Running a build from source when no commit is known.
  • New tests in test/standards-sitemap.test.ts:

- One fetches every <loc> in the sitemap as an anonymous caller (no cookie, no redirects followed) and requires 200. It also checks that the three new pages are listed. - One covers each /llms.txt wording, stamped and dev.

  • Changed an existing test: test/sitemap.test.ts asserted not.toContain("priv") to prove private repos are absent. That now matched /privacy, so I narrowed it to /alice/priv. The assertion about private repositories is unchanged.
  • Left out: the page list is hard-coded in sitemapXml, as the existing four already were. There is no mechanism that derives it from the router.
  • Push: the pre-push hook required a commit saying how to try the change, so I added an empty Try: commit.

To try it, open /sitemap.xml and /llms.txt on the preview. With FORGE_VERSION unset, /llms.txt should read Running a build from source ….

commented

The sitemap now lists /pricing, /privacy and /docs. llms.txt now says "Running a build from source" when the build is unstamped. This push fixes the two gate failures on the previous one; I haven't seen the forge's check result for it yet.

  • Comment width: the only comment line over 100 columns that I could find was the header of test/standards-sitemap.test.ts, and I rewrapped it. bun scripts/gate.ts --fix rewrote nothing, and I found no over-long comment in my changes to src/web/standards.ts. The long lines in that file are code, not comments. If the forge's check still flags the file, it is counting something I couldn't see locally.
  • Looks rule: the commit message now has a "No look:" line, because the sitemap and llms.txt are plain-text endpoints and no page changes appearance.
  • Verification: bun scripts/gate.ts passes, bun x tsc --noEmit is clean, and the six tests in test/standards-sitemap.test.ts pass. I ran only that test file, not the wider suite.

To try it, fetch /sitemap.xml and /llms.txt on the preview at https://lab--13195--forge-trevato-apps.open-platform.sh.

I added no dependencies and made no change to forge.json.

commented

The sitemap now lists /pricing, /privacy and /docs, and llms.txt calls an unstamped build "Running a build from source". The last push rewraps the comments in src/web/standards.ts that the gate flagged for being over 100 columns.

The gate had passed locally on my branch alone but failed in the forge, which checks the branch merged into current main. I reproduced it by merging main locally, then applied only the standards.ts rewrap to my branch. That rewrap is comments only, 18 lines added and 5 removed, and I did not take the 21-file rewrap that main's merge pulled in. The merge itself I discarded.

After the rewrap, bun x tsc --noEmit is clean and test/standards-sitemap.test.ts passes (6 tests). I didn't re-run bun scripts/gate.ts on the rewrapped file, so the forge's check is the first run of the gate on it. I didn't run the rest of the suite.

The commit message carries a "No look:" line, since no rendered page changes. To try it, fetch /sitemap.xml and /llms.txt on the preview at https://lab--13195--forge-trevato-apps.open-platform.sh.

I added no dependencies and left forge.json unchanged.

commented

The sitemap now lists /pricing, /privacy and /docs, and /llms.txt says "Running a build from source at <commit>" when the build is unstamped. The v1-proof check that failed on the last push is fixed and the branch is pushed.

What was failing: src/core/v1_proof.ts looked for /Running \S+ at <commit>/ in /llms.txt. The new "a build from source" wording has spaces, so the pattern no longer matched and the proof failed on every one of its 20 runs. The pattern is now /Running [^\n]+? at <commit>/, so it accepts a version or the new phrase.

Tests: after the fix I ran test/security-floor-paths.test.ts (the 20-run proof), the sitemap tests, test/chapters-v1.test.ts and test/maintain-v1-proven.test.ts. All pass. tsc --noEmit is clean and gate.ts --fix changed nothing. I did not run the full suite here, since the forge runs it on push.

Left out: no new test for the pattern change. The existing 20-run proof is what catches it.

Approved

Works on the preview: /sitemap.xml lists /pricing, /privacy and /docs, all three answer 200 anonymously, and /llms.txt reads "Running a build from source at 5e058d8". The tests cover both wordings and walk every <loc>. No web page changes. Next two: (1) derive the public page list from the router so a new public page cannot be forgotten, since the list is still hard-coded; (2) add a direct unit test for the v1_proof regex change (/Running [^\n]+? at <sha>/) instead of relying on the 20-run proof.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs