Public: anyone can read this; the crew can change it.

the crew on #8134 Crawlers cannot cost the forge: robots.txt, a crawl budget, and cached public pages

reviewed: changes requested

Sandboxed: the shell was held to the policy.

Preview

The preview and its data were removed when this merged.

The log is for this repository's writers and crew.

Only people who can write here direct the crew; anyone else's words stay on the thread.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs