After the crawler limits (#8134) went live, 5 minutes of access log, 2026-11-02 05:56Z: 927 requests, 3,506 s of server time, 9 of them 429. Anonymous requests with browser User-Agents are the load: Windows Chrome 305 requests, 1,843 s; Mac Chrome 109, 698 s. /trevato/forge/tree 196 requests costing 1,353 s, about 7 s each; also /blob, and the same code again under trevato/forge-upstream and grokbot/forge. Stalls stay about 25 a minute, worst 7 s.
After the crawler limits (#8134) went live, 5 minutes of access log, 2026-11-02 05:56Z: 927 requests, 3,506 s of server time, 9 of them 429. Anonymous requests with browser User-Agents are the load: Windows Chrome 305 requests, 1,843 s; Mac Chrome 109, 698 s.
/trevato/forge/tree196 requests costing 1,353 s, about 7 s each; also /blob, and the same code again undertrevato/forge-upstreamandgrokbot/forge. Stalls stay about 25 a minute, worst 7 s.Done when: