Public: anyone can read this; the crew can change it.
Big repos always clone: transfers end on stalled progress, not a 5-minute clock, and partial clone works #4501
Check · Review · Land · Live done — Landed and live in production.
Clones of `upload-pack`, over HTTP and SSH, now end when they stop moving instead of on a 5- or 10-minute clock. Upload-pack also serves partial clones, so `git clone --filter=blob:none <url>` works with stock git.
shipped production is serving it deploy 8ae6610
Clones of upload-pack, over HTTP and SSH, now end when they stop moving instead of on a 5- or 10-minute clock. Upload-pack also serves partial clones, so git clone --filter=blob:none <url> works with stock git.
Transfers end on a stall (src/git/progress.ts, new). transferDeadline is the one deadline both smart-http.ts and smart-ssh.ts now use:
- Fetch: ended once fewer than 16 KiB reach the client in 30 s, or after a 2 h cap. Git's own keep-alive (
0005\x01, sent every 5 s while it prepares the pack) counts as moving, so a quiet pack-objects phase isn't taken for a stall. - Push: keeps its 10-minute clock. After the pack arrives, index-pack and the hooks run with nothing to send, and that quiet isn't a stall.
- Strangers:
transferTimeoutMsgoes from 300 000 to 7 200 000 ms to match the new cap. Their pool share, per-address limit and 30 s negotiation idle are unchanged.
What I found along the way. Bun reads a child's stdout as fast as the child writes it, even with no reader: 300 MB lands in memory in about a second. So git usually exits early, with the pack buffered in the forge, which means the old kill timer rarely hit a running git. Two consequences in this pull:
- The deadline follows the stream the client reads, not git's process. Ending it kills git and cuts the stream, which frees what was buffered for a client that stopped reading.
- The meter hands the buffer on in 64 KiB pieces and counts each piece when the client takes it, so the count follows the client.
A slow client that keeps reading still holds its whole pack in memory. That may be part of why trevato/arm clones were reset (a memory kill resets every connection), but I couldn't check production. I filed it as #4502 and linked the epic #4481 to wait on it.
Partial clone. serviceCommand() in smart-http.ts builds upload-pack's command for both transports with -c uploadpack.allowFilter=true -c uploadpack.allowAnySHA1InWant=true -c uploadpack.keepAlive=5. Using -c keeps gitEnv's blanked global and system config blank. Protocol v2 already accepts a want for any object id; this gives v0 clients the same. A lazy blob fetch passes the same read check as a clone, and forks never borrow objects from a private parent.
Tests. bun x tsc --noEmit is clean. test/big-clone.test.ts (new) plus the git, strangers and cross suites: 216 pass, 0 fail, 16 skipped. The SSH suites skip here because this machine has no ssh client; the check runs them.
- Scaled down: the issue asked for a 200 MB repo through a 1 MB/s throttle. The test uses a 4 MiB pack through a 1 MiB/s link with a 1 s window, so the clone lasts several windows.
- Big-clone cases:
- the throttled clone finishes and its bytes match;
- the same clone under a 2 s wall clock fails;
- a client that stops reading for three windows gets cut off with early EOF, while a pause shorter than a window doesn't;
- --filter=blob:limit=1m leaves the big blob missing and reset --hard fetches it, over v0 and v2;
- --filter=blob:none clones and checks out.
- Read check (real app): a stranger can't fetch a private repo's blob by id over v0 or v2, a partial clone of a public repo can't lazily fetch a private repo's object, and a reader's partial clone of the private repo fetches it lazily.
- SSH (
test/git-ssh.test.ts): a guest's partial clone over v0 and v2 with lazy checkout, and a guest refused a private blob by id. - No existing test was changed.
Docs. docs/GIT.md (new) covers cloning a big repo and when a clone ends; the README links it next to the SSH clone line.
To try it: git clone --filter=blob:none https://lab--<pull>--forge-trevato-apps.open-platform.sh/<owner>/<repo>.git, then git checkout.
🤖 Generated with Claude Code
Approved. This does what #4482 asks. Fetches over HTTP and SSH now end when they stall (under 16 KiB in 30 s) or after a 2 h cap, instead of on a 5- or 10-minute clock. Pushes keep their 10-minute clock. The stranger pool share, the per-address limit and the 30 s negotiation idle are unchanged. uploadpack.allowFilter and allowAnySHA1InWant are passed with -c, so gitEnv's blanked config stays blank, and the argv is fixed with no user text in it.
Security: access is still decided before upload-pack spawns, and the lazy by-id fetch goes through that same check. Tests cover it: a stranger is refused a private blob by id over HTTP v0 and v2 and over SSH, and a public repo's partial clone cannot reach a private object. The stream is cut on a stall, which frees the buffer held for a client that stopped reading, so this is tighter than before. Under the old code the Bun-buffered stdout outlived the kill.
Tests: the meter, the stall, the cap and the keep-alive each have unit tests. The throttled clone is tested against both the stall rule and the wall clock, and so are the pause and stall cases and the filtered clone over v0 and v2. Each would fail without the change. Docs are in docs/GIT.md, linked from the README. The preview answers / (200). It has no public repository to clone, so partial clone could not be tried end to end there. The tests cover it against real git.
One low point, not blocking: see the inline comment on the fork alternates exposure.
Clones of
upload-pack, over HTTP and SSH, now end when they stop moving instead of on a 5- or 10-minute clock. Upload-pack also serves partial clones, sogit clone --filter=blob:none <url>works with stock git.Transfers end on a stall (
src/git/progress.ts, new).transferDeadlineis the one deadline bothsmart-http.tsandsmart-ssh.tsnow use:0005\x01, sent every 5 s while it prepares the pack) counts as moving, so a quiet pack-objects phase isn't taken for a stall.transferTimeoutMsgoes from 300 000 to 7 200 000 ms to match the new cap. Their pool share, per-address limit and 30 s negotiation idle are unchanged.What I found along the way. Bun reads a child's stdout as fast as the child writes it, even with no reader: 300 MB lands in memory in about a second. So git usually exits early, with the pack buffered in the forge, which means the old kill timer rarely hit a running git. Two consequences in this pull:
A slow client that keeps reading still holds its whole pack in memory. That may be part of why trevato/arm clones were reset (a memory kill resets every connection), but I couldn't check production. I filed it as #4502 and linked the epic #4481 to wait on it.
Partial clone.
serviceCommand()insmart-http.tsbuilds upload-pack's command for both transports with-c uploadpack.allowFilter=true -c uploadpack.allowAnySHA1InWant=true -c uploadpack.keepAlive=5. Using-ckeepsgitEnv's blanked global and system config blank. Protocol v2 already accepts a want for any object id; this gives v0 clients the same. A lazy blob fetch passes the same read check as a clone, and forks never borrow objects from a private parent.Tests.
bun x tsc --noEmitis clean.test/big-clone.test.ts(new) plus the git, strangers and cross suites: 216 pass, 0 fail, 16 skipped. The SSH suites skip here because this machine has no ssh client; the check runs them.- the throttled clone finishes and its bytes match; - the same clone under a 2 s wall clock fails; - a client that stops reading for three windows gets cut off with
early EOF, while a pause shorter than a window doesn't; ---filter=blob:limit=1mleaves the big blob missing andreset --hardfetches it, over v0 and v2; ---filter=blob:noneclones and checks out.test/git-ssh.test.ts): a guest's partial clone over v0 and v2 with lazy checkout, and a guest refused a private blob by id.Docs.
docs/GIT.md(new) covers cloning a big repo and when a clone ends; the README links it next to the SSH clone line.To try it:
git clone --filter=blob:none https://lab--<pull>--forge-trevato-apps.open-platform.sh/<owner>/<repo>.git, thengit checkout.🤖 Generated with Claude Code
Closes #4482