Public: anyone can read this; nobody can change it.

Pack transfers at once follow the host's memory, so a 512 MiB cell does not run sixteen pushes and their hooks past its limit #61

Merged opened this view on origin

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

**Failing:** `test/agents.test.ts` › *fifty agents at once › fifty pushes to distinct branches all complete over the hook socket*. It timed out after 180000 ms, and the pushes were rejected with `remote: forge: hook relay failed (TimeoutError: The operation timed out.)` / `! [remote rejected] agent-N (pre-receive hook declined)` / `curl 18 transfer closed`.

shipped merged

opened this view on origin

Failing: test/agents.test.ts › fifty agents at once › fifty pushes to distinct branches all complete over the hook socket. It timed out after 180000 ms, and the pushes were rejected with remote: forge: hook relay failed (TimeoutError: The operation timed out.) / ! [remote rejected] agent-N (pre-receive hook declined) / curl 18 transfer closed.

Runner: plat-standby-forgejo-runner-isolated-598cbbb49c-jvj8r (memory.max 512 MiB, swap 66 MiB, cpu.max 50000/100000, nproc 4). mem_peak reached the limit in every run there.

Not caused by #49. gate (1) results for this test, by runner:

  • vxrail (16 GiB): 3–6 s in every run.
  • isolated: passes in 40–56 s on main 6ec6c1c9, 18917b93, 006d3977 and bde2a088, and on #46 ee9e9234, whose tree is identical to main cfb4cf16 (run 1065, 41.4 s). It fails at the 120 s relay timeout on #42 (1042, 1046) and on #49 (1072, 1074). The outcome is close to a coin flip on that runner, and the code under test does not change it: #49's diff touches nothing on the push path. Locally, bde2a088, cfb4cf16 and 0f4fc22b all run it in about 2.3 s.

Root cause: the receive-pack transfer gate is fixed at 16, whatever the memory. Each push holds receive-pack plus a pre-receive hook, which is a whole Bun runtime. Measured locally at 16: 13 hooks and 14 receive-packs held at once, 320 MB RSS. The fifty clients come to about 1 GB RSS but only about 120 MB PSS, because most of that is shared pages. So on a 512 MiB cell, the server side is what pushes memory past the limit.

Fix: the default is one transfer per 64 MiB of the cgroup/host memory, clamped to 2–16 (memoryLimitBytes). That gives 8 on 512 MiB and leaves every host with 1 GiB or more at 16. FORGE_GIT_TRANSFER_CONCURRENCY still overrides it. A queued push waits for its permit before its hook starts, so the wait does not count against the relay's deadline. With 8, the measured peak is 7 hooks and 8 receive-packs, 155 MB, and the time is unchanged (2.1 s). The fifty-push test is unchanged, and a unit test covers transfersForMemory. tsc --noEmit is clean, and agents.test.ts is 29/29.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs