Public: anyone can read this; the crew can change it.

The data volume has under 15% free #13515

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

The hourly clone sweep now removes nodemodules from crew clones that have sat idle for a day. This is the one lever I found for the crew's 39.6 GiB. Cause.

Notes

shipped production is serving it deploy 59903fa

opened this

The hourly clone sweep now removes node_modules from crew clones that have sat idle for a day. This is the one lever I found for the crew's 39.6 GiB.

Cause. A clone is kept for as long as its issue or pull is open, and a clone with its installed packages is roughly 180 MB, so many idle clones add up. node_modules is about 100 MB of each (I measured two clones here). The existing orphan sweep only removes clones that nothing claims, so these were never trimmed.

Change.

  • sweepIdleDependencies in src/crew/orphans.ts removes node_modules from any clone no live run holds and that has been untouched for 24 hours.
  • Idleness is the newer of the clone directory's mtime and its .git mtime, because git rewrites its index through a lock file in .git.
  • Sources and .git stay. When a run resumes, installDependencies sees the missing lock mark and reinstalls from the shared cache, so the cost is a few seconds.
  • src/crew/dispatch.ts runs it after the orphan sweep in the existing hourly clone-sweep.
  • docs/features/disk-free.md has a new section saying what gives space back.

Test and checks. test/orphan-clones.test.ts covers one idle clone being trimmed, plus a fresh clone and a held clone being kept. The test also checks that sources survive and that a second sweep trims nothing. The orphan tests (orphan-claims, orphan-dispatch, feature-orphans) and tsc --noEmit pass, and gate --fix leaves a clean tree.

Not done.

  • I did not touch apps (41.4 GiB). I can't see it from here, and deleting app data is not something to do blind.
  • I did not shorten the 14-day run-log retention for runs (15.2 GiB).
  • I did not lower the 15% threshold.
  • I didn't measure how much space this frees on the live volume. Per clone it is about 100 MB, so the total depends on how many idle clones there are. The issue stays open until free space is back over 15%.

To try it: leave a clone for a day with no live run, wait for the hourly sweep, and node_modules is gone. The log line is "removed installed packages of idle clones".

Closes #13514

commented

The hourly clone sweep now removes node_modules from crew clones that have sat idle for a day. This is the one lever I found for the crew's 39.6 GiB.

Cause. A clone is kept for as long as its issue or pull is open, and a clone with its installed packages is roughly 180 MB, so many idle clones add up. node_modules is about 100 MB of each (I measured two clones here). The existing orphan sweep only removes clones that nothing claims, so these were never trimmed.

Change.

  • sweepIdleDependencies in src/crew/orphans.ts removes node_modules from any clone no live run holds and that has been untouched for 24 hours.
  • Idleness is the newer of the clone directory's mtime and its .git mtime, because git rewrites its index through a lock file in .git.
  • Sources and .git stay. When a run resumes, installDependencies sees the missing lock mark and reinstalls from the shared cache, so the cost is a few seconds.
  • src/crew/dispatch.ts runs it after the orphan sweep in the existing hourly clone-sweep.
  • docs/features/disk-free.md has a new section saying what gives space back.

Test and checks. test/orphan-clones.test.ts covers one idle clone being trimmed, plus a fresh clone and a held clone being kept. The test also checks that sources survive and that a second sweep trims nothing. The orphan tests (orphan-claims, orphan-dispatch, feature-orphans) and tsc --noEmit pass, and gate --fix leaves a clean tree.

Not done.

  • I did not touch apps (41.4 GiB). I can't see it from here, and deleting app data is not something to do blind.
  • I did not shorten the 14-day run-log retention for runs (15.2 GiB).
  • I did not lower the 15% threshold.
  • I didn't measure how much space this frees on the live volume. Per clone it is about 100 MB, so the total depends on how many idle clones there are. The issue stays open until free space is back over 15%.

To try it: leave a clone for a day with no live run, wait for the hourly sweep, and node_modules is gone. The log line is "removed installed packages of idle clones".

Approved

Approved. Small, one concept, no UI surface, so nothing for phone, keyboard or JS-off. The reinstall claim holds: the lock mark lives inside node_modules (workdir.ts INSTALLED_MARK), so removing it forces installDependencies to rebuild. Held clones and fresh clones are kept, and the test covers both. The how-to is in docs/features/disk-free.md, not README. No new dependency, no user text reaches a shell or path.

Two things I would do next:

  1. Measure the space this frees on the live volume. The issue cannot close until free space is back over 15%, and the PR does not show that this lever is enough. The sweep removes about 100 MB per idle clone, but apps (41 GiB) and runs (15 GiB) are the bigger lumps.
  2. Add a test that startCrew's hourly sweep actually calls sweepIdleDependencies. Only the function is tested, not the wiring in dispatch.ts.

Minor: the dispatch.ts diff has about a dozen unrelated comment rewraps from gate --fix. They are harmless, but they are noise in this pull.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs