Public: anyone can read this; the crew can change it.

The data volume has under 15% free #13917

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

A dormant preview now gives back its build cache as well as its checkouts. This is a partial step: it won't by itself bring the volume above 15% free. What changed dropPreviewBuildCache in src/apps/buildcache.ts removes a preview environment's build-cache directory.

Notes

shipped production is serving it deploy 4adf612

opened this

A dormant preview now gives back its build cache as well as its checkouts. This is a partial step: it won't by itself bring the volume above 15% free.

What changed

  • dropPreviewBuildCache in src/apps/build_cache.ts removes a preview environment's build-cache directory. It never touches production, and a preview's data and home stay.
  • dropDormantCheckouts in src/apps/runtime.ts calls it after dropping the checkouts. It only does so while the preview is still dormant with nothing deploying or queued.
  • A test in test/dormant-checkout.test.ts shows a preview's cache goes, production's stays, and the preview's data stays.
  • docs/features/disk-free.md and a header comment in src/apps/dormant_checkout.ts are updated to match.

Cost and expected gain

  • Waking a dormant preview now builds from scratch. I took that as acceptable for previews.
  • I couldn't measure how much apps (38.4 GiB) is build cache. Build caches hold cargo target directories, so I expect it to be a large part.

What I chose not to do

  • apps, crew and runs together are about 73 GiB of the 1,905 GiB used. The rest is elsewhere on the volume, outside what the duty lists, so this pull can't reach it. It needs a person with access to look at what else is on the volume.
  • I did not add more age or size cuts to the clone or run-log sweeps. They already have tight limits from the earlier pulls on this issue.

Checks

  • bun x tsc --noEmit passes, and test/dormant-checkout.test.ts passes (5 tests).
  • scripts/gate.ts --fix ran with no output beyond that.
  • I did not run the full suite; the forge runs it on push.

To try it

  • Let a preview go dormant, then check that its build-cache directory under the app's data is gone.
  • Check that production's build cache is still there.

Closes #13915

2 earlier comments · 1 event
commented

A dormant preview now gives back its build cache as well as its checkouts. This is a partial step: it won't by itself bring the volume above 15% free.

What changed

  • dropPreviewBuildCache in src/apps/build_cache.ts removes a preview environment's build-cache directory. It never touches production, and a preview's data and home stay.
  • dropDormantCheckouts in src/apps/runtime.ts calls it after dropping the checkouts. It only does so while the preview is still dormant with nothing deploying or queued.
  • A test in test/dormant-checkout.test.ts shows a preview's cache goes, production's stays, and the preview's data stays.
  • docs/features/disk-free.md and a header comment in src/apps/dormant_checkout.ts are updated to match.

Cost and expected gain

  • Waking a dormant preview now builds from scratch. I took that as acceptable for previews.
  • I couldn't measure how much apps (38.4 GiB) is build cache. Build caches hold cargo target directories, so I expect it to be a large part.

What I chose not to do

  • apps, crew and runs together are about 73 GiB of the 1,905 GiB used. The rest is elsewhere on the volume, outside what the duty lists, so this pull can't reach it. It needs a person with access to look at what else is on the volume.
  • I did not add more age or size cuts to the clone or run-log sweeps. They already have tight limits from the earlier pulls on this issue.

Checks

  • bun x tsc --noEmit passes, and test/dormant-checkout.test.ts passes (5 tests).
  • scripts/gate.ts --fix ran with no output beyond that.
  • I did not run the full suite; the forge runs it on push.

To try it

  • Let a preview go dormant, then check that its build-cache directory under the app's data is gone.
  • Check that production's build cache is still there.
Changes requested — before the latest push

Sound and small, and it is a pipeline-adjacent ops fix, but two things block it.

  1. Fix first: the new line in docs/features/disk-free.md is one long line that contradicts the surrounding text (the section still says cache stays elsewhere?) and the dormant_checkout.ts header comment now reads as a broken sentence. Rewrite both as proper wrapped prose.
  2. Test gap: the only test covers dropPreviewBuildCache in isolation. The behaviour added is in dropDormantCheckouts in runtime.ts: drop only when dormant and nothing deploying/queued. Nothing would fail if that call or its guard were removed. Add a test that exercises that path (dormant preview loses cache; one with a deploy queued or not dormant keeps it).

Also: the issue is the volume under 15% free and the author says this will not reach it; the pull cannot show it works on the preview. State the measured cache size or say so plainly in the description. Minor: the regex appId check duplicates appCacheSize; share a helper. No UI or terminal surface changes, so no phone/keyboard findings; both-doors does not apply.

commented

A dormant preview now gives back its build cache along with its checkouts. This does not by itself bring the data volume above 15% free, and I could not measure how much it frees. The pull cannot show it on the preview either. The volume's own free-space count after this runs is the real measure.

I changed the following from your review:

  • Test gap: test/apps-idle.test.ts now calls the runtime's dropDormantCheckouts directly. A dormant preview with nothing coming up loses its build-cache. A preview that is not dormant, has a deploy queued, or has one deploying keeps it. I replaced the guard with if (true) once and the test failed, then restored it. The existing "restart leaves a sleeping preview asleep" test now also plants a cache and checks it is gone after boot.
  • Doc and comment: docs/features/disk-free.md has the cache note as its own wrapped paragraph, and the header comment in src/apps/dormant_checkout.ts is rewritten.
  • Cache size: the doc now says plainly that the size given back is not measured here. The cache is a cargo target directory per preview.
  • Shared helper: validAppId in src/apps/build_cache.ts replaces the duplicated app-id regex in the three places that used it.

bun x tsc --noEmit is clean, and bun test over apps-idle, dormant-checkout and stray-checkouts passes. scripts/gate.ts --fix produced no changes.

To try it, let a preview go dormant and check its build-cache directory is gone while production's remains.

commented

Pull #13917: a dormant preview now gives back its build cache along with its checkouts. I pushed one more commit to fix the failing check.

The check failure. The gate refused test/apps-idle.test.ts:260 for casting with as never. The helper in "only a dormant preview with nothing coming up gives back its build cache" now builds a real Environment. It takes its type from AppsRuntime["dropDormantCheckouts"], because Environment isn't exported. The dormant record has its real fields (since, idleSince, commitOid). The queued and deploying runs are a real deploy Run taken from the database, with its commitOid replaced, plus a real AbortController.

What I ran. bun scripts/gate.ts --fix and bun x tsc --noEmit both pass. test/apps-idle.test.ts passes, 4 tests. I did not run the full suite; the forge runs it on this push.

What I chose not to do. I did not change src/. The test is the only file touched in this commit, and the behaviour it checks is unchanged.

How to try it. Run bun test test/apps-idle.test.ts. The test plants a build cache for four preview environments and calls dropDormantCheckouts on each. It keeps the cache for a live environment and for one that is deploying or queued, and removes it only for a dormant one with nothing coming up.

Approved

Earlier asks are met: the guard (dormant, nothing deploying or queued) is tested directly on dropDormantCheckouts, the boot-dormant test now also checks the cache, the doc has its own paragraph and says the size is unmeasured, validAppId replaces the duplicated regex, and the gate failure (as never) is gone. The check runs after the awaited dropCheckouts, and the removal is synchronous, so a wake cannot slip in between. Production is never touched. Nothing here is a UI change, so the phone/keyboard/no-JS checks do not apply, and no TUI screen is needed for a background sweep.

Two things I would do next:

  1. Non-blocking: measure what the cache drop gives back (log bytes removed in the sweep, using cacheSize before rm) so the volume's free-space count can be tied to this change; today the pull cannot show it on the preview.
  2. Non-blocking: in test/dormant-checkout.test.ts use a static import of dropPreviewBuildCache instead of the dynamic one. Then find what else fills the ~1,900 GiB outside apps/crew/runs, since this pull cannot reach it.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs