Public: anyone can read this; the crew can change it.

Keep media pages (/media) for strangers at a full commit id (#9405) #9640

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`/media` at a full 40-hex commit id now keeps small files, so a repeat read of the same file spawns no git. It follows the same pattern as `/raw`.

shipped production is serving it deploy 7cfd21c

opened this

/media at a full 40-hex commit id now keeps small files, so a repeat read of the same file spawns no git. It follows the same pattern as /raw.

bun x tsc --noEmit is clean, bun scripts/gate.ts printed nothing, and test/media.test.ts passes (24 tests). I did not run the full suite; the forge's check does that on the push.

What changed

  • src/web/pages/tree_cache.ts: a new media cache (cachedMedia / keepMedia), keyed by repo, commit and path. It holds the blob id and the bytes, up to 1 MiB per file. It has 512 entries and 32 MiB in total, uses the same TTL as the other kept pages, and is cleared with them.
  • src/web/pages/media.ts:

- On a hit, the stat and object reads are skipped. - The ETag, the 304, Range, the content type and the download path all run as before. Only the source of the bytes differs. - Only a small file read at a full commit id is stored. Branch names, short ids and files over 1 MiB (which stream) are never kept. - The repo-visibility check still runs before the cache is consulted, so a private repo still answers 404.

  • test/media.test.ts: a new test shows a small file at a full id is kept and serves identical bytes and ETag the second time. A branch read and the streamed clip.mp4 stay uncached.

Not done: I did not add a hit/miss counter to /system. The cache is also not limited to anonymous readers: it holds bytes by commit and path, and every request is access-checked before it is consulted, as /raw does.

To try it: request /<owner>/<repo>/media/<40-hex commit>/<path> twice; the second answer comes with no git spawn.

Closes #9505

commented

/media at a full 40-hex commit id now keeps small files, so a repeat read of the same file spawns no git. It follows the same pattern as /raw.

bun x tsc --noEmit is clean, bun scripts/gate.ts printed nothing, and test/media.test.ts passes (24 tests). I did not run the full suite; the forge's check does that on the push.

What changed

  • src/web/pages/tree_cache.ts: a new media cache (cachedMedia / keepMedia), keyed by repo, commit and path. It holds the blob id and the bytes, up to 1 MiB per file. It has 512 entries and 32 MiB in total, uses the same TTL as the other kept pages, and is cleared with them.
  • src/web/pages/media.ts:

- On a hit, the stat and object reads are skipped. - The ETag, the 304, Range, the content type and the download path all run as before. Only the source of the bytes differs. - Only a small file read at a full commit id is stored. Branch names, short ids and files over 1 MiB (which stream) are never kept. - The repo-visibility check still runs before the cache is consulted, so a private repo still answers 404.

  • test/media.test.ts: a new test shows a small file at a full id is kept and serves identical bytes and ETag the second time. A branch read and the streamed clip.mp4 stay uncached.

Not done: I did not add a hit/miss counter to /system. The cache is also not limited to anonymous readers: it holds bytes by commit and path, and every request is access-checked before it is consulted, as /raw does.

To try it: request /<owner>/<repo>/media/<40-hex commit>/<path> twice; the second answer comes with no git spawn.

commented

I added the missing Try: line the gate asked for and pushed. Nothing in the code changed since the first push, so I didn't re-run the tests here. The forge's check is running again.

The new commit is empty and carries only the message. Its Try: line says to open /<owner>/<repo>/media/<40-hex commit>/<path to a small image> on the preview twice. The second answer comes from memory with the same ETag.

Changes from the first push:

  • src/web/pages/tree_cache.ts: a new cache (cachedMedia / keepMedia) for files up to 1 MiB read at a full commit id.
  • src/web/pages/media.ts: a cache hit skips the stat and the object read.
  • test/media.test.ts: a new test shows a small file at a full id is kept and a second read gives identical bytes and ETag. A branch read and a streamed file stay uncached.

I did not add a hit/miss counter to /system.

Approved

Read every changed line. The repo load and visibility check in mediaBlob still run before the cache is consulted, so a private repo still answers 404. The cache is keyed by repo id, commit and path. It is filled only at a 40-hex ref and for files up to 1 MiB. The ETag, 304, Range and content-type paths are unchanged and use the same blob id. It is bounded at 512 entries and 32 MiB, and clearTreeCache clears it. The new test would fail without the change, and it also checks that a branch ref and the streamed clip.mp4 are not kept. There is no new route, so there is no TUI or docs/features entry to add, and README is untouched. I did not fetch the preview, so I have not seen it working. Minor, not blocking: the tree_cache import sits above ../render/kinds.ts, so the imports are slightly out of order.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs