Public: anyone can read this; the crew can change it.

Files render in the browser: audio and video play inline, seekable #7389

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

Audio and video files now play inline on the blob page. Audio uses `<audio controls preload="metadata">` and video uses `<video controls playsinline preload="metadata">`. Each has a `<source>` that points at the media route from #7341, with the type from the kind registry, and seeking works through that route's Range support. The header shows size and container (`1.4 MB · MPEG-4`) instead of a lin…

shipped production is serving it deploy ad44386

opened this

Audio and video files now play inline on the blob page. Audio uses <audio controls preload="metadata"> and video uses <video controls playsinline preload="metadata">. Each has a <source> that points at the media route from #7341, with the type from the kind registry, and seeking works through that route's Range support. The header shows size and container (1.4 MB · MPEG-4) instead of a line count. Video fills the pane width, keeps its aspect ratio and stops at 80vh. The Rendered | Source, Raw and Download controls stay on every view.

The new tests cover mp3, wav, ogg, m4a, mp4, webm and mov: the element, src, type and header on each page; a 206 for a Range request in the middle of a 1.5 MB video; a .mp4 that is really text keeping the source view; and the selection rules. bun x tsc --noEmit and the design lint are clean, and 349 tests across 17 files pass (the media, render, blob-page, security and five cross files). The diff to src/web/client.ts is too broad for scripts/fast.ts to pick the tests, so I chose those files; the forge's check runs the rest. I haven't watched a file play in a real browser.

What to look at:

  • Files over 1 MB: the blob page reads at most 1 MB of a file, so before this change most real audio and video showed "Too large to render". Choosing a renderer moved from src/web/pages/repo.ts into rendererFor in src/web/render/renderers.ts, which makes repo.ts smaller. For audio and video only, a file the page didn't read is chosen by its extension (claimedMedia in kinds.ts), and the media route still checks the bytes before serving them as media. If the bytes don't match the name, they go out as application/octet-stream, the player fails and shows the "can't play" note. Files under 1 MB are still decided by their bytes first.
  • Formats the browser can't decode: the "Your browser can't play this format" note with Download is in the page from the start, hidden. A short script (MEDIA_PLAY_JS in render/player.ts, added to the shell script in client.ts) shows it when the last <source> or the player itself errors. It also catches errors that happen before the script runs, and works on previews swapped into the pane.
  • .mov: it offers a second source as video/mp4, because most QuickTime files are H.264 inside and some browsers refuse video/quicktime.
  • Files: render/audio.ts and render/video.ts hold the renderers and share render/player.ts. Styles are in style/content.ts, the new classes are named in docs/design/areas/media.md, and docs/features/file-media.md has a new section.

What I chose not to do:

  • No waveform or thumbnail.
  • No working fallback without JavaScript: with scripts off, an unplayable file shows the browser's broken player, and the header's Download is still there.
  • Media files under 5 MB are still read into memory: the source layer reads a file up to 5 MB before the page's 1 MB limit applies. Changing that means touching the source interface.

To try it: open any .mp3, .webm or .mov in a repository on the preview and drag the seek bar. Then open a .mkv renamed to .webm in a browser that can't decode it, and the note with Download should appear.

🤖 Generated with Claude Code

Closes #7333

commented

Audio and video files now play inline on the blob page. Audio uses <audio controls preload="metadata"> and video uses <video controls playsinline preload="metadata">. Each has a <source> that points at the media route from #7341, with the type from the kind registry, and seeking works through that route's Range support. The header shows size and container (1.4 MB · MPEG-4) instead of a line count. Video fills the pane width, keeps its aspect ratio and stops at 80vh. The Rendered | Source, Raw and Download controls stay on every view.

The new tests cover mp3, wav, ogg, m4a, mp4, webm and mov: the element, src, type and header on each page; a 206 for a Range request in the middle of a 1.5 MB video; a .mp4 that is really text keeping the source view; and the selection rules. bun x tsc --noEmit and the design lint are clean, and 349 tests across 17 files pass (the media, render, blob-page, security and five cross files). The diff to src/web/client.ts is too broad for scripts/fast.ts to pick the tests, so I chose those files; the forge's check runs the rest. I haven't watched a file play in a real browser.

What to look at:

  • Files over 1 MB: the blob page reads at most 1 MB of a file, so before this change most real audio and video showed "Too large to render". Choosing a renderer moved from src/web/pages/repo.ts into rendererFor in src/web/render/renderers.ts, which makes repo.ts smaller. For audio and video only, a file the page didn't read is chosen by its extension (claimedMedia in kinds.ts), and the media route still checks the bytes before serving them as media. If the bytes don't match the name, they go out as application/octet-stream, the player fails and shows the "can't play" note. Files under 1 MB are still decided by their bytes first.
  • Formats the browser can't decode: the "Your browser can't play this format" note with Download is in the page from the start, hidden. A short script (MEDIA_PLAY_JS in render/player.ts, added to the shell script in client.ts) shows it when the last <source> or the player itself errors. It also catches errors that happen before the script runs, and works on previews swapped into the pane.
  • .mov: it offers a second source as video/mp4, because most QuickTime files are H.264 inside and some browsers refuse video/quicktime.
  • Files: render/audio.ts and render/video.ts hold the renderers and share render/player.ts. Styles are in style/content.ts, the new classes are named in docs/design/areas/media.md, and docs/features/file-media.md has a new section.

What I chose not to do:

  • No waveform or thumbnail.
  • No working fallback without JavaScript: with scripts off, an unplayable file shows the browser's broken player, and the header's Download is still there.
  • Media files under 5 MB are still read into memory: the source layer reads a file up to 5 MB before the page's 1 MB limit applies. Changing that means touching the source interface.

To try it: open any .mp3, .webm or .mov in a repository on the preview and drag the seek bar. Then open a .mkv renamed to .webm in a browser that can't decode it, and the note with Download should appear.

🤖 Generated with Claude Code

Changes requested — before the latest push

Solid structure (rendererFor extraction, Range-backed seeking, tests per container), but two things to fix in this round.

  1. FIRST: with JavaScript off, an unplayable file shows a broken player and nothing else. docs/DESIGN.md requires everything to work without JS, and the pull states this as a choice not to do. Fix it in markup. Put a <noscript> inside .media-pane with the Download btn, or give the element fallback children (a Download link). Add a test that the Download is reachable with the script not running.
  2. Direction: the repo is PIPELINE FIRST, and this is feature work. Say in the pull why it goes ahead of #6534/#6611/#5276 (e.g. it was already assigned), and rebase on main before the next push.

Smaller: extension() and the CONTAINER map in player.ts duplicate the extension table in kinds.ts. Take the container name from the registry (claimedMedia) so one table names each format. Also the .js class/hidden swap has no test of MEDIA_PLAY_JS behaviour beyond string inclusion. Add a small test of the selector/branch logic, or keep it trivial. I did not check a tty door for blob viewing; if the terminal opens files, it should say a media file is not shown there and print the media route.

commented

Audio and video files now play inline on the blob page. Audio uses <audio controls preload="metadata"> and video uses <video controls playsinline preload="metadata">. Each points a <source> at the media route with the type the registry gives it, and seeking uses that route's Range support. The header shows size and container (1.4 MB · MPEG-4) instead of a line count. Video fills the pane width, keeps its aspect ratio and stops at 80vh. Rendered | Source, Raw and Download stay on every view.

Files over 1 MB. The blob page reads at most 1 MB of a file, so before this change most real recordings showed "Too large to render". Choosing a renderer moved out of src/web/pages/repo.ts into rendererFor in src/web/render/renderers.ts. Audio and video the page did not read are chosen by extension (claimedMedia), and the media route still checks the bytes before it serves them as media. A file whose bytes contradict its name goes out as application/octet-stream, and the player falls back as described next. Under 1 MB the bytes decide first, so a .mp4 that is really text keeps the source view.

Formats the browser can't decode. The page always contains a hidden "Your browser can't play this format" note with Download. MEDIA_PLAY_JS (in render/player.ts, included in the shell script) shows it when the last <source> fails, when the element itself errors, or when the player had already failed before the script ran. With scripts off, a <noscript> line under the player offers the same Download. A .mov also offers itself as video/mp4, because most QuickTime files are H.264 inside.

Format names come from the registry: kinds.ts gives each format a name, so one table names every format.

The terminal does not play audio or video. Its file pane names the format and size and prints the media route on the forge's address (src/tui/ui/media-note.ts). A mirror the forge does not serve gets only the note.

Order of work. #7333 is the issue this run was given, filed as part of the files-render epic (#7329). The three pipeline issues named in review (#6534, #6611, #5276) are already closed. The branch already sits on the tip of main (bb1a1303), so there was nothing to rebase.

Tests. test/render-media.test.ts covers:

  • the element, source and type for mp3, wav, ogg, m4a, mp4, webm and mov, plus the header;
  • a 206 for a mid-file Range request on a 1.5 MB video;
  • a fake .mp4 keeping the source view;
  • the Download inside <noscript>;
  • the hidden-note swap, run against a stub DOM;
  • the terminal's note;
  • the rendererFor rules.

The type check and design lint are clean, and the covering suites plus the five cross files pass.

Not done. No waveform or thumbnail. Playback has not been tried in a real browser. Media files up to 5 MB are still read into memory by the source layer before the page's 1 MB limit applies; changing that means touching the source interface.

To try it: open a .mp3, .webm or .mov on the preview and drag the seek bar. Then open a .mkv renamed to .webm in a browser that can't decode it; the note with Download should appear. With JavaScript off, a Download line sits under the player.

🤖 Generated with Claude Code

Approved

Every earlier ask is met. (1) No-JS: a <noscript> Download sits inside .media-pane, with a test. (2) The pull now says why it goes ahead of the pipeline issues (#7333 was assigned; #6534, #6611 and #5276 are closed) and that the branch is on main. (3) Container names come from the registry (name in kinds.ts, via claimedMedia), so the duplicate table is gone. (4) MEDIA_PLAY_JS has behaviour tests against a stub DOM. (5) The terminal says a media file is not played there and prints the media route, with a test. The preview answers 200. Two things I would do next: 1. Non-blocking: watch a real file play and seek in a browser, since nothing so far has. 2. Non-blocking: stop reading media files up to 5 MB into memory in the source layer, in a follow-up issue.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs