trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns12g rEventsFlows

Files render in the browser: audio and video play inline, seekable #7389

Merged torquecrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/forge:torque/7333-files-render-in-the-browser-audio-and-vi → main 2 commits fork 1 thread · 1 unresolved

  • Merged. torque/7333-files-render-in-the-browser-audio-and-vi was merged into main as 9b36147.

Review

Sign in to review this change.
13 files changed +529 −56
docs/design/areas/media.md added +9
@@ -0,0 +1,9 @@
1 # Media (audio and video on the blob page)
2
3 An audio or video file plays inline, from the media route (`src/web/render/player.ts`, styled in
4 `src/web/style/content.ts`).
5
6 - `.media-pane` — wraps the player and its hidden "can't play" note; `data-kind` is `audio` or `video`.
7 - `.media-player` — the `<audio>` or `<video>` element: full pane width; a video keeps its aspect ratio
8 (`height: auto`) and stops at 80vh.
9 - `.media-unplayable` — the note and Download shown, by the shell script, when the browser cannot decode the file.
docs/features/file-media.md +30 −1
@@ -51,10 +51,39 @@ its module; a kind with none yet keeps the source or binary view it always
5151 had. Where a renderer exists the header offers `Rendered | Source`; every
5252 file's header has the **Raw** and **Download** buttons.
5353
54 ## Audio and video (#7333)
55
56 An audio file is `<audio controls preload="metadata">` and a video
57 `<video controls playsinline preload="metadata">`, each with a `<source>` on
58 the media route and the registry's type (`render/audio.ts`,
59 `render/video.ts`, `render/player.ts`). Range on the media route is what
60 seeks. A `.mov` also offers itself as `video/mp4`, which is what most
61 QuickTime files are inside, for browsers that refuse `video/quicktime`.
62
63 - **Header:** size and container (`3.2 MB · WebM`) in place of a line count.
64 - **Video** is the pane's width at its own aspect ratio, at most 80vh tall.
65 - **A format the browser cannot decode** fires `error`; the shell script
66 (`MEDIA_PLAY_JS`) hides the player and shows "Your browser can't play this
67 format" with Download, already in the markup.
68 - **With scripts off,** a `<noscript>` line under the player offers the same
69 Download, so an undecodable file is never a dead end.
70 - **Container names** come from the registry (`name` in `kinds.ts`,
71 returned by `claimedMedia`): one table names each format.
72 - **In the terminal** an audio or video file is not played; the file pane
73 says so, names the format and size, and prints the media route on the
74 forge's address (`src/tui/ui/media-note.ts`).
75 - **A long file:** the page reads at most 1 MB of a file. Audio and video are
76 drawn from the media route alone (`STREAMED` in `render/renderers.ts`), so
77 past that a file is drawn by its extension (`claimedMedia`) and its bytes
78 are confirmed by the media route, which serves a contradicted one as
79 `application/octet-stream` and the player shows the note. Under 1 MB, the
80 bytes decide first, and a `.mp4` that is text keeps the source view.
81 - Over `RENDER_CAP` (500 MB audio, 4 GB video) the page offers Download.
82
5483 ## Adding a renderer
5584
5685 Write `src/web/render/<kind>.ts` exporting a `Renderer` (plain data in:
5786 path, size, body, and `media`, the media route's address; HTML out), and
58 name it once in `RENDERERS` (`render/renderers.ts`). Point any element
87 name it once in `RENDERERS` (`render/renderers.ts`); `rendererFor` picks it. Point any element
5988 that needs the bytes at `input.media`, never at an inline copy, so
6089 repository content only ever runs under the media route's sandbox. `render/markdown.ts` is the example.
src/tui/screens/workspace.ts +23 −9
@@ -75,6 +75,7 @@ import {
7575 } from "../ui/image.ts";
7676 import { fill, len, split } from "../ui/layout.ts";
7777 import { renderMarkdown, type Line } from "../ui/markdown.ts";
78 import { mediaNote } from "../ui/media-note.ts";
7879 import { tokColor, type Tokens } from "../ui/theme.ts";
7980 import { card, emptyState, input, scrollbar } from "../ui/widgets.ts";
8081 import type { Session } from "../session.ts";
@@ -470,6 +471,11 @@ export function workspaceScreen(
470471 ): Screen {
471472 const key = memoryKey(src);
472473 const memory = recall(f, key);
474 /** This repository on the web, where its media route plays a file; none for a mirror. */
475 const mediaWeb = (): string | undefined =>
476 src.where.kind === "local" && f.app.config.url !== undefined
477 ? `${f.app.config.url.replace(/\/+$/, "")}/${src.where.repo.ownerName}/${src.where.repo.name}`
478 : undefined;
473479 // An explicit `open`/`ref` from the caller wins over what we remembered:
474480 // arriving from a search hit means "show me *that*", not "show me where I
475481 // was". Everything else is restored.
@@ -668,15 +674,23 @@ export function workspaceScreen(
668674 read.error === "not_found"
669675 ? "No such file at this revision."
670676 : errorText(read.error);
671 else if (read.value.kind === "too_large")
672 note = `${formatSize(read.value.size)} — too large to display.`;
673 else if (read.value.kind === "binary") {
674 // A binary blob whose *name* says picture is the one binary we do
675 // something with. Everything else keeps the honest note: showing
676 // bytes as text is how a terminal ends up beeping at you.
677 if (looksLikeImage(t.path)) picture = read.value.body;
678 else note = `Binary file, ${formatSize(read.value.size)}.`;
679 } else raw = read.value.body;
677 else {
678 // Audio and video are not played here; the note says where they are.
679 const media =
680 read.value.kind === "text"
681 ? undefined
682 : mediaNote(t.path, formatSize(read.value.size), mediaWeb(), ref);
683 if (media !== undefined) note = media;
684 else if (read.value.kind === "too_large")
685 note = `${formatSize(read.value.size)} — too large to display.`;
686 else if (read.value.kind === "binary") {
687 // A binary blob whose *name* says picture is the one binary we do
688 // something with. Everything else keeps the honest note: showing
689 // bytes as text is how a terminal ends up beeping at you.
690 if (looksLikeImage(t.path)) picture = read.value.body;
691 else note = `Binary file, ${formatSize(read.value.size)}.`;
692 } else raw = read.value.body;
693 }
680694 entry = { note, raw, picture };
681695 remember(key, entry);
682696 }
src/tui/ui/media-note.ts added +22
@@ -0,0 +1,22 @@
1 /**
2 * What the terminal says for an audio or video file it cannot play: that it
3 * is not played here, and where a browser plays it — the media route, on the
4 * forge's own address when one is configured. `undefined` for any other file.
5 */
6
7 import { claimedMedia } from "../../web/render/kinds.ts";
8
9 export function mediaNote(
10 path: string,
11 size: string,
12 /** `https://forge…/owner/name`, or undefined for a mirror the forge does not serve. */
13 web: string | undefined,
14 ref: string,
15 ): string | undefined {
16 const media = claimedMedia(path);
17 if (media === undefined || (media.kind !== "audio" && media.kind !== "video")) return undefined;
18 const what = `${media.kind === "audio" ? "Audio" : "Video"}, ${media.name}, ${size} — not played in the terminal.`;
19 return web === undefined
20 ? what
21 : `${what} Play it in a browser: ${web}/media/${encodeURIComponent(ref)}/${path.split("/").map(encodeURIComponent).join("/")}`;
22 }
src/web/client.ts +4
@@ -12,6 +12,7 @@
1212
1313 import { STAGE_BADGE_JS } from "./stage-row.ts";
1414 import { paletteHelpersJs } from "./palette.ts";
15 import { MEDIA_PLAY_JS } from "./render/player.ts";
1516 import { NOT_TOLD_WORDS } from "../actions/tell_words.ts";
1617
1718 /**
@@ -1279,6 +1280,9 @@ export const shellJs = /* js */ `
12791280 // until one is: no dead buttons on a page with JavaScript switched off.
12801281 doc.documentElement.classList.add('js');
12811282
1283 // ---- a file the browser cannot play: its note, with Download ----
1284 ${MEDIA_PLAY_JS}
1285
12821286 // ---- menus (details) ----
12831287 function closeMenus(except) {
12841288 doc.querySelectorAll('details.menu[open]').forEach(function (m) {
src/web/pages/repo.ts +16 −21
@@ -71,8 +71,7 @@ import { field, readForm } from "../form.ts";
7171 import { html, when, type Html } from "../html.ts";
7272 import { flashTo, fragment, page, redirect } from "../layout.ts";
7373 import { renderMarkdown } from "../markdown.ts";
74 import { fileKind, HEAD_BYTES, RENDER_CAP } from "../render/kinds.ts";
75 import { RENDERERS } from "../render/renderers.ts";
74 import { rendererFor } from "../render/renderers.ts";
7675 import { mediaHref } from "./media.ts";
7776 import { browseBase, localScope, type RepoScope } from "../scope.ts";
7877 import { repoForkOf } from "../../core/origins.ts";
@@ -1025,23 +1024,17 @@ export function renderBlob(
10251024 const blob = o.blob;
10261025 // The registry decides what this file is and whether anything draws it;
10271026 // a kind with no renderer yet, or one over its cap, keeps the source view.
1028 const renderer =
1029 blob.kind === "too_large"
1030 ? undefined
1031 : (() => {
1032 const kind = fileKind(o.path, blob.body.subarray(0, HEAD_BYTES));
1033 return blob.size <= RENDER_CAP[kind] ? RENDERERS[kind] : undefined;
1034 })();
1027 const renderer = rendererFor(o.path, blob);
10351028 const view: BlobView = renderer ? (o.view ?? "rendered") : "source";
1036 const { content, lines } =
1037 view === "rendered" && renderer && blob.kind !== "too_large"
1038 ? renderer({
1029 const { content, lines, summary } =
1030 view === "rendered" && renderer
1031 ? renderer.render({
10391032 path: o.path,
10401033 size: blob.size,
1041 body: blob.body,
1034 body: renderer.body,
10421035 media: mediaHref(base, o.ref, o.path),
10431036 })
1044 : blobBody(base, o.ref, o.path, blob, o.highlighted, o.all);
1037 : { ...blobBody(base, o.ref, o.path, blob, o.highlighted, o.all), summary: undefined };
10451038 const sizeText =
10461039 blob.kind === "too_large" ? "" : sizeWithTokens(blob.size, o.path);
10471040 const fileHref = `${base}/blob/${encoded}/${o.path}`;
@@ -1055,13 +1048,15 @@ export function renderBlob(
10551048 )}
10561049 ${pathCrumbs(base, scope.facts.name, o.ref, o.path, true)}
10571050 <span class="spacer"></span>
1058 ${when(
1059 lines > 0,
1060 () =>
1061 html`<span class="muted small num"
1062 >${lines.toLocaleString()} lines · ${sizeText}</span
1063 >`,
1064 )}
1051 ${summary !== undefined
1052 ? html`<span class="muted small num">${summary}</span>`
1053 : when(
1054 lines > 0,
1055 () =>
1056 html`<span class="muted small num"
1057 >${lines.toLocaleString()} lines · ${sizeText}</span
1058 >`,
1059 )}
10651060 ${when(
10661061 renderer !== undefined,
10671062 () =>
src/web/render/audio.ts added +8
@@ -0,0 +1,8 @@
1 /** An audio file plays inline: `<audio controls preload="metadata">`. */
2
3 import { player } from "./player.ts";
4 import type { Rendered, RenderInput } from "./renderers.ts";
5
6 export function audio(input: RenderInput): Rendered {
7 return player("audio", input);
8 }
src/web/render/kinds.ts +37 −23
@@ -83,6 +83,8 @@ const svgRoot: Magic = (head) =>
8383
8484 interface Media {
8585 readonly kind: FileKind;
86 /** The format as a person says it, for a header: `WebM`, `QuickTime`. */
87 readonly name: string;
8688 readonly type: string;
8789 /** Absent for a format with no signature (STL); the extension decides. */
8890 readonly magic?: Magic;
@@ -90,29 +92,29 @@ interface Media {
9092
9193 /** The allowlist. A media type that is not here is never served. */
9294 const MEDIA: Readonly<Record<string, Media>> = {
93 png: { kind: "image", type: "image/png", magic: bytes(0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a) },
94 jpg: { kind: "image", type: "image/jpeg", magic: bytes(0xff, 0xd8, 0xff) },
95 jpeg: { kind: "image", type: "image/jpeg", magic: bytes(0xff, 0xd8, 0xff) },
96 gif: { kind: "image", type: "image/gif", magic: any(ascii("GIF87a"), ascii("GIF89a")) },
97 webp: { kind: "image", type: "image/webp", magic: riff("WEBP") },
98 avif: { kind: "image", type: "image/avif", magic: (h) => ftyp(h) && (at(h, 8, "avif") || at(h, 8, "avis")) },
99 bmp: { kind: "image", type: "image/bmp", magic: ascii("BM") },
100 ico: { kind: "image", type: "image/x-icon", magic: bytes(0x00, 0x00, 0x01, 0x00) },
101 svg: { kind: "svg", type: "image/svg+xml", magic: svgRoot },
102 pdf: { kind: "pdf", type: "application/pdf", magic: ascii("%PDF-") },
103 mp3: { kind: "audio", type: "audio/mpeg", magic: mpegAudio },
104 ogg: { kind: "audio", type: "audio/ogg", magic: ascii("OggS") },
105 oga: { kind: "audio", type: "audio/ogg", magic: ascii("OggS") },
106 wav: { kind: "audio", type: "audio/wav", magic: riff("WAVE") },
107 flac: { kind: "audio", type: "audio/flac", magic: ascii("fLaC") },
108 m4a: { kind: "audio", type: "audio/mp4", magic: ftyp },
109 aac: { kind: "audio", type: "audio/aac", magic: any(adts, ftyp) },
110 mp4: { kind: "video", type: "video/mp4", magic: ftyp },
111 m4v: { kind: "video", type: "video/mp4", magic: ftyp },
112 webm: { kind: "video", type: "video/webm", magic: bytes(0x1a, 0x45, 0xdf, 0xa3) },
113 mov: { kind: "video", type: "video/quicktime", magic: quicktime },
114 stl: { kind: "model3d", type: "model/stl" },
115 glb: { kind: "model3d", type: "model/gltf-binary", magic: ascii("glTF") },
95 png: { kind: "image", name: "PNG", type: "image/png", magic: bytes(0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a) },
96 jpg: { kind: "image", name: "JPEG", type: "image/jpeg", magic: bytes(0xff, 0xd8, 0xff) },
97 jpeg: { kind: "image", name: "JPEG", type: "image/jpeg", magic: bytes(0xff, 0xd8, 0xff) },
98 gif: { kind: "image", name: "GIF", type: "image/gif", magic: any(ascii("GIF87a"), ascii("GIF89a")) },
99 webp: { kind: "image", name: "WebP", type: "image/webp", magic: riff("WEBP") },
100 avif: { kind: "image", name: "AVIF", type: "image/avif", magic: (h) => ftyp(h) && (at(h, 8, "avif") || at(h, 8, "avis")) },
101 bmp: { kind: "image", name: "BMP", type: "image/bmp", magic: ascii("BM") },
102 ico: { kind: "image", name: "ICO", type: "image/x-icon", magic: bytes(0x00, 0x00, 0x01, 0x00) },
103 svg: { kind: "svg", name: "SVG", type: "image/svg+xml", magic: svgRoot },
104 pdf: { kind: "pdf", name: "PDF", type: "application/pdf", magic: ascii("%PDF-") },
105 mp3: { kind: "audio", name: "MP3", type: "audio/mpeg", magic: mpegAudio },
106 ogg: { kind: "audio", name: "Ogg", type: "audio/ogg", magic: ascii("OggS") },
107 oga: { kind: "audio", name: "Ogg", type: "audio/ogg", magic: ascii("OggS") },
108 wav: { kind: "audio", name: "WAV", type: "audio/wav", magic: riff("WAVE") },
109 flac: { kind: "audio", name: "FLAC", type: "audio/flac", magic: ascii("fLaC") },
110 m4a: { kind: "audio", name: "MPEG-4 audio", type: "audio/mp4", magic: ftyp },
111 aac: { kind: "audio", name: "AAC", type: "audio/aac", magic: any(adts, ftyp) },
112 mp4: { kind: "video", name: "MPEG-4", type: "video/mp4", magic: ftyp },
113 m4v: { kind: "video", name: "MPEG-4", type: "video/mp4", magic: ftyp },
114 webm: { kind: "video", name: "WebM", type: "video/webm", magic: bytes(0x1a, 0x45, 0xdf, 0xa3) },
115 mov: { kind: "video", name: "QuickTime", type: "video/quicktime", magic: quicktime },
116 stl: { kind: "model3d", name: "STL", type: "model/stl" },
117 glb: { kind: "model3d", name: "glTF binary", type: "model/gltf-binary", magic: ascii("glTF") },
116118 };
117119
118120 /** Kinds read as text: the extension names the kind, the bytes must be text. */
@@ -167,6 +169,18 @@ export function fileKind(path: string, head: Uint8Array): FileKind {
167169 return TEXTUAL[extension(path)] ?? "text";
168170 }
169171
172 /**
173 * What a file's name alone claims to be, unconfirmed: for a page that never
174 * read the bytes (a long video) and only points an element at the media
175 * route, which confirms them before it sends one with this type.
176 */
177 export function claimedMedia(
178 path: string,
179 ): { kind: FileKind; type: string; name: string } | undefined {
180 const media = MEDIA[extension(path)];
181 return media === undefined ? undefined : { kind: media.kind, type: media.type, name: media.name };
182 }
183
170184 /**
171185 * The one content type the media route sends a file under: an allowlisted
172186 * media type its bytes confirm, or `application/octet-stream`.
src/web/render/player.ts added +84
@@ -0,0 +1,84 @@
1 /**
2 * The browser's own player for an audio or video file, pointed at the media
3 * route. Seeking is HTTP Range on that route; the bytes never pass through
4 * the page, so a long video costs the blob page nothing to draw.
5 *
6 * A format the browser cannot decode is decided by the browser, not by us:
7 * the element fails, and `MEDIA_PLAY_JS` swaps in the "can't play" note that
8 * is already in the markup, hidden, with a Download beside it. With scripts
9 * off, a `<noscript>` line offers the same Download under the player.
10 */
11
12 import { html } from "../html.ts";
13 import { btn, empty, fileSize } from "../ui.ts";
14 import { claimedMedia } from "./kinds.ts";
15 import type { Rendered, RenderInput } from "./renderers.ts";
16
17 /**
18 * The `<source>` types offered, in order. A QuickTime file is nearly always
19 * H.264 in an MP4-shaped box, which browsers that answer "no" to
20 * `video/quicktime` play as `video/mp4`; offering both lets each try.
21 */
22 function sourceTypes(path: string): string[] {
23 const type = claimedMedia(path)?.type ?? "application/octet-stream";
24 return type === "video/quicktime" ? [type, "video/mp4"] : [type];
25 }
26
27 export function player(kind: "audio" | "video", input: RenderInput): Rendered {
28 const sources = sourceTypes(input.path).map(
29 (type) => html`<source src="${input.media}" type="${type}" />`,
30 );
31 const element =
32 kind === "video"
33 ? html`<video class="media-player" controls playsinline preload="metadata">${sources}</video>`
34 : html`<audio class="media-player" controls preload="metadata">${sources}</audio>`;
35 const container = claimedMedia(input.path)?.name ?? "Unknown format";
36 const download = btn("Download", {
37 href: `${input.media}?download=1`,
38 variant: "primary",
39 icon: "download",
40 });
41 return {
42 lines: 0,
43 summary: `${fileSize(input.size)} · ${container}`,
44 content: html`<div class="media-pane" data-kind="${kind}">
45 ${element}
46 <div class="media-unplayable" hidden>
47 ${empty({
48 icon: "file",
49 title: "Your browser can't play this format",
50 note: `${container}, ${fileSize(input.size)}. Download it to play it elsewhere.`,
51 action: download,
52 })}
53 </div>
54 <noscript>
55 <p class="muted small">If it does not play here, download it. ${download}</p>
56 </noscript>
57 </div>`,
58 };
junipercrew outdated

No-JS fallback: add a <noscript> Download (or fallback children inside the element) here so an undecodable file is not a dead player when scripts are off.
59 }
60
61 /**
62 * Part of the shell script. `error` does not bubble, so it is caught on the
63 * way down; a failing `<source>` matters only when it is the last one (the
64 * browser tries the next itself), a failing element always does. A player
65 * that failed before this script ran is found by its `networkState`.
66 */
67 export const MEDIA_PLAY_JS = /* js */ `
68 function unplayable(el) {
69 var pane = el && el.closest ? el.closest('.media-pane') : null;
70 var note = pane ? pane.querySelector('.media-unplayable') : null;
71 if (!note) return;
72 el.hidden = true;
73 note.hidden = false;
74 }
75 doc.addEventListener('error', function (e) {
76 var t = e.target;
77 if (!t || !t.tagName) return;
78 if (t.tagName === 'SOURCE' && !t.nextElementSibling) unplayable(t.parentElement);
79 else if (t.classList && t.classList.contains('media-player')) unplayable(t);
80 }, true);
81 doc.querySelectorAll('.media-player').forEach(function (m) {
82 if (m.error || m.networkState === 3) unplayable(m);
83 });
84 `;
src/web/render/renderers.ts +35 −2
@@ -5,14 +5,20 @@
55 */
66
77 import type { Html } from "../html.ts";
8 import type { FileKind } from "./kinds.ts";
8 import type { Blob } from "../../view/source.ts";
9 import { audio } from "./audio.ts";
10 import { claimedMedia, fileKind, HEAD_BYTES, NO_BYTES, RENDER_CAP, type FileKind } from "./kinds.ts";
911 import { markdown } from "./markdown.ts";
12 import { video } from "./video.ts";
1013
1114 /** What a renderer is handed: the file, and where its bytes can be fetched. */
1215 export interface RenderInput {
1316 readonly path: string;
1417 readonly size: number;
15 /** The whole body, when the page read it; a heavy viewer fetches `media`. */
18 /**
19 * The whole body, when the page read it; empty for a file over the page's
20 * read ceiling, which only a `STREAMED` kind is drawn from.
21 */
1622 readonly body: Uint8Array;
1723 /** `/:owner/:name/media/:ref/*path`, for `<img>`, `<video>` and `fetch()`. */
1824 readonly media: string;
@@ -22,10 +28,37 @@ export interface Rendered {
2228 readonly content: Html;
2329 /** Lines the file has, for the header's count; 0 when lines mean nothing. */
2430 readonly lines: number;
31 /** The header's facts in place of the line count, as `1.2 MB · WebM`. */
32 readonly summary?: string;
2533 }
2634
2735 export type Renderer = (input: RenderInput) => Rendered;
2836
2937 export const RENDERERS: Partial<Readonly<Record<FileKind, Renderer>>> = {
3038 markdown,
39 audio,
40 video,
3141 };
42
43 /**
44 * Kinds drawn wholly from the media route, never from the body: a file of
45 * one of these too large for the page to read is still drawn, by its name,
46 * and the media route confirms its bytes before it serves them as media.
47 */
48 const STREAMED: ReadonlySet<FileKind> = new Set<FileKind>(["audio", "video"]);
49
50 /** The renderer for a file and the bytes it is handed, or none: the source view. */
51 export function rendererFor(
52 path: string,
53 blob: Blob,
54 ): { render: Renderer; body: Uint8Array } | undefined {
55 const kind =
56 blob.kind === "too_large"
57 ? claimedMedia(path)?.kind
58 : fileKind(path, blob.body.subarray(0, HEAD_BYTES));
59 if (kind === undefined || blob.size > RENDER_CAP[kind]) return undefined;
60 if (blob.kind === "too_large" && !STREAMED.has(kind)) return undefined;
61 const render = RENDERERS[kind];
62 if (render === undefined) return undefined;
63 return { render, body: blob.kind === "too_large" ? NO_BYTES : blob.body };
64 }
src/web/render/video.ts added +11
@@ -0,0 +1,11 @@
1 /**
2 * A video plays inline, as wide as the pane and no wider, at its own aspect
3 * ratio: `<video controls playsinline preload="metadata">`.
4 */
5
6 import { player } from "./player.ts";
7 import type { Rendered, RenderInput } from "./renderers.ts";
8
9 export function video(input: RenderInput): Rendered {
10 return player("video", input);
11 }
src/web/style/content.ts +4
@@ -282,6 +282,10 @@ form.inline { display: inline; margin-left: auto; }
282282 one path header, and the switch between the two views in it. */
283283 .blob-view { margin-left: 8px; }
284284 .blob-view .seg { font-size: 12.5px; }
285 /* An audio or video file, played inline (render/player.ts). The video is as
286 wide as the pane and no wider; height follows its own aspect ratio. */
287 .media-pane .media-player { display: block; width: 100%; max-width: 100%; }
288 .media-pane video.media-player { height: auto; max-height: 80vh; background: var(--muted); border-radius: var(--radius-sm); }
285289
286290 /* ---- prose (markdown) ---- */
287291 .prose { padding: 16px 20px; font-size: 15px; line-height: 1.65; overflow-wrap: break-word; }
test/render-media.test.ts added +246
@@ -0,0 +1,246 @@
1 /**
2 * Audio and video play inline on the blob page (#7333): the right element,
3 * pointed at the media route with the right type, for every container; a
4 * file too long for the page to read still plays, and seeks by Range; a
5 * name the bytes contradict keeps the source view; and the "can't play"
6 * note is in the markup for the shell script to show.
7 */
8
9 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
10 import { mkdtemp, rm } from "node:fs/promises";
11 import { tmpdir } from "node:os";
12 import { join } from "node:path";
13 import { createApp, type App } from "../src/app.ts";
14 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
15 import { serve, type Servers } from "../src/http/server.ts";
16 import { shellJs } from "../src/web/client.ts";
17 import { MEDIA_PLAY_JS } from "../src/web/render/player.ts";
18 import { rendererFor } from "../src/web/render/renderers.ts";
19 import { mediaNote } from "../src/tui/ui/media-note.ts";
20 import { seedAdmin } from "./helpers/admin.ts";
21 import { quietLogs } from "./helpers/globals.ts";
22
23 quietLogs();
24
25 const enc = (s: string): Uint8Array => new TextEncoder().encode(s);
26 /** A few bytes after a signature, so each file is what its name says. */
27 const file = (head: string | number[], size = 64): Uint8Array => {
28 const out = new Uint8Array(size);
29 for (let i = 0; i < size; i++) out[i] = (i * 13 + 1) & 0xff;
30 out.set(typeof head === "string" ? enc(head) : new Uint8Array(head), 0);
31 return out;
32 };
33
34 const FILES: ReadonlyArray<{ path: string; bytes: Uint8Array; element: string; types: string[]; container: string }> = [
35 { path: "a.mp3", bytes: file("ID3\x03\0"), element: "audio", types: ["audio/mpeg"], container: "MP3" },
36 { path: "a.wav", bytes: file("RIFF\0\0\0\0WAVEfmt "), element: "audio", types: ["audio/wav"], container: "WAV" },
37 { path: "a.ogg", bytes: file("OggS\0"), element: "audio", types: ["audio/ogg"], container: "Ogg" },
38 { path: "a.m4a", bytes: file("\0\0\0\x20ftypM4A "), element: "audio", types: ["audio/mp4"], container: "MPEG-4 audio" },
39 { path: "v.mp4", bytes: file("\0\0\0\x20ftypisom"), element: "video", types: ["video/mp4"], container: "MPEG-4" },
40 { path: "v.webm", bytes: file([0x1a, 0x45, 0xdf, 0xa3, 0x9f]), element: "video", types: ["video/webm"], container: "WebM" },
41 { path: "v.mov", bytes: file("\0\0\0\x14ftypqt "), element: "video", types: ["video/quicktime", "video/mp4"], container: "QuickTime" },
42 ];
43
44 /** Over the page's 1 MB read ceiling: drawn by name, bytes streamed. */
45 const LONG = file("\0\0\0\x20ftypisom", 1_500_000);
46
47 let dataDir = "";
48 let app: App;
49 let servers: Servers;
50 let base = "";
51
52 const page = async (path: string): Promise<string> => {
53 const res = await fetch(`${base}/alice/tapes/blob/main/${path}`);
54 expect(res.status).toBe(200);
55 return res.text();
56 };
57
58 beforeAll(async () => {
59 dataDir = await mkdtemp(join(tmpdir(), "forge-render-media-"));
60 app = createApp({ dataDir, port: 0 });
61 servers = await serve(app);
62 base = servers.url;
63 const a = await seedAdmin(app, { username: "alice", email: "[email protected]", password: "a-long-media-passphrase" });
64 if (!a.ok) throw new Error("signup failed");
65 const repo = await createRepo(app, a.value, { name: "tapes", visibility: "public" });
66 if (!repo.ok) throw new Error("repo failed");
67 const made = await commitToBranch(app, a.value, repo.value, {
68 message: "tapes",
69 files: [
70 ...FILES.map((f) => ({ path: f.path, content: f.bytes })),
71 { path: "long.mp4", content: LONG },
72 { path: "fake.mp4", content: "not a video, just words\n" },
73 ],
74 });
75 if (!made.ok) throw new Error("commit failed");
76 });
77
78 afterAll(async () => {
79 await servers.stop();
80 app.close();
81 await rm(dataDir, { recursive: true, force: true });
82 });
83
84 describe("audio and video on the blob page", () => {
85 for (const f of FILES) {
86 test(`${f.path} is a <${f.element}> with its source and type`, async () => {
87 const text = await page(f.path);
88 const media = `/alice/tapes/media/main/${f.path}`;
89 if (f.element === "video")
90 expect(text).toContain('<video class="media-player" controls playsinline preload="metadata">');
91 else expect(text).toContain('<audio class="media-player" controls preload="metadata">');
92 const sources = [...text.matchAll(/<source src="([^"]*)" type="([^"]*)"/g)];
93 expect(sources.map((m) => m[1])).toEqual(f.types.map(() => media));
94 expect(sources.map((m) => m[2])).toEqual(f.types);
95 // The header names the size and the container, not a line count.
96 expect(text).toContain(`${f.bytes.length} B · ${f.container}`);
97 // Every view keeps the switch, Raw and Download.
98 expect(text).toContain(`href="/alice/tapes/blob/main/${f.path}?view=source"`);
99 expect(text).toContain(`href="/alice/tapes/raw/main/${f.path}"`);
100 expect(text).toContain(`href="${media}?download=1"`);
101 });
102 }
103
104 test("the can't-play note and its Download are in the markup, hidden", async () => {
105 const text = await page("v.webm");
106 expect(text).toMatch(/<div class="media-unplayable" hidden>[\s\S]*Your browser can(&#39;|&#x27;|')t play this format/);
107 expect(shellJs).toContain(MEDIA_PLAY_JS);
108 });
109
110 test("with no script running, the pane itself still offers Download", async () => {
111 const text = await page("v.webm");
112 const pane = text.slice(text.indexOf('<div class="media-pane"'));
113 const noscript = pane.match(/<noscript>([\s\S]*?)<\/noscript>/)?.[1] ?? "";
114 expect(noscript).toContain('href="/alice/tapes/media/main/v.webm?download=1"');
115 expect(noscript).toContain("Download");
116 });
117
118 test("Source drops the player", async () => {
119 const res = await fetch(`${base}/alice/tapes/blob/main/v.mp4?view=source`);
120 const text = await res.text();
121 expect(text).not.toContain("<video");
122 expect(text).toContain("Binary file");
123 });
124
125 test("a video past the page's read ceiling still plays, and seeks mid-file", async () => {
126 const text = await page("long.mp4");
127 expect(text).toContain('<source src="/alice/tapes/media/main/long.mp4" type="video/mp4"');
128 expect(text).toContain("1.4 MB · MPEG-4");
129 const res = await fetch(`${base}/alice/tapes/media/main/long.mp4`, {
130 headers: { range: "bytes=750000-750099" },
131 });
132 expect(res.status).toBe(206);
133 expect(res.headers.get("content-range")).toBe(`bytes 750000-750099/${LONG.length}`);
134 expect(res.headers.get("content-type")).toBe("video/mp4");
135 expect(new Uint8Array(await res.arrayBuffer())).toEqual(LONG.slice(750000, 750100));
136 });
137
138 test("a name the bytes contradict keeps the source view", async () => {
139 const text = await page("fake.mp4");
140 expect(text).not.toContain("<video");
141 expect(text).toContain("not a video, just words");
142 });
143 });
144
145 describe("rendererFor", () => {
146 test("an unread file is drawn only when its kind streams", () => {
147 expect(rendererFor("x.webm", { kind: "too_large", size: 9_000_000 })?.body.length).toBe(0);
148 expect(rendererFor("x.ogg", { kind: "too_large", size: 9_000_000 })).toBeDefined();
149 expect(rendererFor("x.md", { kind: "too_large", size: 2_000_000 })).toBeUndefined();
150 expect(rendererFor("x.bin", { kind: "too_large", size: 2_000_000 })).toBeUndefined();
151 });
152
153 test("over its kind's cap, nothing draws it", () => {
154 expect(rendererFor("x.mp3", { kind: "too_large", size: 600_000_000 })).toBeUndefined();
155 expect(rendererFor("x.mp4", { kind: "too_large", size: 3_000_000_000 })).toBeDefined();
156 });
157 });
158
159 /** Just enough DOM for the shell's media part: elements, a pane, a note. */
160 function fakeDom() {
161 type El = {
162 tagName: string;
163 hidden: boolean;
164 error: unknown;
165 networkState: number;
166 nextElementSibling: El | null;
167 parentElement: El | null;
168 classList: { contains: (c: string) => boolean };
169 closest: (sel: string) => El | null;
170 querySelector: (sel: string) => El | null;
171 };
172 const el = (tagName: string, cls: string): El => ({
173 tagName,
174 hidden: false,
175 error: null,
176 networkState: 1,
177 nextElementSibling: null,
178 parentElement: null,
179 classList: { contains: (c) => c === cls },
180 closest: () => null,
181 querySelector: () => null,
182 });
183 const pane = el("DIV", "media-pane");
184 const note = el("DIV", "media-unplayable");
185 note.hidden = true;
186 const video = el("VIDEO", "media-player");
187 const first = el("SOURCE", "");
188 const last = el("SOURCE", "");
189 first.nextElementSibling = last;
190 first.parentElement = last.parentElement = video;
191 pane.querySelector = (sel) => (sel === ".media-unplayable" ? note : null);
192 video.closest = (sel) => (sel === ".media-pane" ? pane : null);
193 let onError: ((e: { target: El }) => void) | undefined;
194 const doc = {
195 addEventListener: (type: string, fn: (e: { target: El }) => void, capture: boolean) => {
196 if (type === "error" && capture) onError = fn;
197 },
198 querySelectorAll: (sel: string) => (sel === ".media-player" ? [video] : []),
199 };
200 const run = (): void => new Function("doc", MEDIA_PLAY_JS)(doc);
201 const fire = (target: El): void => onError?.({ target });
202 return { video, note, first, last, run, fire };
203 }
204
205 describe("the shell's can't-play swap", () => {
206 test("a source with another after it is left for the browser to try next", () => {
207 const d = fakeDom();
208 d.run();
209 d.fire(d.first);
210 expect(d.video.hidden).toBe(false);
211 expect(d.note.hidden).toBe(true);
212 });
213
214 test("the last source failing hides the player and shows the note", () => {
215 const d = fakeDom();
216 d.run();
217 d.fire(d.last);
218 expect(d.video.hidden).toBe(true);
219 expect(d.note.hidden).toBe(false);
220 });
221
222 test("a decode error on the element itself does the same", () => {
223 const d = fakeDom();
224 d.run();
225 d.fire(d.video);
226 expect(d.video.hidden).toBe(true);
227 expect(d.note.hidden).toBe(false);
228 });
229
230 test("a player that failed before the script ran is found on load", () => {
231 const d = fakeDom();
232 d.video.networkState = 3;
233 d.run();
234 expect(d.note.hidden).toBe(false);
235 });
236 });
237
238 describe("the terminal's note for a media file", () => {
239 test("names the format and prints the media route", () => {
240 expect(mediaNote("clips/a b.webm", "3.2 M", "https://forge.dev/alice/tapes", "main")).toBe(
241 "Video, WebM, 3.2 M — not played in the terminal. Play it in a browser: https://forge.dev/alice/tapes/media/main/clips/a%20b.webm",
242 );
243 expect(mediaNote("a.mp3", "4 K", undefined, "main")).toBe("Audio, MP3, 4 K — not played in the terminal.");
244 expect(mediaNote("a.png", "4 K", "https://forge.dev/a/b", "main")).toBeUndefined();
245 });
246 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs