99844ff · 439dfe9 · c54572a · f722709 · ba2a4b5 · e85b01f
@@ -180,7 +180,8 @@
src/
              header, settings shell, the one 404), flash, layout (CSP+nonce,
              topbar, theme switcher), client (progressive enhancement, as a
              string), style/{base,components,content} assembled by style.ts,
              markdown, highlight, code, worddiff, diff-view, live, and
              markdown, highlight, code, worddiff, diff-view, live, render/ (the
              file-kind registry, kinds.ts, and one renderer per kind), and
              pages/{site,repo,issues,pulls,pull,thread,commits,refs,search,
              org,account,settings}, and studio/ — the app-as-the-unit
              surface, one module per concept: words (the vocabulary every
@@ -359,6 +359,7 @@
This page is the front door. Everything else is a file of its own under
- [`docs/FEED.md`](docs/FEED.md) — the feed: what happened, ranked, each item with why it is there. The home composer keeps your draft across a reload.
- [`docs/PULLS.md`](docs/PULLS.md) — resolving a conflicted pull without a clone.
- [`docs/GIT.md`](docs/GIT.md) — cloning over HTTPS and SSH, and partial clones of big repositories.
- [`docs/features/file-media.md`](docs/features/file-media.md) — a file in the browser: every file page has **Raw** and **Download** (any size), and `/:owner/:name/media/:ref/*path` serves images, audio, video and PDFs under their real type, sandboxed, with byte ranges.
- [`docs/ROLES.md`](docs/ROLES.md) — guest, member and admin, and what each may do.
- [`docs/MCP.md`](docs/MCP.md) — the forge as a connector; [`docs/STANDARDS.md`](docs/STANDARDS.md) is what it answers by default: `.well-known`, feeds, cards, the manifest.
@@ -0,0 +1,60 @@
# Files in the browser: the media route and the kind registry
The foundation for rendering files on the blob page (#7329, #7330).
## The media route
`GET /:owner/:name/media/:ref/*path` serves a file's bytes under a real
media type, for `<img>`, `<video>`, `<audio>`, `<object>` and `fetch()`.
- **Types** come only from the allowlist in `src/web/render/kinds.ts`
  (png, jpeg, gif, webp, avif, bmp, ico, svg, pdf, mp3, ogg/oga, wav, flac,
  m4a, aac, mp4, m4v, webm, mov, stl, glb). The extension proposes a type and
  the file's first bytes confirm it; a mismatch, or any other file, is
  `application/octet-stream`. Never HTML, never script.
- **Headers:** `X-Content-Type-Options: nosniff` and
  `Content-Security-Policy: sandbox; default-src 'none'; img-src 'self' data:; media-src 'self'; style-src 'unsafe-inline'`,
  so an SVG with a `<script>` is drawn and never runs.
- **Range:** one `bytes=` range answers 206 with `Content-Range`; a suffix
  (`bytes=-N`) is the last N bytes; past the end is 416. A list of ranges is
  answered whole, which the RFC allows. `If-Range` is honoured.
- **Caching:** `ETag` is the blob id, and `If-None-Match` answers 304. A
  full commit id as `:ref` is `private, max-age=31536000, immutable`; a
  branch or tag is `private, max-age=0` and revalidates.
- **Access** is `/raw`'s: a repository you cannot see is the same 404 as one
  that is not there.
- **Streaming:** a file over 1 MB streams off `git cat-file` and is never
  held whole. Streams take permits from their own gate (`mediaGate`,
  `FORGE_MEDIA_CONCURRENCY`, default 16), so a page of videos never takes
  the plumbing permits every page needs. The permit is taken before the
  stream exists; a stream gives it back when it ends, when its reader hangs
  up, or when nothing has been pulled for 30 s (`MEDIA_IDLE_MS`), so a
  reader that never reads cannot hold one for good. With 32 requests
  already waiting (`MEDIA_QUEUE_MAX`) the next is a 503 with
  `Retry-After: 5`. A file of 1 MB or less never waits: it is read through
  the pooled reader.
- A 304 is answered from the blob id alone, before any byte is read.
- `?download=1` sends the file as an attachment under its own name, at any
  size. The blob page's Download button points here.
## The kind registry
`fileKind(path, firstBytes)` answers `image | svg | pdf | audio | video |
csv | model3d | config | markdown | text | binary`. `RENDER_CAP` is the
largest file each kind draws inline; over it, the page offers Download.
`kinds.ts` is plain data with no web dependency: the editor and the
terminal ask it whether a path is Markdown, as the blob page does.
`renderBlob` asks `render/renderers.ts`. A kind in `RENDERERS` is drawn by
its module; a kind with none yet keeps the source or binary view it always
had. Where a renderer exists the header offers `Rendered | Source`; every
file's header has the **Raw** and **Download** buttons.
## Adding a renderer
Write `src/web/render/<kind>.ts` exporting a `Renderer` (plain data in:
path, size, body, and `media`, the media route's address; HTML out), and
name it once in `RENDERERS` (`render/renderers.ts`). Point any element
that needs the bytes at `input.media`, never at an inline copy, so
repository content only ever runs under the media route's sandbox. `render/markdown.ts` is the example.
@@ -195,6 +195,13 @@
const limitFromEnv = (name: string, fallback: number): number => {
 */
export const gitGate = new Gate(limitFromEnv("FORGE_GIT_CONCURRENCY", 8));
/**
 * Blob streams for the media route: a video seeks for minutes, so a stream
 * holds its permit until the last byte or the reader hangs up. A pool of its
 * own, so a page of videos never takes the plumbing permits every page needs.
 */
export const mediaGate = new Gate(limitFromEnv("FORGE_MEDIA_CONCURRENCY", 16));
/**
 * How many pack transfers a host of `bytes` memory runs at once by default:
 * one per 64 MiB, at least two, at most sixteen. A push is not one process
@@ -7,8 +7,8 @@
import { defect } from "../kernel/error.ts";
import { err, ok, type Result } from "../kernel/result.ts";
import { gitEnv, runGit, runGitRaw, type GitFailure } from "./env.ts";
import { isSafeRef } from "./guard.ts";
import { gitEnv, mediaGate, runGit, runGitRaw, type GitFailure } from "./env.ts";
import { isSafeOid, isSafeRef } from "./guard.ts";
export type ObjectType = "blob" | "tree" | "commit" | "tag";
@@ -603,3 +603,95 @@
export async function refTip(dir: string, ref: string): Promise<string | undefin
  }
  return undefined;
}
/** How many requests may wait for a media permit before the next is turned away. */
export const MEDIA_QUEUE_MAX = 32;
/** A stream nobody has pulled from for this long is ended and its permit returned. */
export const MEDIA_IDLE_MS = 30_000;
/**
 * Bytes `start`..`end` (inclusive) of the blob `oid`, streamed straight off
 * `git cat-file` and never held whole: a 2 GB video costs a pipe, not 2 GB
 * of memory. Git cannot seek inside a blob, so the bytes before `start` are
 * read and dropped; past `end` the process is killed.
 *
 * The permit is taken before the stream exists, so there is no stream to
 * cancel while it is waiting; a reader whose `signal` aborts in the queue
 * gets `"gone"` and the permit goes back unspent. With `MEDIA_QUEUE_MAX`
 * already waiting, the answer is `"busy"` (a 503) rather than one more
 * waiter. Once open, the stream gives its permit back when it ends, when its
 * reader hangs up or its signal aborts, or when nothing has pulled from it
 * for `idleMs`, which keeps a reader that never reads from holding a permit
 * for good.
 */
export async function blobStream(
  dir: string,
  oid: string,
  start: number,
  end: number,
      
        sentinelcrew
        17d ago
        outdated
      
      High: if the reader cancels while this await is pending, cancel() → finish() runs with release/proc undefined. This line then takes a permit and spawns a git cat-file that is never read or killed. Set a cancelled flag in cancel() and check it here: release and return without spawning. Also add an idle deadline per stream so a reader that never reads cannot hold the permit forever.
    
  limits: { idleMs?: number; queueMax?: number; signal?: AbortSignal } = {},
): Promise<ReadableStream<Uint8Array> | "busy" | "gone"> {
  if (!isSafeOid(oid)) defect("git.unsafe_oid", `unsafe oid: ${JSON.stringify(oid)}`);
  const free = mediaGate.stats();
  if (free.active >= free.limit && free.waiting >= (limits.queueMax ?? MEDIA_QUEUE_MAX))
    return "busy";
  const release = await mediaGate.acquire();
  // The reader left while it waited: the permit goes straight back, unspent.
  if (limits.signal?.aborted) {
    release();
    return "gone";
  }
  const proc = Bun.spawn(["git", "cat-file", "blob", oid], {
    cwd: dir,
    env: gitEnv(),
    stdin: "ignore",
    stdout: "pipe",
    stderr: "ignore",
  });
  const source = proc.stdout.getReader();
  const idleMs = limits.idleMs ?? MEDIA_IDLE_MS;
  let idle: ReturnType<typeof setTimeout> | undefined;
  let offset = 0;
  let ended = false;
  const finish = (): void => {
    if (ended) return;
    ended = true;
    clearTimeout(idle);
    limits.signal?.removeEventListener("abort", finish);
    proc.kill("SIGKILL");
    release();
  };
  limits.signal?.addEventListener("abort", finish, { once: true });
  return new ReadableStream<Uint8Array>({
    start(controller) {
      // Armed from the first moment, so a response nobody ever reads ends too.
      idle = setTimeout(() => {
        finish();
        controller.error(new Error("media stream idle"));
      }, idleMs);
    },
    async pull(controller) {
      idle?.refresh();
      while (true) {
        const next = await source.read();
        const chunk = next.value;
        if (next.done || chunk === undefined) break;
        const from = offset;
        offset += chunk.length;
        if (offset <= start) continue;
        // A copy, not a view: Bun may recycle the pipe's buffer.
        controller.enqueue(
          chunk.slice(Math.max(0, start - from), Math.min(chunk.length, end + 1 - from)),
        );
        if (offset > end) break;
        return;
      }
      if (ended) return;
      finish();
      controller.close();
    },
    cancel() {
      finish();
    },
  });
}
@@ -0,0 +1,43 @@
/**
 * One HTTP byte range (RFC 9110 §14), as plain data. A video seeks and a PDF
 * streams by asking for a window of a file; this reads which window.
 *
 * Only a single range is honoured. A list of ranges, a unit other than
 * bytes, or a header that does not parse is ignored, which the RFC allows:
 * the whole file goes out with 200, and nothing is refused that a browser
 * would ask for.
 */
/** Inclusive at both ends, as `Content-Range` writes it. */
export interface ByteRange {
  readonly start: number;
  readonly end: number;
}
/**
 * The window `header` asks for in a file of `size` bytes: a range, `null`
 * for the whole file, or `"unsatisfiable"` (416) when it starts past the end.
 */
export function parseRange(
  header: string | null,
  size: number,
): ByteRange | "unsatisfiable" | null {
  if (header === null) return null;
  const match = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
  if (match === null) return null;
  const [, first = "", last = ""] = match;
  if (first === "" && last === "") return null;
  if (first === "") {
    // A suffix: the last N bytes.
    const length = Number(last);
    if (!Number.isSafeInteger(length)) return null;
    if (length === 0 || size === 0) return "unsatisfiable";
    return { start: Math.max(0, size - length), end: size - 1 };
  }
  const start = Number(first);
  const end = last === "" ? Number.MAX_SAFE_INTEGER : Number(last);
  if (!Number.isSafeInteger(start) || !Number.isSafeInteger(end)) return null;
  if (end < start) return null;
  if (start >= size) return "unsatisfiable";
  return { start, end: Math.min(end, size - 1) };
}
@@ -205,6 +205,7 @@
import {
  repoSyncSubmit,
  treePage,
} from "../web/pages/repo.ts";
import { mediaBlob } from "../web/pages/media.ts";
import {
  branchCreate,
  branchDelete,
@@ -1324,6 +1325,7 @@
export function buildRouter(): Router<Handler> {
  router.add("GET", "/:owner/:name/tree/:ref/*path", treePage);
  router.add("GET", "/:owner/:name/blob/:ref/*path", blobPage);
  router.add("GET", "/:owner/:name/raw/:ref/*path", rawBlob);
  router.add("GET", "/:owner/:name/media/:ref/*path", mediaBlob);
  // Every file path at a ref, for the fuzzy finder. JSON, `canRead`-gated.
  router.add("GET", "/:owner/:name/paths/:ref", pathsJson);
@@ -45,6 +45,7 @@
import { stopMirroring } from "../../actions/mirror_target.ts";
import { getMirrorTarget, mirrorLine } from "../../core/migrate_mirror.ts";
import { canAdmin, DECISIONS_BY, getRepoById, type Repo } from "../../core/repos.ts";
import type { TreeEntry } from "../../git/read.ts";
import { fileKind, NO_BYTES } from "../../web/render/kinds.ts";
import type { RepoFacts, RepoSource } from "../../view/source.ts";
import { detectLanguage, highlightLines } from "../../web/highlight.ts";
import type { Ctx, Screen } from "../app/app.ts";
@@ -257,7 +258,7 @@
type TabKind = "image" | "markdown" | "code";
/** `README.md`, `NOTES.markdown`, `readme` — the prose a repo opens on. */
function isMarkdownPath(path: string): boolean {
  const name = path.slice(path.lastIndexOf("/") + 1);
  return /\.(md|markdown)$/i.test(name) || /^readme(\.|$)/i.test(name);
  return fileKind(name, NO_BYTES) === "markdown" || /^readme(\.|$)/i.test(name);
}
class Tab {
@@ -17,6 +17,7 @@
 * preview are `editorJs`, and none of them is load-bearing.
 */
import { fileKind, NO_BYTES } from "../render/kinds.ts";
import { editFiles, MAX_FILE_BYTES } from "../../actions/contents.ts";
import { contribute } from "../../actions/contribute.ts";
import { NO_WRITE_ACCESS } from "../../actions/pull_words.ts";
@@ -555,8 +556,9 @@
function mirrorHtml(content: string, lang: string): string {
    .join("");
}
/** A file not yet named is written as prose until its name says otherwise. */
const isMarkdownPath = (path: string): boolean =>
  path === "" || /\.(md|markdown|mdx)$/i.test(path);
  path === "" || fileKind(path, NO_BYTES) === "markdown";
/**
 * The repository and the situation an edit route needs, or the refusal. A
@@ -0,0 +1,126 @@
/**
 * GET /:owner/:name/media/:ref/*path — a file's bytes under a real media
 * type, so a browser can show them in `<img>`, `<video>`, `<audio>`,
 * `<object>` and `fetch()`. `/raw` is the text you would copy; this is the
 * picture, the song, the PDF.
 *
 * The type comes only from the allowlist in `render/kinds.ts`, confirmed by
 * the file's own first bytes; anything else is `application/octet-stream`.
 * Never HTML, never script, and the sandbox policy below means an SVG that
 * carries a `<script>` is drawn as a picture and never runs on this origin.
 *
 * Bytes stream from git, never buffered whole, with HTTP Range so a video
 * seeks and a PDF streams. Access is `/raw`'s: a repository you cannot see
 * answers the same 404 as one that is not there.
 */
import { isSafeRef, isSafeTreePath } from "../../git/guard.ts";
import { blobStream, readerFor } from "../../git/read.ts";
import { repoDir } from "../../git/repos.ts";
import type { Ctx } from "../../http/context.ts";
import { holds } from "../../http/etag.ts";
import { parseRange } from "../../http/range.ts";
import { loadRepo, notFoundPage } from "../chrome.ts";
import { HEAD_BYTES, mediaType } from "../render/kinds.ts";
/** The policy every media answer leaves with. */
export const MEDIA_POLICY =
  "sandbox; default-src 'none'; img-src 'self' data:; media-src 'self'; style-src 'unsafe-inline'";
/** At or under this, the pooled reader hands the body back without a spawn. */
const SMALL_BYTES = 1_000_000;
const FULL_OID = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/;
/** `/media/:ref/:path`, the address a renderer points its element at. */
export const mediaHref = (base: string, ref: string, path: string): string =>
  `${base}/media/${encodeURIComponent(ref)}/${path}`;
export async function mediaBlob(ctx: Ctx): Promise<Response> {
  const repo = loadRepo(ctx);
  if (!repo) return notFoundPage(ctx);
  const ref = ctx.params.ref ?? "";
  const path = ctx.params.path ?? "";
  if (!isSafeRef(ref) || !isSafeTreePath(path)) return notFoundPage(ctx);
  const dir = repoDir(ctx.app.reposRoot, repo.ownerName, repo.name);
  const reader = readerFor(dir);
  const stat = await reader.stat(`${ref}:${path}`).catch(() => undefined);
  if (stat === undefined || !stat.ok || stat.value.type !== "blob")
    return notFoundPage(ctx);
  const { oid, size } = stat.value;
  const download = ctx.url.searchParams.get("download") === "1";
  const etag = `"${oid}"`;
  const headers: Record<string, string> = {
    "x-content-type-options": "nosniff",
    "content-security-policy": MEDIA_POLICY,
    "referrer-policy": "same-origin",
    "accept-ranges": "bytes",
    etag,
    // A full commit id names bytes that can never change; a branch does not.
    "cache-control": FULL_OID.test(ref)
      ? "private, max-age=31536000, immutable"
      
        sentinelcrew
        17d ago
        outdated
      
      The sniff for a large file takes a mediaGate permit and waits with no timeout. Once 16 stalled streams hold the gate, this await hangs every later request for a large file. Cap the queue and answer 503 + Retry-After instead of waiting without end.
    
      : "private, max-age=0",
  };
  if (download) headers["content-disposition"] = attachment(path);
  // The tag is the blob id `stat` already gave, so a revalidation reads no byte.
  if (holds(ctx.request, etag)) return new Response(null, { status: 304, headers });
  // A small file is read whole through the pool; a large one is sniffed from
  // its first bytes and streamed, so its size never becomes memory.
  let body: Uint8Array | undefined;
  let head: Uint8Array;
  if (size <= SMALL_BYTES) {
    const object = await reader.object(oid).catch(() => undefined);
    if (object === undefined || !object.ok) return notFoundPage(ctx);
    body = object.value.body;
    head = body.subarray(0, HEAD_BYTES);
  } else {
    const sniff = await blobStream(dir, oid, 0, HEAD_BYTES - 1, { signal: ctx.request.signal });
    if (sniff === "busy" || sniff === "gone") return busy();
      
        sentinelcrew
        17d ago
        outdated
      
      The ETag is known from stat (line 51), so check holds() before the object read at lines 56-59 and before the sniff. A revalidation should not read up to 1 MB.
    
    head = await new Response(sniff).bytes();
  }
  headers["content-type"] = download ? "application/octet-stream" : mediaType(path, head);
  // A Range past an If-Range that no longer holds gets the whole new file.
  const ifRange = ctx.request.headers.get("if-range");
  const range =
    ifRange === null || ifRange === etag
      ? parseRange(ctx.request.headers.get("range"), size)
      : null;
  if (range === "unsatisfiable") {
    headers["content-range"] = `bytes */${size}`;
    return new Response(null, { status: 416, headers });
  }
  const start = range?.start ?? 0;
  const end = range?.end ?? size - 1;
  if (range !== null) headers["content-range"] = `bytes ${start}-${end}/${size}`;
  const status = range === null ? 200 : 206;
  if (size === 0) return new Response(null, { status, headers });
  // A streamed body goes out chunked (Bun does not keep a length on a large
  // stream); `Content-Range` still names the whole size, which is what a
  // seeking player reads.
  if (body === undefined) {
    const stream = await blobStream(dir, oid, start, end, { signal: ctx.request.signal });
    if (stream === "busy" || stream === "gone") return busy();
    return new Response(stream, { status, headers });
  }
  headers["content-length"] = String(end + 1 - start);
  return new Response(body.subarray(start, end + 1), { status, headers });
}
/** Every media permit is held and the queue is full: come back shortly. */
function busy(): Response {
  return new Response("Busy streaming other files; try again shortly.\n", {
    status: 503,
    headers: { "content-type": "text/plain; charset=utf-8", "retry-after": "5", "cache-control": "no-store" },
  });
}
/** Saved under its own name, quoted so no byte of it can end the header. */
function attachment(path: string): string {
  const name = path.slice(path.lastIndexOf("/") + 1);
  const plain = name.replace(/[^\x20-\x7e]|["\\]/g, "_");
  return `attachment; filename="${plain}"; filename*=UTF-8''${encodeURIComponent(name)}`;
}
@@ -71,6 +71,9 @@
import { field, readForm } from "../form.ts";
import { html, when, type Html } from "../html.ts";
import { flashTo, fragment, page, redirect } from "../layout.ts";
import { renderMarkdown } from "../markdown.ts";
import { fileKind, HEAD_BYTES, RENDER_CAP } from "../render/kinds.ts";
import { RENDERERS } from "../render/renderers.ts";
import { mediaHref } from "./media.ts";
import { browseBase, localScope, type RepoScope } from "../scope.ts";
import { repoForkOf } from "../../core/origins.ts";
import {
@@ -602,8 +605,9 @@
export async function readmeFor(
 */
export type BlobView = "source" | "rendered";
export const isMarkdownPath = (path: string): boolean =>
  /\.(md|markdown|mdx)$/i.test(path);
/** The file saved under its own name, streamed at any size. */
const downloadHref = (base: string, ref: string, path: string): string =>
  `${mediaHref(base, ref, path)}?download=1`;
export function blobView(value: string | null): BlobView {
  return value === "source" ? "source" : "rendered";
@@ -627,7 +631,7 @@
export function blobBody(
        title: "Binary file",
        note: `${fileSize(blob.size)} — not shown, but you can download it.`,
        action: btn("Download", {
          href: `${base}/raw/${encoded}/${path}`,
          href: downloadHref(base, ref, path),
          variant: "primary",
          icon: "download",
        }),
@@ -640,10 +644,11 @@
export function blobBody(
      content: empty({
        icon: "alert",
        title: "Too large to render",
        note: `${fileSize(blob.size)} — over the render cap. Fetch the raw bytes instead.`,
        action: btn("View raw", {
          href: `${base}/raw/${encoded}/${path}`,
        note: `${fileSize(blob.size)} — over the render cap. Download it instead.`,
        action: btn("Download", {
          href: downloadHref(base, ref, path),
          variant: "primary",
          icon: "download",
        }),
      }),
    };
@@ -1017,14 +1022,28 @@
export function renderBlob(
): Html {
  const base = scope.base;
  const encoded = encodeURIComponent(o.ref);
  const markdown = o.blob.kind === "text" && isMarkdownPath(o.path);
  const view: BlobView = markdown ? (o.view ?? "rendered") : "source";
  const blob = o.blob;
  // The registry decides what this file is and whether anything draws it;
  // a kind with no renderer yet, or one over its cap, keeps the source view.
  const renderer =
    blob.kind === "too_large"
      ? undefined
      : (() => {
          const kind = fileKind(o.path, blob.body.subarray(0, HEAD_BYTES));
          return blob.size <= RENDER_CAP[kind] ? RENDERERS[kind] : undefined;
        })();
  const view: BlobView = renderer ? (o.view ?? "rendered") : "source";
  const { content, lines } =
    view === "rendered" && o.blob.kind === "text"
      ? renderedMarkdown(o.blob)
      : blobBody(base, o.ref, o.path, o.blob, o.highlighted, o.all);
    view === "rendered" && renderer && blob.kind !== "too_large"
      ? renderer({
          path: o.path,
          size: blob.size,
          body: blob.body,
          media: mediaHref(base, o.ref, o.path),
        })
      : blobBody(base, o.ref, o.path, blob, o.highlighted, o.all);
  const sizeText =
    o.blob.kind === "too_large" ? "" : sizeWithTokens(o.blob.size, o.path);
    blob.kind === "too_large" ? "" : sizeWithTokens(blob.size, o.path);
  const fileHref = `${base}/blob/${encoded}/${o.path}`;
  return html`<div class="code-pane" id="code-pane" data-path="${o.path}">
    <div class="bar mb">
@@ -1044,20 +1063,20 @@
export function renderBlob(
          >`,
      )}
      ${when(
        markdown,
        renderer !== undefined,
        () =>
          html`<span class="blob-view"
            >${seg([
              {
                label: "Source",
                href: `${fileHref}?view=source`,
                current: view === "source",
              },
              {
                label: "Rendered",
                href: fileHref,
                current: view === "rendered",
              },
      
        sentinelcrew
        17d ago
        outdated
      
      Raw and Download are actions, not views of this page: they can never be current. Keep seg for Rendered/Source (shown only when a renderer exists). Keep Raw as the outline sm btn it was, and add Download as a btn with the download icon.
    
              {
                label: "Source",
                href: `${fileHref}?view=source`,
                current: view === "source",
              },
            ])}</span
          >`,
      )}
@@ -1066,6 +1085,12 @@
export function renderBlob(
        size: "sm",
        variant: "outline",
      })}
      ${btn("Download", {
        href: downloadHref(base, o.ref, o.path),
        size: "sm",
        variant: "outline",
        icon: "download",
      })}
      ${btn("History", {
        href: `${base}/commits?ref=${encoded}&path=${encodeURIComponent(o.path)}`,
        size: "sm",
@@ -1078,18 +1103,6 @@
export function renderBlob(
  </div>`;
}
/** A Markdown file as prose, framed exactly the way a README is. */
function renderedMarkdown(blob: Blob & { kind: "text" }): {
  content: Html;
  lines: number;
} {
  const text = new TextDecoder().decode(blob.body);
  return {
    lines: text === "" ? 0 : text.replace(/\n$/, "").split("\n").length,
    content: html`<div class="card prose">${renderMarkdown(text)}</div>`,
  };
}
/** The bytes, with the headers that stop them ever executing as a page. */
export function rawResponse(
  blob: Blob,
@@ -0,0 +1,182 @@
/**
 * One registry of file kinds: what a file in a repository is, the one media
 * type the forge will ever serve it as, and the module that draws it.
 *
 * It is plain data with no web dependency, so the terminal asks it too. The
 * media route (`pages/media.ts`) asks for a type, and
 * gets one only from the allowlist below, decided by extension and confirmed
 * by the file's own first bytes where its format has a signature; anything
 * else, or a file whose bytes say it is not what its name says, goes out as
 * `application/octet-stream`. Never HTML, never script. `renderBlob` asks
 * `renderers.ts` what draws a kind, and a kind with nothing there keeps the
 * source view it always had.
 */
export type FileKind =
  | "image"
  | "svg"
  | "pdf"
  | "audio"
  | "video"
  | "csv"
  | "model3d"
  | "config"
  | "markdown"
  | "text"
  | "binary";
const MB = 1_000_000;
/**
 * The largest file each kind draws inline. Over it the page offers Download,
 * which streams at any size. A text kind is bounded by the page's own
 * render ceiling too, since its body is read to draw it.
 */
export const RENDER_CAP: Readonly<Record<FileKind, number>> = {
  image: 50 * MB,
  svg: 5 * MB,
  pdf: 100 * MB,
  audio: 500 * MB,
  video: 4_000 * MB,
  csv: 10 * MB,
  model3d: 100 * MB,
  config: 1 * MB,
  markdown: 1 * MB,
  text: 1 * MB,
  binary: 0,
};
type Magic = (head: Uint8Array) => boolean;
const at = (head: Uint8Array, offset: number, text: string): boolean => {
  if (head.length < offset + text.length) return false;
  for (let i = 0; i < text.length; i++)
    if (head[offset + i] !== text.charCodeAt(i)) return false;
  return true;
};
const bytes =
  (...sig: number[]): Magic =>
  (head) =>
    head.length >= sig.length && sig.every((b, i) => head[i] === b);
const ascii =
  (text: string, offset = 0): Magic =>
  (head) =>
    at(head, offset, text);
const any =
  (...checks: Magic[]): Magic =>
  (head) =>
    checks.some((c) => c(head));
/** ISO base media (mp4, m4a, mov, avif): a box size, then `ftyp`. */
const ftyp = ascii("ftyp", 4);
/** An older QuickTime file may open on another top-level atom. */
const quicktime = any(ftyp, ...["moov", "mdat", "wide", "free", "skip", "pnot"].map((a) => ascii(a, 4)));
const riff = (form: string): Magic => (head) => at(head, 0, "RIFF") && at(head, 8, form);
/** MPEG audio: an ID3 tag, or a frame sync (eleven set bits). */
const mpegAudio: Magic = (head) =>
  at(head, 0, "ID3") || (head[0] === 0xff && ((head[1] ?? 0) & 0xe0) === 0xe0);
/** Raw AAC in ADTS frames: a twelve-bit sync, layer zero. */
const adts: Magic = (head) => head[0] === 0xff && ((head[1] ?? 0) & 0xf6) === 0xf0;
/** SVG is text, and has no signature but its root element. */
const svgRoot: Magic = (head) =>
  !head.includes(0) && /<svg[\s>]/i.test(new TextDecoder().decode(head));
interface Media {
  readonly kind: FileKind;
  readonly type: string;
  /** Absent for a format with no signature (STL); the extension decides. */
  readonly magic?: Magic;
}
/** The allowlist. A media type that is not here is never served. */
const MEDIA: Readonly<Record<string, Media>> = {
  png: { kind: "image", type: "image/png", magic: bytes(0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a) },
  jpg: { kind: "image", type: "image/jpeg", magic: bytes(0xff, 0xd8, 0xff) },
  jpeg: { kind: "image", type: "image/jpeg", magic: bytes(0xff, 0xd8, 0xff) },
  gif: { kind: "image", type: "image/gif", magic: any(ascii("GIF87a"), ascii("GIF89a")) },
  webp: { kind: "image", type: "image/webp", magic: riff("WEBP") },
  avif: { kind: "image", type: "image/avif", magic: (h) => ftyp(h) && (at(h, 8, "avif") || at(h, 8, "avis")) },
  bmp: { kind: "image", type: "image/bmp", magic: ascii("BM") },
  ico: { kind: "image", type: "image/x-icon", magic: bytes(0x00, 0x00, 0x01, 0x00) },
  svg: { kind: "svg", type: "image/svg+xml", magic: svgRoot },
  pdf: { kind: "pdf", type: "application/pdf", magic: ascii("%PDF-") },
  mp3: { kind: "audio", type: "audio/mpeg", magic: mpegAudio },
  ogg: { kind: "audio", type: "audio/ogg", magic: ascii("OggS") },
  oga: { kind: "audio", type: "audio/ogg", magic: ascii("OggS") },
  wav: { kind: "audio", type: "audio/wav", magic: riff("WAVE") },
  flac: { kind: "audio", type: "audio/flac", magic: ascii("fLaC") },
  m4a: { kind: "audio", type: "audio/mp4", magic: ftyp },
  aac: { kind: "audio", type: "audio/aac", magic: any(adts, ftyp) },
  mp4: { kind: "video", type: "video/mp4", magic: ftyp },
  m4v: { kind: "video", type: "video/mp4", magic: ftyp },
  webm: { kind: "video", type: "video/webm", magic: bytes(0x1a, 0x45, 0xdf, 0xa3) },
  mov: { kind: "video", type: "video/quicktime", magic: quicktime },
  stl: { kind: "model3d", type: "model/stl" },
  glb: { kind: "model3d", type: "model/gltf-binary", magic: ascii("glTF") },
};
/** Kinds read as text: the extension names the kind, the bytes must be text. */
const TEXTUAL: Readonly<Record<string, FileKind>> = {
  csv: "csv",
  tsv: "csv",
  gltf: "model3d",
  json: "config",
  jsonc: "config",
  yaml: "config",
  yml: "config",
  toml: "config",
  ini: "config",
  conf: "config",
  md: "markdown",
  markdown: "markdown",
  mdx: "markdown",
};
/** For a caller that has only a name: the kind its extension gives. */
export const NO_BYTES: Uint8Array = new Uint8Array(0);
/** How many leading bytes the registry needs to decide. */
export const HEAD_BYTES = 4096;
const extension = (path: string): string => {
  const name = path.slice(path.lastIndexOf("/") + 1);
  const dot = name.lastIndexOf(".");
  return dot <= 0 ? "" : name.slice(dot + 1).toLowerCase();
};
/** Git's own heuristic: a NUL in the first 8 KB means binary. */
const looksBinary = (head: Uint8Array): boolean => head.subarray(0, 8000).includes(0);
/** The allowlisted media entry for a file, if its bytes agree with its name. */
function confirmed(path: string, head: Uint8Array): Media | undefined {
  const media = MEDIA[extension(path)];
  if (media === undefined) return undefined;
  if (media.magic !== undefined && !media.magic(head)) return undefined;
  return media;
}
/**
 * What a file is, from its name and its first bytes (`HEAD_BYTES` is enough).
 * A name the bytes contradict — a `.png` that is not a PNG — is read as the
 * text or binary its bytes say it is.
 */
export function fileKind(path: string, head: Uint8Array): FileKind {
  const media = confirmed(path, head);
  if (media !== undefined) return media.kind;
  if (looksBinary(head)) return "binary";
  return TEXTUAL[extension(path)] ?? "text";
}
/**
 * The one content type the media route sends a file under: an allowlisted
 * media type its bytes confirm, or `application/octet-stream`.
 */
export function mediaType(path: string, head: Uint8Array): string {
  return confirmed(path, head)?.type ?? "application/octet-stream";
}
/** Every extension the registry names, for the test that walks them all. */
export const KNOWN_EXTENSIONS: readonly string[] = [
  ...Object.keys(MEDIA),
  ...Object.keys(TEXTUAL),
];
@@ -0,0 +1,13 @@
/** A Markdown file as prose, framed exactly the way a README is. */
import { html } from "../html.ts";
import { renderMarkdown } from "../markdown.ts";
import type { Rendered, RenderInput } from "./renderers.ts";
export function markdown(input: RenderInput): Rendered {
  const text = new TextDecoder().decode(input.body);
  return {
    lines: text === "" ? 0 : text.replace(/\n$/, "").split("\n").length,
    content: html`<div class="card prose">${renderMarkdown(text)}</div>`,
  };
}
@@ -0,0 +1,31 @@
/**
 * What draws each kind of file on the blob page. A renderer lives in its own
 * file beside this one (`render/<kind>.ts`) and is named once, in
 * `RENDERERS`; a kind missing here keeps the view it always had.
 */
import type { Html } from "../html.ts";
import type { FileKind } from "./kinds.ts";
import { markdown } from "./markdown.ts";
/** What a renderer is handed: the file, and where its bytes can be fetched. */
export interface RenderInput {
  readonly path: string;
  readonly size: number;
  /** The whole body, when the page read it; a heavy viewer fetches `media`. */
  readonly body: Uint8Array;
  /** `/:owner/:name/media/:ref/*path`, for `<img>`, `<video>` and `fetch()`. */
  readonly media: string;
}
export interface Rendered {
  readonly content: Html;
  /** Lines the file has, for the header's count; 0 when lines mean nothing. */
  readonly lines: number;
}
export type Renderer = (input: RenderInput) => Rendered;
export const RENDERERS: Partial<Readonly<Record<FileKind, Renderer>>> = {
  markdown,
};
@@ -0,0 +1,331 @@
/**
 * The media route, `/:owner/:name/media/:ref/*path`, against a real forge:
 * an allowlisted type confirmed by the bytes, the sandbox policy that keeps
 * an SVG's script from running, byte ranges on a buffered and a streamed
 * file, the ETag and its 304, and `/raw`'s access — a private repository is
 * a 404 to a stranger. And the blob page's switch points at it.
 */
import { afterAll, beforeAll, describe, expect, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createApp, type App } from "../src/app.ts";
import { commitToBranch, createRepo } from "../src/actions/repos.ts";
import { createSession } from "../src/core/sessions.ts";
import { mediaGate } from "../src/git/env.ts";
import { blobStream, MEDIA_QUEUE_MAX, refTip, readerFor } from "../src/git/read.ts";
import { repoDir } from "../src/git/repos.ts";
import { serve, type Servers } from "../src/http/server.ts";
import { MEDIA_POLICY } from "../src/web/pages/media.ts";
import { seedAdmin } from "./helpers/admin.ts";
import { quietLogs } from "./helpers/globals.ts";
quietLogs();
const PNG = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13, 1, 2, 3]);
const EVIL_SVG =
  '<svg xmlns="http://www.w3.org/2000/svg"><script>alert(document.cookie)</script><rect width="1" height="1"/></svg>\n';
/** Over the pooled read, so it is sniffed and streamed off git. */
const CLIP = (() => {
  const bytes = new Uint8Array(1_500_000);
  for (let i = 0; i < bytes.length; i++) bytes[i] = (i * 7) & 0xff;
  bytes.set(new TextEncoder().encode("\0\0\0\x20ftypisom"), 0);
  return bytes;
})();
let dataDir = "";
let app: App;
let servers: Servers;
let base = "";
let alice = "";
let sha = "";
const get = (path: string, headers: Record<string, string> = {}): Promise<Response> =>
  fetch(`${base}${path}`, { redirect: "manual", headers });
beforeAll(async () => {
  dataDir = await mkdtemp(join(tmpdir(), "forge-media-"));
  app = createApp({ dataDir, port: 0 });
  servers = await serve(app);
  base = servers.url;
  const a = await seedAdmin(app, { username: "alice", email: "[email protected]", password: "a-long-media-passphrase" });
  if (!a.ok) throw new Error("signup failed");
  alice = `forge_session=${createSession(app.db, a.value.id).token}`;
  const shown = await createRepo(app, a.value, { name: "gallery", visibility: "public" });
  if (!shown.ok) throw new Error("repo failed");
  const made = await commitToBranch(app, a.value, shown.value, {
    message: "pictures",
    files: [
      { path: "art/logo.png", content: PNG },
      { path: "art/fake.png", content: "not a png at all\n" },
      { path: "art/evil.svg", content: EVIL_SVG },
      { path: "art/page.html", content: "<script>alert(1)</script>" },
      { path: "clip.mp4", content: CLIP },
      { path: "README.md", content: "# gallery\n" },
      { path: "src/a.ts", content: "export const a = 1\n" },
      { path: "empty.png", content: "" },
    ],
  });
  if (!made.ok) throw new Error("commit failed");
  sha = (await refTip(repoDir(app.reposRoot, "alice", "gallery"), "refs/heads/main")) ?? "";
  const hidden = await createRepo(app, a.value, { name: "vault", visibility: "private" });
  if (!hidden.ok) throw new Error("private repo failed");
  await commitToBranch(app, a.value, hidden.value, {
    message: "hidden",
    files: [{ path: "secret.png", content: PNG }],
  });
});
afterAll(async () => {
  await servers.stop();
  app.close();
  await rm(dataDir, { recursive: true, force: true });
});
describe("the media route", () => {
  test("a confirmed PNG is served as image/png with its bytes", async () => {
    const res = await get("/alice/gallery/media/main/art/logo.png");
    expect(res.status).toBe(200);
    expect(res.headers.get("content-type")).toBe("image/png");
    expect(res.headers.get("accept-ranges")).toBe("bytes");
    expect(res.headers.get("x-content-type-options")).toBe("nosniff");
    expect(new Uint8Array(await res.arrayBuffer())).toEqual(PNG);
  });
  test("a name the bytes contradict, or a type off the allowlist, is octet-stream", async () => {
    const fake = await get("/alice/gallery/media/main/art/fake.png");
    expect(fake.status).toBe(200);
    expect(fake.headers.get("content-type")).toBe("application/octet-stream");
    const page = await get("/alice/gallery/media/main/art/page.html");
    expect(page.headers.get("content-type")).toBe("application/octet-stream");
  });
  test("an SVG with a script is an image under the sandbox policy", async () => {
    const res = await get("/alice/gallery/media/main/art/evil.svg");
    expect(res.status).toBe(200);
    expect(res.headers.get("content-type")).toBe("image/svg+xml");
    expect(res.headers.get("content-security-policy")).toBe(MEDIA_POLICY);
    expect(MEDIA_POLICY).toStartWith("sandbox;");
    expect(MEDIA_POLICY).toContain("default-src 'none'");
    expect(MEDIA_POLICY).not.toContain("script-src");
    expect(res.headers.get("x-content-type-options")).toBe("nosniff");
  });
  test("a single range is 206 with its window", async () => {
    const res = await get("/alice/gallery/media/main/art/logo.png", { range: "bytes=1-3" });
    expect(res.status).toBe(206);
    expect(res.headers.get("content-range")).toBe(`bytes 1-3/${PNG.length}`);
    expect(res.headers.get("content-length")).toBe("3");
    expect(new Uint8Array(await res.arrayBuffer())).toEqual(PNG.subarray(1, 4));
  });
  test("a suffix range is the last bytes", async () => {
    const res = await get("/alice/gallery/media/main/art/logo.png", { range: "bytes=-4" });
    expect(res.status).toBe(206);
    expect(res.headers.get("content-range")).toBe(`bytes ${PNG.length - 4}-${PNG.length - 1}/${PNG.length}`);
    expect(new Uint8Array(await res.arrayBuffer())).toEqual(PNG.subarray(PNG.length - 4));
  });
  test("a range past the end is 416", async () => {
    const res = await get("/alice/gallery/media/main/art/logo.png", { range: `bytes=${PNG.length}-` });
    expect(res.status).toBe(416);
    expect(res.headers.get("content-range")).toBe(`bytes */${PNG.length}`);
    const empty = await get("/alice/gallery/media/main/empty.png", { range: "bytes=0-" });
    expect(empty.status).toBe(416);
  });
  test("a large file streams off git, whole and in windows", async () => {
    const whole = await get("/alice/gallery/media/main/clip.mp4");
    expect(whole.status).toBe(200);
    expect(whole.headers.get("content-type")).toBe("video/mp4");
    expect(new Uint8Array(await whole.arrayBuffer())).toEqual(CLIP);
    // A window across git's pipe chunks, the kind a seeking video asks for.
    const mid = await get("/alice/gallery/media/main/clip.mp4", { range: "bytes=700000-900122" });
    expect(mid.status).toBe(206);
    expect(mid.headers.get("content-range")).toBe(`bytes 700000-900122/${CLIP.length}`);
    expect(new Uint8Array(await mid.arrayBuffer())).toEqual(CLIP.subarray(700000, 900123));
    const tail = await get("/alice/gallery/media/main/clip.mp4", { range: "bytes=-10" });
    expect(new Uint8Array(await tail.arrayBuffer())).toEqual(CLIP.subarray(CLIP.length - 10));
  });
  test("a reader that hangs up mid-stream hands its permit back", async () => {
    const res = await get("/alice/gallery/media/main/clip.mp4");
    const reader = res.body!.getReader();
    expect((await reader.read()).done).toBe(false);
    await reader.cancel();
    for (let i = 0; i < 200 && mediaGate.stats().active > 0; i++) await Bun.sleep(10);
    expect(mediaGate.stats().active).toBe(0);
  });
  test("If-Range that still holds gets the range; a stale one gets the whole file", async () => {
    const etag = (await get("/alice/gallery/media/main/art/logo.png")).headers.get("etag") ?? "";
    const held = await get("/alice/gallery/media/main/art/logo.png", { range: "bytes=1-3", "if-range": etag });
    expect(held.status).toBe(206);
    expect(new Uint8Array(await held.arrayBuffer())).toEqual(PNG.subarray(1, 4));
    const stale = await get("/alice/gallery/media/main/art/logo.png", {
      range: "bytes=1-3",
      "if-range": `"${"0".repeat(40)}"`,
    });
    expect(stale.status).toBe(200);
    expect(stale.headers.get("content-range")).toBeNull();
    expect(new Uint8Array(await stale.arrayBuffer())).toEqual(PNG);
  });
  test("the ETag is the blob id, and holding it is a 304", async () => {
    const res = await get("/alice/gallery/media/main/art/logo.png");
    const etag = res.headers.get("etag") ?? "";
    expect(etag).toMatch(/^"[0-9a-f]{40}"$/);
    const again = await get("/alice/gallery/media/main/art/logo.png", { "if-none-match": etag });
    expect(again.status).toBe(304);
    expect(await again.text()).toBe("");
  });
  test("a 304 is answered from the stat alone: no sniff, no permit", async () => {
    const etag = (await get("/alice/gallery/media/main/clip.mp4", { range: "bytes=0-0" })).headers.get("etag") ?? "";
    const held = await holdAll();
    try {
      // Every permit is taken, so a 304 that sniffed would wait here.
      const again = await get("/alice/gallery/media/main/clip.mp4", { "if-none-match": etag });
      expect(again.status).toBe(304);
      expect(again.headers.get("content-type")).toBeNull();
      expect(again.headers.get("etag")).toBe(etag);
    } finally {
      held.forEach((release) => release());
    }
  });
  test("a full commit id is cached for a year; a branch revalidates", async () => {
    const pinned = await get(`/alice/gallery/media/${sha}/art/logo.png`);
    expect(pinned.status).toBe(200);
    expect(pinned.headers.get("cache-control")).toBe("private, max-age=31536000, immutable");
    const moving = await get("/alice/gallery/media/main/art/logo.png");
    expect(moving.headers.get("cache-control")).toBe("private, max-age=0");
  });
  test("?download=1 is an attachment under its own name, never rendered", async () => {
    const res = await get("/alice/gallery/media/main/art/evil.svg?download=1");
    expect(res.headers.get("content-type")).toBe("application/octet-stream");
    expect(res.headers.get("content-disposition")).toStartWith('attachment; filename="evil.svg"');
  });
  test("a private repository is a 404 signed out, and its owner's to see", async () => {
    const out = await get("/alice/vault/media/main/secret.png");
    expect(out.status).toBe(404);
    expect(out.headers.get("content-type") ?? "").not.toStartWith("image/");
    const owner = await get("/alice/vault/media/main/secret.png", { cookie: alice });
    expect(owner.status).toBe(200);
    expect(owner.headers.get("content-type")).toBe("image/png");
  });
  test("a missing file, a tree, and an unsafe ref are the same 404", async () => {
    expect((await get("/alice/gallery/media/main/nope.png")).status).toBe(404);
    expect((await get("/alice/gallery/media/main/art")).status).toBe(404);
    expect((await get("/alice/gallery/media/--output=x/art/logo.png")).status).toBe(404);
  });
});
/** Take every media permit, as sixteen stalled streams would. */
async function holdAll(): Promise<(() => void)[]> {
  const held: (() => void)[] = [];
  while (mediaGate.stats().active < mediaGate.limit) held.push(await mediaGate.acquire());
  return held;
}
const settled = async (): Promise<number> => {
  for (let i = 0; i < 300 && mediaGate.stats().active > 0; i++) await Bun.sleep(10);
  return mediaGate.stats().active;
};
describe("the media permits", () => {
  const clip = async (): Promise<{ dir: string; oid: string }> => {
    const dir = repoDir(app.reposRoot, "alice", "gallery");
    const stat = await readerFor(dir).stat("main:clip.mp4");
    if (!stat.ok) throw new Error("no clip");
    return { dir, oid: stat.value.oid };
  };
  test("a reader that stalls gets its permit taken back", async () => {
    const { dir, oid } = await clip();
    const stream = await blobStream(dir, oid, 0, CLIP.length - 1, { idleMs: 50 });
    if (typeof stream === "string") throw new Error(stream);
    const reader = stream.getReader();
    expect((await reader.read()).done).toBe(false);
    // Then it never pulls again.
    expect(await settled()).toBe(0);
    await expect(reader.read()).rejects.toThrow("idle");
  });
  test("a stream nobody ever reads ends too", async () => {
    const { dir, oid } = await clip();
    const stream = await blobStream(dir, oid, 0, 9, { idleMs: 50 });
    expect(typeof stream).not.toBe("string");
    expect(await settled()).toBe(0);
  });
  test("a reader that leaves while it waits for a permit spends none", async () => {
    const { dir, oid } = await clip();
    const held = await holdAll();
    const gone = new AbortController();
    const waiting = blobStream(dir, oid, 0, 9, { signal: gone.signal });
    expect(mediaGate.stats().waiting).toBe(1);
    gone.abort();
    held.forEach((release) => release());
    expect(await waiting).toBe("gone");
    expect(await settled()).toBe(0);
  });
  test("a full queue is a 503 with Retry-After, not one more waiter", async () => {
    const { dir, oid } = await clip();
    const held = await holdAll();
    const queued = Array.from({ length: MEDIA_QUEUE_MAX }, () => blobStream(dir, oid, 0, 9));
    try {
      expect(mediaGate.stats().waiting).toBe(MEDIA_QUEUE_MAX);
      const res = await get("/alice/gallery/media/main/clip.mp4");
      expect(res.status).toBe(503);
      expect(res.headers.get("retry-after")).toBe("5");
      expect(mediaGate.stats().waiting).toBe(MEDIA_QUEUE_MAX);
      // A small file never queues: it is read through the pool.
      expect((await get("/alice/gallery/media/main/art/logo.png")).status).toBe(200);
    } finally {
      held.forEach((release) => release());
      for (const stream of await Promise.all(queued))
        if (typeof stream !== "string") await stream.cancel();
    }
    expect(await settled()).toBe(0);
  });
});
describe("the blob page's switch", () => {
  const flat = (s: string): string => s.replace(/\s+/g, " ");
  test("Markdown chooses Rendered or Source, beside Raw and Download", async () => {
    const html = flat(await (await get("/alice/gallery/blob/main/README.md")).text());
    expect(html).toContain('aria-current="page" >Rendered');
    expect(html).toContain('href="/alice/gallery/blob/main/README.md?view=source"');
    expect(html).toContain('href="/alice/gallery/raw/main/README.md"');
    expect(html).toContain('href="/alice/gallery/media/main/README.md?download=1"');
    // Raw and Download are actions, not views: never inside the switch.
    const views = /<div class="seg">(.*?)<\/div>/.exec(html)?.[1] ?? "";
    expect(views).toContain("Rendered");
    expect(views).not.toContain("Raw");
    expect(views).not.toContain("Download");
  });
  test("a kind with no renderer keeps its view, no switch, with Raw and Download", async () => {
    const code = flat(await (await get("/alice/gallery/blob/main/src/a.ts")).text());
    expect(code).toContain('class="card code"');
    expect(code).not.toContain('class="blob-view"');
    expect(code).not.toContain(">Rendered");
    expect(code).toContain('href="/alice/gallery/raw/main/src/a.ts"');
    expect(code).toContain('href="/alice/gallery/media/main/src/a.ts?download=1"');
    const png = flat(await (await get("/alice/gallery/blob/main/art/logo.png")).text());
    expect(png).toContain("Binary file");
    expect(png).toContain('href="/alice/gallery/media/main/art/logo.png?download=1"');
  });
});
@@ -0,0 +1,125 @@
/**
 * The registry of file kinds and the byte-range reader, as plain data: every
 * extension the registry names lands on its kind, a name its bytes
 * contradict falls back, and a Range header reads as the window it asks for.
 */
import { describe, expect, test } from "bun:test";
import { parseRange } from "../src/http/range.ts";
import { fileKind, KNOWN_EXTENSIONS, mediaType, type FileKind } from "../src/web/render/kinds.ts";
const enc = (s: string): Uint8Array => new TextEncoder().encode(s);
const of = (...b: number[]): Uint8Array => new Uint8Array(b);
const box = (brand: string): Uint8Array => enc(`\0\0\0\x20ftyp${brand}`);
/** One true head per extension, and the kind and type it must come back as. */
const SAMPLES: Record<string, [Uint8Array, FileKind, string]> = {
  png: [of(0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0), "image", "image/png"],
  jpg: [of(0xff, 0xd8, 0xff, 0xe0), "image", "image/jpeg"],
  jpeg: [of(0xff, 0xd8, 0xff, 0xe1), "image", "image/jpeg"],
  gif: [enc("GIF89a\0"), "image", "image/gif"],
  webp: [enc("RIFF\0\0\0\0WEBPVP8 "), "image", "image/webp"],
  avif: [box("avif"), "image", "image/avif"],
  bmp: [enc("BM\0\0"), "image", "image/bmp"],
  ico: [of(0, 0, 1, 0, 1), "image", "image/x-icon"],
  svg: [enc('<?xml version="1.0"?>\n<svg xmlns="http://www.w3.org/2000/svg"/>'), "svg", "image/svg+xml"],
  pdf: [enc("%PDF-1.7\n"), "pdf", "application/pdf"],
  mp3: [enc("ID3\x04\0"), "audio", "audio/mpeg"],
  ogg: [enc("OggS\0"), "audio", "audio/ogg"],
  oga: [enc("OggS\0"), "audio", "audio/ogg"],
  wav: [enc("RIFF\0\0\0\0WAVEfmt "), "audio", "audio/wav"],
  flac: [enc("fLaC\0"), "audio", "audio/flac"],
  m4a: [box("M4A "), "audio", "audio/mp4"],
  aac: [of(0xff, 0xf1, 0x50, 0x80), "audio", "audio/aac"],
  mp4: [box("isom"), "video", "video/mp4"],
  m4v: [box("M4V "), "video", "video/mp4"],
  webm: [of(0x1a, 0x45, 0xdf, 0xa3, 0), "video", "video/webm"],
  mov: [box("qt  "), "video", "video/quicktime"],
  stl: [enc("solid cube\n"), "model3d", "model/stl"],
  glb: [enc("glTF\x02\0\0\0"), "model3d", "model/gltf-binary"],
  csv: [enc("a,b\n1,2\n"), "csv", "application/octet-stream"],
  tsv: [enc("a\tb\n"), "csv", "application/octet-stream"],
  gltf: [enc('{"asset":{}}'), "model3d", "application/octet-stream"],
  json: [enc("{}"), "config", "application/octet-stream"],
  jsonc: [enc("{}"), "config", "application/octet-stream"],
  yaml: [enc("a: 1\n"), "config", "application/octet-stream"],
  yml: [enc("a: 1\n"), "config", "application/octet-stream"],
  toml: [enc("a = 1\n"), "config", "application/octet-stream"],
  ini: [enc("[a]\n"), "config", "application/octet-stream"],
  conf: [enc("a 1\n"), "config", "application/octet-stream"],
  md: [enc("# a\n"), "markdown", "application/octet-stream"],
  markdown: [enc("# a\n"), "markdown", "application/octet-stream"],
  mdx: [enc("# a\n"), "markdown", "application/octet-stream"],
};
describe("the registry of file kinds", () => {
  test("every extension it names maps to its kind and its one media type", () => {
    expect(KNOWN_EXTENSIONS.length).toBeGreaterThan(0);
    for (const ext of KNOWN_EXTENSIONS) {
      const sample = SAMPLES[ext];
      expect(sample, `a sample for .${ext}`).toBeDefined();
      const [head, kind, type] = sample!;
      expect([ext, fileKind(`dir/file.${ext}`, head)]).toEqual([ext, kind]);
      expect([ext, mediaType(`dir/FILE.${ext.toUpperCase()}`, head)]).toEqual([ext, type]);
    }
  });
  test("a name the bytes contradict falls back to octet-stream and to what the bytes are", () => {
    expect(mediaType("logo.png", enc("not a png"))).toBe("application/octet-stream");
    expect(fileKind("logo.png", enc("not a png"))).toBe("text");
    expect(mediaType("clip.mp4", of(0, 1, 2, 0))).toBe("application/octet-stream");
    expect(fileKind("clip.mp4", of(0, 1, 2, 0))).toBe("binary");
    // A PNG wearing another format's name is still not served as that format.
    const png = SAMPLES.png![0];
    expect(mediaType("photo.jpg", png)).toBe("application/octet-stream");
    // An SVG with no root element is not an SVG.
    expect(mediaType("x.svg", enc("<html><script>1</script></html>"))).toBe(
      "application/octet-stream",
    );
  });
  test("nothing it can answer is HTML or script", () => {
    for (const ext of [...KNOWN_EXTENSIONS, "html", "htm", "js", "mjs", "xhtml", "xml"]) {
      const head = SAMPLES[ext]?.[0] ?? enc("<script>alert(1)</script>");
      const type = mediaType(`a.${ext}`, head);
      expect(type).not.toMatch(/html|javascript|ecmascript|xml(?!$)/);
      expect(type).not.toBe("text/html");
    }
    expect(mediaType("index.html", enc("<html>"))).toBe("application/octet-stream");
  });
  test("a file with no known extension is text or binary by its bytes", () => {
    expect(fileKind("Makefile", enc("all:\n"))).toBe("text");
    expect(fileKind("src/a.ts", enc("export {}\n"))).toBe("text");
    expect(fileKind("blob.dat", of(1, 0, 2))).toBe("binary");
    expect(fileKind(".png", SAMPLES.png![0])).toBe("binary");
  });
});
describe("a byte range", () => {
  test("a single range, clamped to the file", () => {
    expect(parseRange("bytes=0-9", 100)).toEqual({ start: 0, end: 9 });
    expect(parseRange("bytes=90-200", 100)).toEqual({ start: 90, end: 99 });
    expect(parseRange("bytes=10-", 100)).toEqual({ start: 10, end: 99 });
  });
  test("a suffix range is the last N bytes", () => {
    expect(parseRange("bytes=-10", 100)).toEqual({ start: 90, end: 99 });
    expect(parseRange("bytes=-500", 100)).toEqual({ start: 0, end: 99 });
  });
  test("past the end is unsatisfiable", () => {
    expect(parseRange("bytes=100-", 100)).toBe("unsatisfiable");
    expect(parseRange("bytes=-0", 100)).toBe("unsatisfiable");
    expect(parseRange("bytes=0-", 0)).toBe("unsatisfiable");
  });
  test("anything else is the whole file", () => {
    expect(parseRange(null, 100)).toBeNull();
    expect(parseRange("bytes=0-1,5-6", 100)).toBeNull();
    expect(parseRange("items=0-1", 100)).toBeNull();
    expect(parseRange("bytes=9-1", 100)).toBeNull();
    expect(parseRange("bytes=-", 100)).toBeNull();
    expect(parseRange("bytes=99999999999999999999-", 100)).toBeNull();
  });
});
@@ -25,7 +25,7 @@
const PIPING: Record<string, { sites: string[]; reason: string }> = {
  "src/crew/workdir.ts": { sites: ["stderr"], reason: "stderr read to its end alongside the exit; stdout ignored" },
  "src/git/env.ts": { sites: ["stdout+stderr", "stdout+stderr"], reason: "stdout and stderr both read to the end, finally clears the timer" },
  "src/git/local.ts": { sites: ["stdout+stderr", "stdout+stderr"], reason: "stdout and stderr both read to the end" },
  "src/git/read.ts": { sites: ["stdout"], reason: "cat-file stdout is read by the long-lived reader; stderr ignored" },
  "src/git/read.ts": { sites: ["stdout", "stdout"], reason: "cat-file stdout is read by the long-lived reader, and by blobStream until its window ends or the reader hangs up (then killed); stderr ignored" },
  "src/git/smart-http.ts": { sites: ["stdout"], reason: "stdout is a pull-pipe FIFO (the response body, read on demand); stderr ignored" },
  "src/git/smart-ssh.ts": { sites: ["stdout+stderr"], reason: "pump() drains stdout (a pull-pipe FIFO) and stderr into the channel" },
  "src/kernel/data_home.ts": { sites: ["stdout"], reason: "spawnSync buffers stdout; stderr ignored" },
		@@ -180,7 +180,8 @@ src/
180	180	header, settings shell, the one 404), flash, layout (CSP+nonce,
181	181	topbar, theme switcher), client (progressive enhancement, as a
182	182	string), style/{base,components,content} assembled by style.ts,
183		markdown, highlight, code, worddiff, diff-view, live, and
	183	markdown, highlight, code, worddiff, diff-view, live, render/ (the
	184	file-kind registry, kinds.ts, and one renderer per kind), and
184	185	pages/{site,repo,issues,pulls,pull,thread,commits,refs,search,
185	186	org,account,settings}, and studio/ — the app-as-the-unit
186	187	surface, one module per concept: words (the vocabulary every
		@@ -359,6 +359,7 @@ This page is the front door. Everything else is a file of its own under
359	359	- [`docs/FEED.md`](docs/FEED.md) — the feed: what happened, ranked, each item with why it is there. The home composer keeps your draft across a reload.
360	360	- [`docs/PULLS.md`](docs/PULLS.md) — resolving a conflicted pull without a clone.
361	361	- [`docs/GIT.md`](docs/GIT.md) — cloning over HTTPS and SSH, and partial clones of big repositories.
	362	- [`docs/features/file-media.md`](docs/features/file-media.md) — a file in the browser: every file page has Raw and Download (any size), and `/:owner/:name/media/:ref/*path` serves images, audio, video and PDFs under their real type, sandboxed, with byte ranges.
362	363	- [`docs/ROLES.md`](docs/ROLES.md) — guest, member and admin, and what each may do.
363	364	- [`docs/MCP.md`](docs/MCP.md) — the forge as a connector; [`docs/STANDARDS.md`](docs/STANDARDS.md) is what it answers by default: `.well-known`, feeds, cards, the manifest.
364	365
		@@ -0,0 +1,60 @@
	1	# Files in the browser: the media route and the kind registry
	2
	3	The foundation for rendering files on the blob page (#7329, #7330).
	4
	5	## The media route
	6
	7	`GET /:owner/:name/media/:ref/*path` serves a file's bytes under a real
	8	media type, for `<img>`, `<video>`, `<audio>`, `<object>` and `fetch()`.
	9
	10	- Types come only from the allowlist in `src/web/render/kinds.ts`
	11	(png, jpeg, gif, webp, avif, bmp, ico, svg, pdf, mp3, ogg/oga, wav, flac,
	12	m4a, aac, mp4, m4v, webm, mov, stl, glb). The extension proposes a type and
	13	the file's first bytes confirm it; a mismatch, or any other file, is
	14	`application/octet-stream`. Never HTML, never script.
	15	- Headers: `X-Content-Type-Options: nosniff` and
	16	`Content-Security-Policy: sandbox; default-src 'none'; img-src 'self' data:; media-src 'self'; style-src 'unsafe-inline'`,
	17	so an SVG with a `<script>` is drawn and never runs.
	18	- Range: one `bytes=` range answers 206 with `Content-Range`; a suffix
	19	(`bytes=-N`) is the last N bytes; past the end is 416. A list of ranges is
	20	answered whole, which the RFC allows. `If-Range` is honoured.
	21	- Caching: `ETag` is the blob id, and `If-None-Match` answers 304. A
	22	full commit id as `:ref` is `private, max-age=31536000, immutable`; a
	23	branch or tag is `private, max-age=0` and revalidates.
	24	- Access is `/raw`'s: a repository you cannot see is the same 404 as one
	25	that is not there.
	26	- Streaming: a file over 1 MB streams off `git cat-file` and is never
	27	held whole. Streams take permits from their own gate (`mediaGate`,
	28	`FORGE_MEDIA_CONCURRENCY`, default 16), so a page of videos never takes
	29	the plumbing permits every page needs. The permit is taken before the
	30	stream exists; a stream gives it back when it ends, when its reader hangs
	31	up, or when nothing has been pulled for 30 s (`MEDIA_IDLE_MS`), so a
	32	reader that never reads cannot hold one for good. With 32 requests
	33	already waiting (`MEDIA_QUEUE_MAX`) the next is a 503 with
	34	`Retry-After: 5`. A file of 1 MB or less never waits: it is read through
	35	the pooled reader.
	36	- A 304 is answered from the blob id alone, before any byte is read.
	37	- `?download=1` sends the file as an attachment under its own name, at any
	38	size. The blob page's Download button points here.
	39
	40	## The kind registry
	41
	42	`fileKind(path, firstBytes)` answers `image \| svg \| pdf \| audio \| video \|
	43	csv \| model3d \| config \| markdown \| text \| binary`. `RENDER_CAP` is the
	44	largest file each kind draws inline; over it, the page offers Download.
	45
	46	`kinds.ts` is plain data with no web dependency: the editor and the
	47	terminal ask it whether a path is Markdown, as the blob page does.
	48
	49	`renderBlob` asks `render/renderers.ts`. A kind in `RENDERERS` is drawn by
	50	its module; a kind with none yet keeps the source or binary view it always
	51	had. Where a renderer exists the header offers `Rendered \| Source`; every
	52	file's header has the Raw and Download buttons.
	53
	54	## Adding a renderer
	55
	56	Write `src/web/render/<kind>.ts` exporting a `Renderer` (plain data in:
	57	path, size, body, and `media`, the media route's address; HTML out), and
	58	name it once in `RENDERERS` (`render/renderers.ts`). Point any element
	59	that needs the bytes at `input.media`, never at an inline copy, so
	60	repository content only ever runs under the media route's sandbox. `render/markdown.ts` is the example.
		@@ -195,6 +195,13 @@ const limitFromEnv = (name: string, fallback: number): number => {
195	195	*/
196	196	export const gitGate = new Gate(limitFromEnv("FORGE_GIT_CONCURRENCY", 8));
197	197
	198	/**
	199	* Blob streams for the media route: a video seeks for minutes, so a stream
	200	* holds its permit until the last byte or the reader hangs up. A pool of its
	201	* own, so a page of videos never takes the plumbing permits every page needs.
	202	*/
	203	export const mediaGate = new Gate(limitFromEnv("FORGE_MEDIA_CONCURRENCY", 16));
	204
198	205	/**
199	206	* How many pack transfers a host of `bytes` memory runs at once by default:
200	207	* one per 64 MiB, at least two, at most sixteen. A push is not one process
		@@ -7,8 +7,8 @@
7	7
8	8	import { defect } from "../kernel/error.ts";
9	9	import { err, ok, type Result } from "../kernel/result.ts";
10		import { gitEnv, runGit, runGitRaw, type GitFailure } from "./env.ts";
11		import { isSafeRef } from "./guard.ts";
	10	import { gitEnv, mediaGate, runGit, runGitRaw, type GitFailure } from "./env.ts";
	11	import { isSafeOid, isSafeRef } from "./guard.ts";
12	12
13	13	export type ObjectType = "blob" \| "tree" \| "commit" \| "tag";
14	14
		@@ -603,3 +603,95 @@ export async function refTip(dir: string, ref: string): Promise<string \| undefin
603	603	}
604	604	return undefined;
605	605	}
	606
	607	/** How many requests may wait for a media permit before the next is turned away. */
	608	export const MEDIA_QUEUE_MAX = 32;
	609	/** A stream nobody has pulled from for this long is ended and its permit returned. */
	610	export const MEDIA_IDLE_MS = 30_000;
	611
	612	/**
	613	* Bytes `start`..`end` (inclusive) of the blob `oid`, streamed straight off
	614	* `git cat-file` and never held whole: a 2 GB video costs a pipe, not 2 GB
	615	* of memory. Git cannot seek inside a blob, so the bytes before `start` are
	616	* read and dropped; past `end` the process is killed.
	617	*
	618	* The permit is taken before the stream exists, so there is no stream to
	619	* cancel while it is waiting; a reader whose `signal` aborts in the queue
	620	* gets `"gone"` and the permit goes back unspent. With `MEDIA_QUEUE_MAX`
	621	* already waiting, the answer is `"busy"` (a 503) rather than one more
	622	* waiter. Once open, the stream gives its permit back when it ends, when its
	623	* reader hangs up or its signal aborts, or when nothing has pulled from it
	624	* for `idleMs`, which keeps a reader that never reads from holding a permit
	625	* for good.
	626	*/
	627	export async function blobStream(
	628	dir: string,
	629	oid: string,
	630	start: number,
	631	end: number,
		sentinelcrew 17d ago outdated High: if the reader cancels while this `await` is pending, `cancel()` → `finish()` runs with `release`/`proc` undefined. This line then takes a permit and spawns a `git cat-file` that is never read or killed. Set a `cancelled` flag in `cancel()` and check it here: release and return without spawning. Also add an idle deadline per stream so a reader that never reads cannot hold the permit forever.
	632	limits: { idleMs?: number; queueMax?: number; signal?: AbortSignal } = {},
	633	): Promise<ReadableStream<Uint8Array> \| "busy" \| "gone"> {
	634	if (!isSafeOid(oid)) defect("git.unsafe_oid", `unsafe oid: ${JSON.stringify(oid)}`);
	635	const free = mediaGate.stats();
	636	if (free.active >= free.limit && free.waiting >= (limits.queueMax ?? MEDIA_QUEUE_MAX))
	637	return "busy";
	638	const release = await mediaGate.acquire();
	639	// The reader left while it waited: the permit goes straight back, unspent.
	640	if (limits.signal?.aborted) {
	641	release();
	642	return "gone";
	643	}
	644	const proc = Bun.spawn(["git", "cat-file", "blob", oid], {
	645	cwd: dir,
	646	env: gitEnv(),
	647	stdin: "ignore",
	648	stdout: "pipe",
	649	stderr: "ignore",
	650	});
	651	const source = proc.stdout.getReader();
	652	const idleMs = limits.idleMs ?? MEDIA_IDLE_MS;
	653	let idle: ReturnType<typeof setTimeout> \| undefined;
	654	let offset = 0;
	655	let ended = false;
	656	const finish = (): void => {
	657	if (ended) return;
	658	ended = true;
	659	clearTimeout(idle);
	660	limits.signal?.removeEventListener("abort", finish);
	661	proc.kill("SIGKILL");
	662	release();
	663	};
	664	limits.signal?.addEventListener("abort", finish, { once: true });
	665	return new ReadableStream<Uint8Array>({
	666	start(controller) {
	667	// Armed from the first moment, so a response nobody ever reads ends too.
	668	idle = setTimeout(() => {
	669	finish();
	670	controller.error(new Error("media stream idle"));
	671	}, idleMs);
	672	},
	673	async pull(controller) {
	674	idle?.refresh();
	675	while (true) {
	676	const next = await source.read();
	677	const chunk = next.value;
	678	if (next.done \|\| chunk === undefined) break;
	679	const from = offset;
	680	offset += chunk.length;
	681	if (offset <= start) continue;
	682	// A copy, not a view: Bun may recycle the pipe's buffer.
	683	controller.enqueue(
	684	chunk.slice(Math.max(0, start - from), Math.min(chunk.length, end + 1 - from)),
	685	);
	686	if (offset > end) break;
	687	return;
	688	}
	689	if (ended) return;
	690	finish();
	691	controller.close();
	692	},
	693	cancel() {
	694	finish();
	695	},
	696	});
	697	}
		@@ -0,0 +1,43 @@
	1	/**
	2	* One HTTP byte range (RFC 9110 §14), as plain data. A video seeks and a PDF
	3	* streams by asking for a window of a file; this reads which window.
	4	*
	5	* Only a single range is honoured. A list of ranges, a unit other than
	6	* bytes, or a header that does not parse is ignored, which the RFC allows:
	7	* the whole file goes out with 200, and nothing is refused that a browser
	8	* would ask for.
	9	*/
	10
	11	/** Inclusive at both ends, as `Content-Range` writes it. */
	12	export interface ByteRange {
	13	readonly start: number;
	14	readonly end: number;
	15	}
	16
	17	/**
	18	* The window `header` asks for in a file of `size` bytes: a range, `null`
	19	* for the whole file, or `"unsatisfiable"` (416) when it starts past the end.
	20	*/
	21	export function parseRange(
	22	header: string \| null,
	23	size: number,
	24	): ByteRange \| "unsatisfiable" \| null {
	25	if (header === null) return null;
	26	const match = /^bytes=(\d)-(\d)$/.exec(header.trim());
	27	if (match === null) return null;
	28	const [, first = "", last = ""] = match;
	29	if (first === "" && last === "") return null;
	30	if (first === "") {
	31	// A suffix: the last N bytes.
	32	const length = Number(last);
	33	if (!Number.isSafeInteger(length)) return null;
	34	if (length === 0 \|\| size === 0) return "unsatisfiable";
	35	return { start: Math.max(0, size - length), end: size - 1 };
	36	}
	37	const start = Number(first);
	38	const end = last === "" ? Number.MAX_SAFE_INTEGER : Number(last);
	39	if (!Number.isSafeInteger(start) \|\| !Number.isSafeInteger(end)) return null;
	40	if (end < start) return null;
	41	if (start >= size) return "unsatisfiable";
	42	return { start, end: Math.min(end, size - 1) };
	43	}
		@@ -205,6 +205,7 @@ import {
205	205	repoSyncSubmit,
206	206	treePage,
207	207	} from "../web/pages/repo.ts";
	208	import { mediaBlob } from "../web/pages/media.ts";
208	209	import {
209	210	branchCreate,
210	211	branchDelete,
		@@ -1324,6 +1325,7 @@ export function buildRouter(): Router<Handler> {
1324	1325	router.add("GET", "/:owner/:name/tree/:ref/*path", treePage);
1325	1326	router.add("GET", "/:owner/:name/blob/:ref/*path", blobPage);
1326	1327	router.add("GET", "/:owner/:name/raw/:ref/*path", rawBlob);
	1328	router.add("GET", "/:owner/:name/media/:ref/*path", mediaBlob);
1327	1329	// Every file path at a ref, for the fuzzy finder. JSON, `canRead`-gated.
1328	1330	router.add("GET", "/:owner/:name/paths/:ref", pathsJson);
1329	1331
		@@ -45,6 +45,7 @@ import { stopMirroring } from "../../actions/mirror_target.ts";
45	45	import { getMirrorTarget, mirrorLine } from "../../core/migrate_mirror.ts";
46	46	import { canAdmin, DECISIONS_BY, getRepoById, type Repo } from "../../core/repos.ts";
47	47	import type { TreeEntry } from "../../git/read.ts";
	48	import { fileKind, NO_BYTES } from "../../web/render/kinds.ts";
48	49	import type { RepoFacts, RepoSource } from "../../view/source.ts";
49	50	import { detectLanguage, highlightLines } from "../../web/highlight.ts";
50	51	import type { Ctx, Screen } from "../app/app.ts";
		@@ -257,7 +258,7 @@ type TabKind = "image" \| "markdown" \| "code";
257	258	/** `README.md`, `NOTES.markdown`, `readme` — the prose a repo opens on. */
258	259	function isMarkdownPath(path: string): boolean {
259	260	const name = path.slice(path.lastIndexOf("/") + 1);
260		return /\.(md\|markdown)$/i.test(name) \|\| /^readme(\.\|$)/i.test(name);
	261	return fileKind(name, NO_BYTES) === "markdown" \|\| /^readme(\.\|$)/i.test(name);
261	262	}
262	263
263	264	class Tab {
		@@ -17,6 +17,7 @@
17	17	* preview are `editorJs`, and none of them is load-bearing.
18	18	*/
19	19
	20	import { fileKind, NO_BYTES } from "../render/kinds.ts";
20	21	import { editFiles, MAX_FILE_BYTES } from "../../actions/contents.ts";
21	22	import { contribute } from "../../actions/contribute.ts";
22	23	import { NO_WRITE_ACCESS } from "../../actions/pull_words.ts";
		@@ -555,8 +556,9 @@ function mirrorHtml(content: string, lang: string): string {
555	556	.join("");
556	557	}
557	558
	559	/** A file not yet named is written as prose until its name says otherwise. */
558	560	const isMarkdownPath = (path: string): boolean =>
559		path === "" \|\| /\.(md\|markdown\|mdx)$/i.test(path);
	561	path === "" \|\| fileKind(path, NO_BYTES) === "markdown";
560	562
561	563	/**
562	564	* The repository and the situation an edit route needs, or the refusal. A
		@@ -0,0 +1,126 @@
	1	/**
	2	* GET /:owner/:name/media/:ref/*path — a file's bytes under a real media
	3	* type, so a browser can show them in `<img>`, `<video>`, `<audio>`,
	4	* `<object>` and `fetch()`. `/raw` is the text you would copy; this is the
	5	* picture, the song, the PDF.
	6	*
	7	* The type comes only from the allowlist in `render/kinds.ts`, confirmed by
	8	* the file's own first bytes; anything else is `application/octet-stream`.
	9	* Never HTML, never script, and the sandbox policy below means an SVG that
	10	* carries a `<script>` is drawn as a picture and never runs on this origin.
	11	*
	12	* Bytes stream from git, never buffered whole, with HTTP Range so a video
	13	* seeks and a PDF streams. Access is `/raw`'s: a repository you cannot see
	14	* answers the same 404 as one that is not there.
	15	*/
	16
	17	import { isSafeRef, isSafeTreePath } from "../../git/guard.ts";
	18	import { blobStream, readerFor } from "../../git/read.ts";
	19	import { repoDir } from "../../git/repos.ts";
	20	import type { Ctx } from "../../http/context.ts";
	21	import { holds } from "../../http/etag.ts";
	22	import { parseRange } from "../../http/range.ts";
	23	import { loadRepo, notFoundPage } from "../chrome.ts";
	24	import { HEAD_BYTES, mediaType } from "../render/kinds.ts";
	25
	26	/** The policy every media answer leaves with. */
	27	export const MEDIA_POLICY =
	28	"sandbox; default-src 'none'; img-src 'self' data:; media-src 'self'; style-src 'unsafe-inline'";
	29
	30	/** At or under this, the pooled reader hands the body back without a spawn. */
	31	const SMALL_BYTES = 1_000_000;
	32
	33	const FULL_OID = /^(?:[0-9a-f]{40}\|[0-9a-f]{64})$/;
	34
	35	/** `/media/:ref/:path`, the address a renderer points its element at. */
	36	export const mediaHref = (base: string, ref: string, path: string): string =>
	37	`${base}/media/${encodeURIComponent(ref)}/${path}`;
	38
	39	export async function mediaBlob(ctx: Ctx): Promise<Response> {
	40	const repo = loadRepo(ctx);
	41	if (!repo) return notFoundPage(ctx);
	42	const ref = ctx.params.ref ?? "";
	43	const path = ctx.params.path ?? "";
	44	if (!isSafeRef(ref) \|\| !isSafeTreePath(path)) return notFoundPage(ctx);
	45
	46	const dir = repoDir(ctx.app.reposRoot, repo.ownerName, repo.name);
	47	const reader = readerFor(dir);
	48	const stat = await reader.stat(`${ref}:${path}`).catch(() => undefined);
	49	if (stat === undefined \|\| !stat.ok \|\| stat.value.type !== "blob")
	50	return notFoundPage(ctx);
	51	const { oid, size } = stat.value;
	52
	53	const download = ctx.url.searchParams.get("download") === "1";
	54	const etag = `"${oid}"`;
	55	const headers: Record<string, string> = {
	56	"x-content-type-options": "nosniff",
	57	"content-security-policy": MEDIA_POLICY,
	58	"referrer-policy": "same-origin",
	59	"accept-ranges": "bytes",
	60	etag,
	61	// A full commit id names bytes that can never change; a branch does not.
	62	"cache-control": FULL_OID.test(ref)
	63	? "private, max-age=31536000, immutable"
		sentinelcrew 17d ago outdated The sniff for a large file takes a `mediaGate` permit and waits with no timeout. Once 16 stalled streams hold the gate, this await hangs every later request for a large file. Cap the queue and answer 503 + Retry-After instead of waiting without end.
	64	: "private, max-age=0",
	65	};
	66	if (download) headers["content-disposition"] = attachment(path);
	67	// The tag is the blob id `stat` already gave, so a revalidation reads no byte.
	68	if (holds(ctx.request, etag)) return new Response(null, { status: 304, headers });
	69
	70	// A small file is read whole through the pool; a large one is sniffed from
	71	// its first bytes and streamed, so its size never becomes memory.
	72	let body: Uint8Array \| undefined;
	73	let head: Uint8Array;
	74	if (size <= SMALL_BYTES) {
	75	const object = await reader.object(oid).catch(() => undefined);
	76	if (object === undefined \|\| !object.ok) return notFoundPage(ctx);
	77	body = object.value.body;
	78	head = body.subarray(0, HEAD_BYTES);
	79	} else {
	80	const sniff = await blobStream(dir, oid, 0, HEAD_BYTES - 1, { signal: ctx.request.signal });
	81	if (sniff === "busy" \|\| sniff === "gone") return busy();
		sentinelcrew 17d ago outdated The ETag is known from `stat` (line 51), so check `holds()` before the object read at lines 56-59 and before the sniff. A revalidation should not read up to 1 MB.
	82	head = await new Response(sniff).bytes();
	83	}
	84	headers["content-type"] = download ? "application/octet-stream" : mediaType(path, head);
	85
	86	// A Range past an If-Range that no longer holds gets the whole new file.
	87	const ifRange = ctx.request.headers.get("if-range");
	88	const range =
	89	ifRange === null \|\| ifRange === etag
	90	? parseRange(ctx.request.headers.get("range"), size)
	91	: null;
	92	if (range === "unsatisfiable") {
	93	headers["content-range"] = `bytes */${size}`;
	94	return new Response(null, { status: 416, headers });
	95	}
	96	const start = range?.start ?? 0;
	97	const end = range?.end ?? size - 1;
	98	if (range !== null) headers["content-range"] = `bytes ${start}-${end}/${size}`;
	99	const status = range === null ? 200 : 206;
	100	if (size === 0) return new Response(null, { status, headers });
	101	// A streamed body goes out chunked (Bun does not keep a length on a large
	102	// stream); `Content-Range` still names the whole size, which is what a
	103	// seeking player reads.
	104	if (body === undefined) {
	105	const stream = await blobStream(dir, oid, start, end, { signal: ctx.request.signal });
	106	if (stream === "busy" \|\| stream === "gone") return busy();
	107	return new Response(stream, { status, headers });
	108	}
	109	headers["content-length"] = String(end + 1 - start);
	110	return new Response(body.subarray(start, end + 1), { status, headers });
	111	}
	112
	113	/** Every media permit is held and the queue is full: come back shortly. */
	114	function busy(): Response {
	115	return new Response("Busy streaming other files; try again shortly.\n", {
	116	status: 503,
	117	headers: { "content-type": "text/plain; charset=utf-8", "retry-after": "5", "cache-control": "no-store" },
	118	});
	119	}
	120
	121	/** Saved under its own name, quoted so no byte of it can end the header. */
	122	function attachment(path: string): string {
	123	const name = path.slice(path.lastIndexOf("/") + 1);
	124	const plain = name.replace(/[^\x20-\x7e]\|["\\]/g, "_");
	125	return `attachment; filename="${plain}"; filename*=UTF-8''${encodeURIComponent(name)}`;
	126	}