Public: anyone can read this; the crew can change it.
A .zip, .tar or .tar.gz file in a repository shows the list of what is inside it, without unpacking it #10325
Check · Review · Land · Live done — Landed and live in production.
torque/forge:torque/10289-a-zip-tar-or-tar-gz-file-in-a-repository → main 4 commits fork 4 threads · 4 unresolved
11 files
changed +649
−5
docs/design/areas/render.md +4
| @@ -8,3 +8,7 @@ No script, no pdf.js. | ||
| 8 | 8 | - `.pdf-fallback` — inside the object, shown only when the viewer cannot draw: an "Open the PDF" link and a `.btn` Download (`?download=1`). |
| 9 | 9 | |
| 10 | 10 | The header shows the size (`1.2 MB · PDF`) through `Rendered.summary`. Both themes use tokens only. |
| 11 | ||
| 12 | ## Archive listing | |
| 13 | ||
| 14 | A .zip, .tar or .tar.gz is drawn by `src/web/render/archive-view.ts` as the existing `.table` alone, with no `.card` around it (path, size, modified; `data-dense`), and a `tfoot` for the count and unpacked size. No new class: `.archive-view` is only a hook on the table and has no rule. How it works: `docs/features/archive-listing.md`. A list that cannot be read falls back to the Download card with the reason as its one sentence. |
docs/features/archive-listing.md added +34
| @@ -0,0 +1,34 @@ | ||
| 1 | # Archive listing | |
| 2 | ||
| 3 | A `.zip`, `.tar` or `.tar.gz` (also `.tgz`) in a repository shows what is inside it, without unpacking it to disk. Code: `src/web/render/archive.ts` (the bounded reader, `listArchive`), `archive-view.ts` (the page), `src/tui/ui/archive-note.ts` (the terminal). | |
| 4 | ||
| 5 | ## What opens | |
| 6 | ||
| 7 | The name picks the kind and the bytes confirm it (zip signature, `ustar` marker, gzip signature). A `.gz` counts only as the end of `.tar.gz`. A file named `.zip` whose bytes are not a zip stays a plain binary file. | |
| 8 | ||
| 9 | ## What is shown | |
| 10 | ||
| 11 | - **Page:** a table of path, size and modified date (UTC), sorted by path, with the entry count and the unpacked size under it. Names are escaped; control and bidi-override characters show as `�`. | |
| 12 | - **Terminal:** in the workspace, the file's note gives the format, the entry count and the unpacked size (`zip, 14 entries, 3.4 MB unpacked.`). Same gating as the page. | |
| 13 | ||
| 14 | ## Limits | |
| 15 | ||
| 16 | | Limit | Value | Where | | |
| 17 | |---|---|---| | |
| 18 | | Entries listed | 2,000 (the rest are counted, and the footer says how many are not listed) | `ENTRY_CAP` | | |
| 19 | | Archive file size | 50 MB | `RENDER_CAP.archive` | | |
| 20 | | Unpacked size of a `.tar.gz` | 128 MB (gunzip stops there) | `GUNZIP_CAP` | | |
| 21 | ||
| 22 | Zip and tar are read from their directory and headers only; no entry's contents are decompressed. Only `.tar.gz` is gunzipped, in memory, under the 128 MB cap. | |
| 23 | ||
| 24 | ## Fallbacks | |
| 25 | ||
| 26 | When the list cannot be shown, the page gives the Download card titled "Archive", with one sentence: | |
| 27 | ||
| 28 | - Over 50 MB: "This archive is over 50 MB, too large to list here. Download it instead." | |
| 29 | - A `.tar.gz` past the cap: "This archive unpacks to more than 128 MB, so its list is not read here." | |
| 30 | - Encrypted zip: "This zip is encrypted, so its list is not shown." | |
| 31 | - Damaged or cut short: "This archive is damaged or cut short, so its list cannot be read." | |
| 32 | - Damaged gzip: "This archive is damaged, so its list cannot be read." | |
| 33 | ||
| 34 | The terminal shows the same sentences. |
src/tui/screens/workspace.ts +5 −1
| @@ -79,6 +79,7 @@ import { | ||
| 79 | 79 | } from "../ui/image.ts"; |
| 80 | 80 | import { fill, len, split } from "../ui/layout.ts"; |
| 81 | 81 | import { renderMarkdown, type Line } from "../ui/markdown.ts"; |
| 82 | import { archiveNote } from "../ui/archive-note.ts"; | |
| 82 | 83 | import { mediaNote } from "../ui/media-note.ts"; |
| 83 | 84 | import { tokColor, type Tokens } from "../ui/theme.ts"; |
| 84 | 85 | import { card, emptyState, input, scrollbar } from "../ui/widgets.ts"; |
| @@ -740,7 +741,10 @@ export function workspaceScreen( | ||
| 740 | 741 | // something with. Everything else keeps the honest note: showing |
| 741 | 742 | // bytes as text is how a terminal ends up beeping at you. |
| 742 | 743 | if (looksLikeImage(t.path)) picture = read.value.body; |
| 743 | else note = `Binary file, ${formatSize(read.value.size)}.`; | |
| 744 | else | |
| 745 | note = | |
| 746 | archiveNote(t.path, read.value.body, read.value.size, formatSize) ?? | |
| 747 | `Binary file, ${formatSize(read.value.size)}.`; | |
| 744 | 748 | } else raw = read.value.body; |
| 745 | 749 | } |
| 746 | 750 | entry = { note, raw, picture }; |
src/tui/ui/archive-note.ts added +26
| @@ -0,0 +1,26 @@ | ||
| 1 | /** | |
| 2 | * What the terminal says for a .zip, .tar or .tar.gz: how many entries it | |
| 3 | * holds and how big they are unpacked, read through `listArchive` and gated | |
| 4 | * as the page is (the kind from name and bytes, no more than the archive | |
| 5 | * size cap). `undefined` for any file that is not a readable-sized archive. | |
| 6 | */ | |
| 7 | ||
| 8 | import { count, listArchive, overSize } from "../../web/render/archive.ts"; | |
| 9 | import { fileKind, HEAD_BYTES, RENDER_CAP } from "../../web/render/kinds.ts"; | |
| 10 | ||
| 11 | const NAME = { zip: "zip", tar: "tar", "tar.gz": "tar.gz" } as const; | |
| 12 | ||
| 13 | export function archiveNote( | |
| 14 | path: string, | |
| 15 | body: Uint8Array, | |
| 16 | size: number, | |
| 17 | /** The terminal's own size formatter, so units read as elsewhere. */ | |
| 18 | fmt: (bytes: number) => string, | |
| 19 | ): string | undefined { | |
| 20 | if (fileKind(path, body.subarray(0, HEAD_BYTES)) !== "archive") return undefined; | |
| 21 | if (size > RENDER_CAP.archive) return `${fmt(size)} — ${overSize(RENDER_CAP.archive)}`; | |
| 22 | const listed = listArchive(body); | |
| 23 | if ("error" in listed) return listed.error; | |
| 24 | const n = listed.count; | |
| 25 | return `${NAME[listed.format]}, ${count(n)} ${n === 1 ? "entry" : "entries"}, ${fmt(listed.bytes)} unpacked.`; | |
| 26 | } |
src/web/pages/repo.ts +15 −4
| @@ -84,8 +84,9 @@ import { type CodeRows, rowsMarkup } from "../code_rows.ts"; | ||
| 84 | 84 | import { rowsOffThread } from "../code_rows_pool.ts"; |
| 85 | 85 | import { flashTo, fragment, page, redirect } from "../layout.ts"; |
| 86 | 86 | import { renderMarkdown } from "../markdown.ts"; |
| 87 | import { overSize } from "../render/archive.ts"; | |
| 87 | 88 | import { csvJs, isCsvPath } from "../render/csv.ts"; |
| 88 | import { HEAD_BYTES } from "../render/kinds.ts"; | |
| 89 | import { fileKind, HEAD_BYTES, RENDER_CAP } from "../render/kinds.ts"; | |
| 89 | 90 | import { RENDERERS, rendererFor } from "../render/renderers.ts"; |
| 90 | 91 | import { mediaHref } from "./media.ts"; |
| 91 | 92 | import { browseBase, localScope, type RepoScope } from "../scope.ts"; |
| @@ -699,6 +700,7 @@ export function blobBody( | ||
| 699 | 700 | highlighted?: (ms: number) => void, |
| 700 | 701 | all = false, |
| 701 | 702 | ready?: CodeRows, |
| 703 | why?: string, | |
| 702 | 704 | ): { content: Html; lines: number } { |
| 703 | 705 | const encoded = encodeURIComponent(ref); |
| 704 | 706 | if (blob.kind === "binary") { |
| @@ -706,8 +708,8 @@ export function blobBody( | ||
| 706 | 708 | lines: 0, |
| 707 | 709 | content: empty({ |
| 708 | 710 | icon: "file", |
| 709 | title: "Binary file", | |
| 710 | note: `${fileSize(blob.size)} — not shown, but you can download it.`, | |
| 711 | title: why !== undefined ? "Archive" : "Binary file", | |
| 712 | note: why !== undefined ? `${fileSize(blob.size)} · ${why}` : `${fileSize(blob.size)} — not shown, but you can download it.`, | |
| 711 | 713 | action: btn("Download", { |
| 712 | 714 | href: downloadHref(base, ref, path), |
| 713 | 715 | variant: "primary", |
| @@ -1095,6 +1097,12 @@ function blobRenderer(scope: RepoScope, path: string, blob: Blob): ReturnType<ty | ||
| 1095 | 1097 | return found?.render === RENDERERS.pdf && scope.kind !== "local" ? undefined : found; |
| 1096 | 1098 | } |
| 1097 | 1099 | |
| 1100 | /** Why an archive too big to list is not listed, as one sentence. */ | |
| 1101 | function overCap(path: string, blob: Blob): string | undefined { | |
| 1102 | if (blob.kind !== "binary" || fileKind(path, blob.body.subarray(0, HEAD_BYTES)) !== "archive") return undefined; | |
| 1103 | return blob.size > RENDER_CAP.archive ? overSize(RENDER_CAP.archive) : undefined; | |
| 1104 | } | |
| 1105 | ||
| 1098 | 1106 | /** Whether the blob page of this file draws a PDF, and so needs `object-src`. */ |
| 1099 | 1107 | export function drawsPdf(scope: RepoScope, path: string, blob: Blob, view: BlobView): boolean { |
| 1100 | 1108 | return ( |
| @@ -1145,7 +1153,10 @@ export function renderBlob( | ||
| 1145 | 1153 | const { content, lines, summary } = |
| 1146 | 1154 | drawn !== undefined && fellBack === undefined |
| 1147 | 1155 | ? drawn |
| 1148 | : { ...blobBody(base, o.ref, o.path, blob, o.highlighted, o.all, o.rows), summary: undefined }; | |
| 1156 | : { | |
| 1157 | ...blobBody(base, o.ref, o.path, blob, o.highlighted, o.all, o.rows, fellBack ?? overCap(o.path, blob)), | |
| 1158 | summary: undefined, | |
| 1159 | }; | |
| 1149 | 1160 | const sizeText = |
| 1150 | 1161 | blob.kind === "too_large" ? "" : sizeWithTokens(blob.size, o.path); |
| 1151 | 1162 | const fileHref = `${base}/blob/${encoded}/${o.path}`; |
src/web/render/archive-view.ts added +50
| @@ -0,0 +1,50 @@ | ||
| 1 | /** | |
| 2 | * A .zip, .tar or .tar.gz as a table of what is inside: path, size and | |
| 3 | * modified date, drawn on the server from `listArchive`. Nothing is unpacked. | |
| 4 | */ | |
| 5 | ||
| 6 | import { escapeHtml, html, raw } from "../html.ts"; | |
| 7 | import { fileSize } from "../ui.ts"; | |
| 8 | import { count, ENTRY_CAP, listArchive, type Entry } from "./archive.ts"; | |
| 9 | import type { Rendered, RenderInput } from "./renderers.ts"; | |
| 10 | ||
| 11 | /** `2026-11-09 12:30`, UTC, as the archive stores it. */ | |
| 12 | const when = (d: Date | null): string => (d === null ? "" : d.toISOString().slice(0, 16).replace("T", " ")); | |
| 13 | ||
| 14 | /** Control and bidi-override characters would hide or reorder a name. */ | |
| 15 | const printable = (s: string): string => | |
| 16 | s.replace(/[\u0000-\u001f\u007f-\u009f--]/g, "�"); | |
| 17 | ||
| 18 | /** The folder part muted, the name plain; both escaped. */ | |
| 19 | function name(e: Entry): string { | |
| 20 | const path = printable(e.path); | |
| 21 | const trimmed = e.dir ? path.slice(0, -1) : path; | |
| 22 | const cut = trimmed.lastIndexOf("/") + 1; | |
| 23 | const tail = escapeHtml(trimmed.slice(cut)) + (e.dir ? "/" : ""); | |
| 24 | return `<span class="muted">${escapeHtml(trimmed.slice(0, cut))}</span>${tail}`; | |
| 25 | } | |
| 26 | ||
| 27 | const FORMAT = { zip: "ZIP", tar: "tar", "tar.gz": "tar.gz" } as const; | |
| 28 | ||
| 29 | export function archive(input: RenderInput): Rendered { | |
| 30 | const listed = listArchive(input.body); | |
| 31 | if ("error" in listed) return { content: html``, lines: 0, fallback: listed.error }; | |
| 32 | let rows = ""; | |
| 33 | for (const e of listed.entries) | |
| 34 | rows += `<tr><td>${name(e)}</td><td class="num">${e.dir ? "" : fileSize(e.size)}</td><td class="num">${when(e.mtime)}</td></tr>`; | |
| 35 | const left = listed.count - listed.entries.length; | |
| 36 | const more = | |
| 37 | left > 0 | |
| 38 | ? ` ${count(left)} more ${left === 1 ? "entry is" : "entries are"} not listed (the page shows the first ${count(ENTRY_CAP)}).` | |
| 39 | : ""; | |
| 40 | const label = input.path.slice(input.path.lastIndexOf("/") + 1); | |
| 41 | return { | |
| 42 | lines: 0, | |
| 43 | summary: `${fileSize(input.size)} · ${FORMAT[listed.format]}`, | |
Principles 6 and 7: the table is wrapped in a |
||
| 44 | content: html`<table class="table archive-view" data-dense aria-label="${label} contents"> | |
| 45 | <thead><tr><th>Path</th><th class="num">Size</th><th class="num">Modified</th></tr></thead> | |
| 46 | <tbody>${raw(rows)}</tbody> | |
| 47 | <tfoot><tr><td colspan="3" class="muted small">${count(listed.count)} ${listed.count === 1 ? "entry" : "entries"} · ${fileSize(listed.bytes)} unpacked.${more}</td></tr></tfoot> | |
| 48 | </table>`, | |
| 49 | }; | |
| 50 | } |
src/web/render/archive.ts added +281
| @@ -0,0 +1,281 @@ | ||
| 1 | /** | |
| 2 | * The entry list of a .zip, .tar or .tar.gz, and nothing else: names, sizes | |
| 3 | * and modified times read from the bytes of the archive itself. No entry is | |
| 4 | * unpacked, written or run; a gzip is inflated in memory under a hard cap, | |
| 5 | * only so its tar headers can be read. | |
| 6 | * | |
| 7 | * Formats: ZIP appnote 6.3.10 (end of central directory §4.3.16, central | |
| 8 | * directory header §4.3.12, zip64 §4.5.3); tar POSIX.1-2001 ustar, with the | |
| 9 | * pax `x` (§ extended header) and GNU `L` long-name records. | |
| 10 | */ | |
| 11 | ||
| 12 | import { gunzipSync } from "node:zlib"; | |
| 13 | ||
| 14 | /** Entries kept for display, whatever the archive holds. */ | |
| 15 | export const ENTRY_CAP = 2_000; | |
| 16 | /** The most a gzip may inflate to before the list is given up. */ | |
| 17 | export const GUNZIP_CAP = 128 * 1024 * 1024; | |
| 18 | ||
| 19 | export interface Entry { | |
| 20 | readonly path: string; | |
| 21 | /** Unpacked bytes. */ | |
| 22 | readonly size: number; | |
| 23 | /** Modified time, or null when the archive gives none a person could read. */ | |
| 24 | readonly mtime: Date | null; | |
| 25 | readonly dir: boolean; | |
| 26 | } | |
| 27 | ||
| 28 | export interface Listing { | |
| 29 | readonly format: "zip" | "tar" | "tar.gz"; | |
| 30 | /** The first `ENTRY_CAP` entries, sorted by path. */ | |
| 31 | readonly entries: readonly Entry[]; | |
| 32 | /** Every entry the archive holds. */ | |
| 33 | readonly count: number; | |
| 34 | /** Unpacked bytes of every entry, shown or not. */ | |
| 35 | readonly bytes: number; | |
| 36 | } | |
| 37 | ||
| 38 | export interface Unreadable { | |
| 39 | readonly error: string; | |
| 40 | } | |
| 41 | ||
| 42 | class Corrupt extends Error {} | |
| 43 | const corrupt = (): never => { | |
| 44 | throw new Corrupt(); | |
| 45 | }; | |
| 46 | ||
| 47 | const UTF8 = new TextDecoder("utf-8"); | |
| 48 | ||
| 49 | // ---- zip ----------------------------------------------------------------- | |
| 50 | ||
| 51 | const EOCD = 0x06054b50; | |
| 52 | const EOCD64 = 0x06064b50; | |
| 53 | const LOCATOR64 = 0x07064b50; | |
| 54 | const CENTRAL = 0x02014b50; | |
| 55 | ||
| 56 | /** MS-DOS date and time, read as UTC; null when the fields are not a date. */ | |
| 57 | function dosTime(date: number, time: number): Date | null { | |
| 58 | const year = ((date >> 9) & 127) + 1980; | |
| 59 | const month = (date >> 5) & 15; | |
| 60 | const day = date & 31; | |
| 61 | const hour = time >> 11; | |
| 62 | const minute = (time >> 5) & 63; | |
| 63 | const second = (time & 31) * 2; | |
| 64 | if (month < 1 || month > 12 || day < 1 || day > 31 || hour > 23 || minute > 59 || second > 59) | |
| 65 | return null; | |
| 66 | const d = new Date(Date.UTC(year, month - 1, day, hour, minute, second)); | |
| 67 | return d.getUTCDate() === day ? d : null; | |
| 68 | } | |
| 69 | ||
| 70 | function readZip(bytes: Uint8Array, keep: number): Omit<Listing, "format"> { | |
| 71 | const v = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); | |
| 72 | let eocd = -1; | |
| 73 | for (let i = bytes.length - 22; i >= Math.max(0, bytes.length - 22 - 0xffff); i--) { | |
| 74 | if (v.getUint32(i, true) === EOCD) { | |
| 75 | eocd = i; | |
| 76 | break; | |
| 77 | } | |
| 78 | } | |
| 79 | if (eocd < 0) corrupt(); | |
| 80 | let total = v.getUint16(eocd + 10, true); | |
| 81 | let offset = v.getUint32(eocd + 16, true); | |
| 82 | if (total === 0xffff || offset === 0xffffffff) { | |
| 83 | const loc = eocd - 20; | |
| 84 | if (loc < 0 || v.getUint32(loc, true) !== LOCATOR64) corrupt(); | |
| 85 | const at = Number(v.getBigUint64(loc + 8, true)); | |
| 86 | if (at < 0 || at + 56 > bytes.length || v.getUint32(at, true) !== EOCD64) corrupt(); | |
| 87 | total = Number(v.getBigUint64(at + 32, true)); | |
| 88 | offset = Number(v.getBigUint64(at + 48, true)); | |
| 89 | } | |
| 90 | if (offset > eocd) corrupt(); | |
| 91 | const kept: Entry[] = []; | |
| 92 | let count = 0; | |
| 93 | let sum = 0; | |
| 94 | let p = offset; | |
| 95 | for (let n = 0; n < total; n++) { | |
| 96 | if (p + 46 > bytes.length || v.getUint32(p, true) !== CENTRAL) corrupt(); | |
| 97 | const flags = v.getUint16(p + 8, true); | |
| 98 | if (flags & 1) throw new Corrupt("encrypted"); | |
| 99 | let size = v.getUint32(p + 24, true); | |
| 100 | const nameLen = v.getUint16(p + 28, true); | |
| 101 | const extraLen = v.getUint16(p + 30, true); | |
| 102 | const commentLen = v.getUint16(p + 32, true); | |
| 103 | const end = p + 46 + nameLen + extraLen + commentLen; | |
| 104 | if (end > bytes.length) corrupt(); | |
| 105 | if (size === 0xffffffff) { | |
| 106 | // zip64 extra field (id 1): the unpacked size comes first. | |
| 107 | let e = p + 46 + nameLen; | |
| 108 | const stop = e + extraLen; | |
| 109 | while (e + 4 <= stop) { | |
| 110 | const id = v.getUint16(e, true); | |
| 111 | const len = v.getUint16(e + 2, true); | |
| 112 | if (id === 1 && len >= 8 && e + 12 <= stop) { | |
| 113 | size = Number(v.getBigUint64(e + 4, true)); | |
| 114 | break; | |
| 115 | } | |
| 116 | e += 4 + len; | |
| 117 | } | |
| 118 | if (size === 0xffffffff) corrupt(); | |
| 119 | } | |
| 120 | count++; | |
| 121 | sum += size; | |
| 122 | if (kept.length < keep) { | |
| 123 | const path = UTF8.decode(bytes.subarray(p + 46, p + 46 + nameLen)); | |
| 124 | kept.push({ | |
| 125 | path, | |
| 126 | size, | |
| 127 | mtime: dosTime(v.getUint16(p + 14, true), v.getUint16(p + 12, true)), | |
| 128 | dir: path.endsWith("/"), | |
| 129 | }); | |
| 130 | } | |
| 131 | p = end; | |
| 132 | } | |
| 133 | return { entries: kept, count, bytes: sum }; | |
| 134 | } | |
| 135 | ||
| 136 | // ---- tar ----------------------------------------------------------------- | |
| 137 | ||
| 138 | const BLOCK = 512; | |
| 139 | ||
| 140 | const cstr = (b: Uint8Array, from: number, len: number): string => { | |
| 141 | let end = from; | |
| 142 | while (end < from + len && b[end] !== 0) end++; | |
| 143 | return UTF8.decode(b.subarray(from, end)); | |
| 144 | }; | |
| 145 | ||
| 146 | /** An octal field, or GNU base-256 when its top bit is set. */ | |
| 147 | function number(b: Uint8Array, from: number, len: number): number { | |
| 148 | if (b[from]! & 0x80) { | |
| 149 | let n = b[from]! & 0x7f; | |
| 150 | for (let i = 1; i < len; i++) n = n * 256 + b[from + i]!; | |
| 151 | return n; | |
| 152 | } | |
| 153 | const text = cstr(b, from, len).trim(); | |
| 154 | if (text === "") return 0; | |
| 155 | if (!/^[0-7]+$/.test(text)) return corrupt(); | |
| 156 | return parseInt(text, 8); | |
| 157 | } | |
| 158 | ||
| 159 | /** The header checksum: every byte summed, the checksum field as spaces. */ | |
| 160 | function checksumOk(b: Uint8Array, at: number): boolean { | |
| 161 | let sum = 0; | |
| 162 | for (let i = 0; i < BLOCK; i++) sum += i >= 148 && i < 156 ? 32 : b[at + i]!; | |
| 163 | return sum === number(b, at + 148, 8); | |
| 164 | } | |
| 165 | ||
| 166 | /** pax records: `len key=value\n`, the length counting itself. */ | |
| 167 | function pax(data: Uint8Array): Map<string, string> { | |
| 168 | const out = new Map<string, string>(); | |
| 169 | let p = 0; | |
| 170 | while (p < data.length) { | |
| 171 | let q = p; | |
| 172 | while (q < data.length && data[q] !== 0x20) q++; | |
| 173 | const len = parseInt(UTF8.decode(data.subarray(p, q)), 10); | |
| 174 | if (!(len > q - p + 1) || p + len > data.length) corrupt(); | |
| 175 | const rec = UTF8.decode(data.subarray(q + 1, p + len - 1)); | |
| 176 | const eq = rec.indexOf("="); | |
| 177 | if (eq > 0) out.set(rec.slice(0, eq), rec.slice(eq + 1)); | |
| 178 | p += len; | |
| 179 | } | |
| 180 | return out; | |
| 181 | } | |
| 182 | ||
| 183 | function readTar(b: Uint8Array, keep: number): Omit<Listing, "format"> { | |
| 184 | const kept: Entry[] = []; | |
| 185 | let count = 0; | |
| 186 | let sum = 0; | |
| 187 | let p = 0; | |
| 188 | let longName: string | undefined; | |
| 189 | let override = new Map<string, string>(); | |
| 190 | while (p + BLOCK <= b.length) { | |
| 191 | if (b.subarray(p, p + BLOCK).every((x) => x === 0)) break; | |
| 192 | if (!checksumOk(b, p)) corrupt(); | |
| 193 | const type = String.fromCharCode(b[p + 156]!); | |
| 194 | let size = number(b, p + 124, 12); | |
| 195 | const data = p + BLOCK; | |
| 196 | if (data + size > b.length) corrupt(); | |
| 197 | const next = data + Math.ceil(size / BLOCK) * BLOCK; | |
| 198 | if (type === "x") { | |
| 199 | override = pax(b.subarray(data, data + size)); | |
| 200 | } else if (type === "L") { | |
| 201 | longName = cstr(b, data, size); | |
| 202 | } else if (type !== "g") { | |
| 203 | const ustar = cstr(b, p + 257, 5) === "ustar"; | |
| 204 | const prefix = ustar ? cstr(b, p + 345, 155) : ""; | |
| 205 | const base = cstr(b, p, 100); | |
| 206 | let path = longName ?? override.get("path") ?? (prefix ? `${prefix}/${base}` : base); | |
| 207 | const sizeOverride = override.get("size"); | |
| 208 | if (sizeOverride !== undefined && /^\d+$/.test(sizeOverride)) size = Number(sizeOverride); | |
| 209 | const mt = override.get("mtime"); | |
| 210 | const secs = mt !== undefined && /^\d+(\.\d+)?$/.test(mt) ? Number(mt) : number(b, p + 136, 12); | |
| 211 | const dir = type === "5" || path.endsWith("/"); | |
| 212 | if (dir && !path.endsWith("/")) path += "/"; | |
| 213 | const shown = type === "0" || type === "\0" || type === "7" ? size : 0; | |
| 214 | count++; | |
| 215 | sum += shown; | |
| 216 | if (kept.length < keep) { | |
| 217 | const t = new Date(secs * 1000); | |
| 218 | kept.push({ path, size: shown, mtime: secs > 0 && !Number.isNaN(t.getTime()) ? t : null, dir }); | |
| 219 | } | |
| 220 | longName = undefined; | |
| 221 | override = new Map(); | |
| 222 | } | |
| 223 | p = next; | |
| 224 | } | |
| 225 | if (count === 0) corrupt(); | |
| 226 | return { entries: kept, count, bytes: sum }; | |
| 227 | } | |
| 228 | ||
| 229 | // ---- entry point ----------------------------------------------------------- | |
| 230 | ||
| 231 | const isZip = (b: Uint8Array): boolean => b[0] === 0x50 && b[1] === 0x4b && b[2] === 3 && b[3] === 4; | |
| 232 | const isGzip = (b: Uint8Array): boolean => b[0] === 0x1f && b[1] === 0x8b; | |
| 233 | ||
| 234 | const byPath = (a: Entry, b: Entry): number => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0); | |
| 235 | ||
| 236 | /** A count with thousands grouped by commas, whatever the server locale. */ | |
| 237 | export const count = (n: number): string => String(n).replace(/\B(?=(\d{3})+$)/g, ","); | |
| 238 | ||
| 239 | const mb = (n: number): string => `${count(Math.round(n / 1_000_000))} MB`; | |
| 240 | ||
| 241 | /** The sentence for an archive over the size the page reads. */ | |
| 242 | export const overSize = (cap: number): string => | |
| 243 | `This archive is over ${count(Math.round(cap / 1_000_000))} MB, too large to list here. Download it instead.`; | |
| 244 | ||
| 245 | /** The list of what an archive holds, or one sentence on why it cannot be. */ | |
| 246 | export function listArchive( | |
| 247 | body: Uint8Array, | |
| 248 | opts: { readonly cap?: number; readonly entries?: number } = {}, | |
| 249 | ): Listing | Unreadable { | |
| 250 | const cap = opts.cap ?? GUNZIP_CAP; | |
| 251 | const keep = opts.entries ?? ENTRY_CAP; | |
| 252 | try { | |
| 253 | let format: Listing["format"]; | |
| 254 | let part: Omit<Listing, "format">; | |
| 255 | if (isZip(body)) { | |
| 256 | format = "zip"; | |
| 257 | part = readZip(body, keep); | |
outdated
The |
||
| 258 | } else if (isGzip(body)) { | |
| 259 | format = "tar.gz"; | |
| 260 | let tar: Uint8Array; | |
| 261 | try { | |
| 262 | tar = gunzipSync(body, { maxOutputLength: cap }); | |
| 263 | } catch (e) { | |
| 264 | if ((e as { code?: string }).code === "ERR_BUFFER_TOO_LARGE") | |
| 265 | return { error: `This archive unpacks to more than ${mb(cap)}, so its list is not read here.` }; | |
| 266 | return { error: "This archive is damaged, so its list cannot be read." }; | |
| 267 | } | |
| 268 | part = readTar(tar, keep); | |
| 269 | } else { | |
| 270 | format = "tar"; | |
| 271 | part = readTar(body, keep); | |
| 272 | } | |
| 273 | return { format, ...part, entries: [...part.entries].sort(byPath) }; | |
| 274 | } catch (e) { | |
| 275 | if (e instanceof Corrupt && e.message === "encrypted") | |
| 276 | return { error: "This zip is encrypted, so its list is not shown." }; | |
| 277 | if (e instanceof Corrupt || e instanceof RangeError) | |
| 278 | return { error: "This archive is damaged or cut short, so its list cannot be read." }; | |
| 279 | throw e; | |
| 280 | } | |
| 281 | } |
src/web/render/kinds.ts +24
| @@ -20,6 +20,7 @@ export type FileKind = | ||
| 20 | 20 | | "video" |
| 21 | 21 | | "csv" |
| 22 | 22 | | "model3d" |
| 23 | | "archive" | |
| 23 | 24 | | "config" |
| 24 | 25 | | "markdown" |
| 25 | 26 | | "text" |
| @@ -40,6 +41,7 @@ export const RENDER_CAP: Readonly<Record<FileKind, number>> = { | ||
| 40 | 41 | video: 4_000 * MB, |
| 41 | 42 | csv: 10 * MB, |
| 42 | 43 | model3d: 100 * MB, |
| 44 | archive: 50 * MB, | |
| 43 | 45 | config: 1 * MB, |
| 44 | 46 | markdown: 1 * MB, |
| 45 | 47 | text: 1 * MB, |
| @@ -134,6 +136,24 @@ const TEXTUAL: Readonly<Record<string, FileKind>> = { | ||
| 134 | 136 | mdx: "markdown", |
| 135 | 137 | }; |
| 136 | 138 | |
| 139 | /** | |
| 140 | * Archives: the extension names the kind and the bytes confirm it. Their | |
| 141 | * media type is never an archive type; the media route sends octet-stream. | |
| 142 | * `gz` counts only as the end of `.tar.gz`. | |
| 143 | */ | |
| 144 | const ARCHIVES: Readonly<Record<string, Magic>> = { | |
| 145 | zip: bytes(0x50, 0x4b, 0x03, 0x04), | |
| 146 | tar: ascii("ustar", 257), | |
| 147 | tgz: bytes(0x1f, 0x8b), | |
| 148 | gz: bytes(0x1f, 0x8b), | |
| 149 | }; | |
| 150 | ||
| 151 | const isArchive = (path: string, head: Uint8Array): boolean => { | |
| 152 | const ext = extension(path); | |
| 153 | if (ext === "gz" && !/\.tar\.gz$/i.test(path)) return false; | |
| 154 | return ARCHIVES[ext]?.(head) === true; | |
| 155 | }; | |
| 156 | ||
| 137 | 157 | /** For a caller that has only a name: the kind its extension gives. */ |
| 138 | 158 | export const NO_BYTES: Uint8Array = new Uint8Array(0); |
| 139 | 159 | |
| @@ -165,6 +185,7 @@ function confirmed(path: string, head: Uint8Array): Media | undefined { | ||
| 165 | 185 | export function fileKind(path: string, head: Uint8Array): FileKind { |
| 166 | 186 | const media = confirmed(path, head); |
| 167 | 187 | if (media !== undefined) return media.kind; |
| 188 | if (isArchive(path, head)) return "archive"; | |
| 168 | 189 | if (looksBinary(head)) return "binary"; |
| 169 | 190 | return TEXTUAL[extension(path)] ?? "text"; |
| 170 | 191 | } |
| @@ -193,4 +214,7 @@ export function mediaType(path: string, head: Uint8Array): string { | ||
| 193 | 214 | export const KNOWN_EXTENSIONS: readonly string[] = [ |
| 194 | 215 | ...Object.keys(MEDIA), |
| 195 | 216 | ...Object.keys(TEXTUAL), |
| 217 | "zip", | |
| 218 | "tar", | |
| 219 | "tgz", | |
| 196 | 220 | ]; |
src/web/render/renderers.ts +2
| @@ -6,6 +6,7 @@ | ||
| 6 | 6 | |
| 7 | 7 | import type { Html } from "../html.ts"; |
| 8 | 8 | import type { Blob } from "../../view/source.ts"; |
| 9 | import { archive } from "./archive-view.ts"; | |
| 9 | 10 | import { audio } from "./audio.ts"; |
| 10 | 11 | import { config } from "./config.ts"; |
| 11 | 12 | import { isFormConfig } from "./config-format.ts"; |
| @@ -54,6 +55,7 @@ export const RENDERERS: Partial<Readonly<Record<FileKind, Renderer>>> = { | ||
| 54 | 55 | audio, |
| 55 | 56 | video, |
| 56 | 57 | config, |
| 58 | archive, | |
| 57 | 59 | }; |
| 58 | 60 | |
| 59 | 61 | /** |
test/render-archive.test.ts added +205
| @@ -0,0 +1,205 @@ | ||
| 1 | /** Archives listed, not unpacked: zip, tar, tar.gz, and the limits on each. */ | |
Add a test for the TUI path once it exists. Also add one for the over-50 MB fallback sentence, which is only checked indirectly through |
||
| 2 | import { describe, expect, test } from "bun:test"; | |
| 3 | import { gzipSync } from "node:zlib"; | |
| 4 | import { ENTRY_CAP, listArchive } from "../src/web/render/archive.ts"; | |
| 5 | import { archive } from "../src/web/render/archive-view.ts"; | |
| 6 | import { fileKind } from "../src/web/render/kinds.ts"; | |
| 7 | import { rendererFor, RENDERERS } from "../src/web/render/renderers.ts"; | |
| 8 | ||
| 9 | const enc = (s: string): Uint8Array => new TextEncoder().encode(s); | |
| 10 | const cat = (...parts: Uint8Array[]): Uint8Array => { | |
| 11 | const out = new Uint8Array(parts.reduce((n, p) => n + p.length, 0)); | |
| 12 | let o = 0; | |
| 13 | for (const p of parts) (out.set(p, o), (o += p.length)); | |
| 14 | return out; | |
| 15 | }; | |
| 16 | ||
| 17 | interface F { name: string; size?: number; dir?: boolean } | |
| 18 | ||
| 19 | /** A zip with only a central directory, as the lister reads it. */ | |
| 20 | function zip(files: F[], flags = 0): Uint8Array { | |
| 21 | const dosDate = ((2026 - 1980) << 9) | (11 << 5) | 9; | |
| 22 | const dosTime = (12 << 11) | (30 << 5); | |
| 23 | const central: Uint8Array[] = []; | |
| 24 | for (const f of files) { | |
| 25 | const name = enc(f.name); | |
| 26 | const h = new Uint8Array(46); | |
| 27 | const v = new DataView(h.buffer); | |
| 28 | v.setUint32(0, 0x02014b50, true); | |
| 29 | v.setUint16(8, flags, true); | |
| 30 | v.setUint16(12, dosTime, true); | |
| 31 | v.setUint16(14, dosDate, true); | |
| 32 | v.setUint32(24, f.size ?? 0, true); | |
| 33 | v.setUint16(28, name.length, true); | |
| 34 | central.push(cat(h, name)); | |
| 35 | } | |
| 36 | const dir = cat(...central); | |
| 37 | const local = enc("PK\x03\x04"); | |
| 38 | const e = new Uint8Array(22); | |
| 39 | const v = new DataView(e.buffer); | |
| 40 | v.setUint32(0, 0x06054b50, true); | |
| 41 | v.setUint16(8, files.length, true); | |
| 42 | v.setUint16(10, files.length, true); | |
| 43 | v.setUint32(12, dir.length, true); | |
| 44 | v.setUint32(16, local.length, true); | |
| 45 | return cat(local, dir, e); | |
| 46 | } | |
| 47 | ||
| 48 | function tarHeader(name: string, size: number, type = "0"): Uint8Array { | |
| 49 | const h = new Uint8Array(512); | |
| 50 | h.set(enc(name), 0); | |
| 51 | h.set(enc(size.toString(8).padStart(11, "0")), 124); | |
| 52 | h.set(enc(Math.floor(Date.UTC(2026, 10, 9, 12, 30) / 1000).toString(8).padStart(11, "0")), 136); | |
| 53 | h.set(enc(" "), 148); | |
| 54 | h[156] = type.charCodeAt(0); | |
| 55 | h.set(enc("ustar\x0000"), 257); | |
| 56 | let sum = 0; | |
| 57 | for (const b of h) sum += b; | |
| 58 | h.set(enc(sum.toString(8).padStart(6, "0") + "\0 "), 148); | |
| 59 | return h; | |
| 60 | } | |
| 61 | function tar(files: F[]): Uint8Array { | |
| 62 | const parts: Uint8Array[] = []; | |
| 63 | for (const f of files) { | |
| 64 | const size = f.dir ? 0 : (f.size ?? 0); | |
| 65 | parts.push(tarHeader(f.name, size, f.dir ? "5" : "0")); | |
| 66 | parts.push(new Uint8Array(Math.ceil(size / 512) * 512)); | |
| 67 | } | |
| 68 | parts.push(new Uint8Array(1024)); | |
| 69 | return cat(...parts); | |
| 70 | } | |
| 71 | ||
| 72 | const ok = (r: ReturnType<typeof listArchive>) => { | |
| 73 | if ("error" in r) throw new Error(r.error); | |
| 74 | return r; | |
| 75 | }; | |
| 76 | ||
| 77 | describe("archive lists", () => { | |
| 78 | test("a zip: sorted by path, sizes summed, DOS date read", () => { | |
| 79 | const r = ok(listArchive(zip([{ name: "b.txt", size: 20 }, { name: "a/", dir: true }, { name: "a/c.txt", size: 5 }]))); | |
| 80 | expect(r.format).toBe("zip"); | |
| 81 | expect(r.entries.map((e) => e.path)).toEqual(["a/", "a/c.txt", "b.txt"]); | |
| 82 | expect(r.entries[0]!.dir).toBe(true); | |
| 83 | expect(r.count).toBe(3); | |
| 84 | expect(r.bytes).toBe(25); | |
| 85 | expect(r.entries[2]!.mtime?.toISOString()).toBe("2026-11-09T12:30:00.000Z"); | |
| 86 | }); | |
| 87 | test("a tar: headers read, directories have no size", () => { | |
| 88 | const r = ok(listArchive(tar([{ name: "src", dir: true }, { name: "src/x.ts", size: 700 }]))); | |
| 89 | expect(r.format).toBe("tar"); | |
| 90 | expect(r.entries.map((e) => [e.path, e.size, e.dir])).toEqual([["src/", 0, true], ["src/x.ts", 700, false]]); | |
| 91 | expect(r.bytes).toBe(700); | |
| 92 | expect(r.entries[1]!.mtime?.toISOString()).toBe("2026-11-09T12:30:00.000Z"); | |
| 93 | }); | |
| 94 | test("a tar.gz: inflated in memory, then read as tar", () => { | |
| 95 | const r = ok(listArchive(gzipSync(tar([{ name: "a.txt", size: 3 }])))); | |
| 96 | expect(r.format).toBe("tar.gz"); | |
| 97 | expect(r.entries.map((e) => e.path)).toEqual(["a.txt"]); | |
| 98 | }); | |
| 99 | }); | |
| 100 | ||
| 101 | describe("archive limits", () => { | |
| 102 | test("a truncated zip says why instead of failing", () => { | |
| 103 | const z = zip([{ name: "a", size: 1 }]); | |
| 104 | const r = listArchive(z.subarray(0, z.length - 10)); | |
| 105 | expect("error" in r && r.error).toMatch(/damaged or cut short/); | |
| 106 | }); | |
| 107 | test("an encrypted zip is not listed", () => { | |
| 108 | const r = listArchive(zip([{ name: "a" }], 1)); | |
| 109 | expect("error" in r && r.error).toMatch(/encrypted/); | |
| 110 | }); | |
| 111 | test("a corrupt tar header fails its checksum", () => { | |
| 112 | const t = tar([{ name: "a", size: 1 }]); | |
| 113 | t[0] = 0x7a; | |
| 114 | expect("error" in listArchive(t)).toBe(true); | |
| 115 | }); | |
| 116 | test("a gzip bomb stops at the cap", () => { | |
| 117 | const bomb = gzipSync(new Uint8Array(8 * 1024 * 1024)); | |
| 118 | expect(bomb.length).toBeLessThan(20_000); | |
| 119 | const r = listArchive(bomb, { cap: 1024 * 1024 }); | |
| 120 | expect("error" in r && r.error).toMatch(/unpacks to more than/); | |
| 121 | }); | |
| 122 | test("more than 2,000 entries: the first 2,000, and the rest counted", () => { | |
| 123 | const files = Array.from({ length: ENTRY_CAP + 25 }, (_, i) => ({ name: `f${i}`, size: 1 })); | |
| 124 | const r = ok(listArchive(zip(files))); | |
| 125 | expect(r.entries.length).toBe(ENTRY_CAP); | |
| 126 | expect(r.count).toBe(ENTRY_CAP + 25); | |
| 127 | expect(r.bytes).toBe(ENTRY_CAP + 25); | |
| 128 | const page = archive({ path: "x.zip", size: 1, body: zip(files), media: "/m" }); | |
| 129 | expect(String(page.content)).toContain("25 more entries are not listed"); | |
| 130 | }); | |
| 131 | }); | |
| 132 | ||
| 133 | describe("archive page", () => { | |
| 134 | const draw = (path: string, body: Uint8Array) => archive({ path, size: body.length, body, media: "/m" }); | |
| 135 | test("a name that is markup is escaped", () => { | |
| 136 | const out = String(draw("x.zip", zip([{ name: "<script>alert(1)</script>", size: 1 }])).content); | |
| 137 | expect(out).not.toContain("<script>"); | |
| 138 | expect(out).toContain("<script>"); | |
| 139 | }); | |
| 140 | test("a footer gives the count and unpacked size", () => { | |
| 141 | const out = String(draw("x.tar", tar([{ name: "a", size: 1000 }, { name: "b", size: 1000 }])).content); | |
| 142 | expect(out).toContain("2 entries"); | |
| 143 | expect(out).toContain("unpacked"); | |
| 144 | }); | |
| 145 | test("a damaged archive falls back with a reason, never throws", () => { | |
| 146 | const r = draw("x.zip", enc("PK\x03\x04garbage")); | |
| 147 | expect(r.fallback).toMatch(/damaged/); | |
| 148 | }); | |
| 149 | test("the extension names the kind and the bytes must agree", () => { | |
| 150 | const z = zip([{ name: "a" }]); | |
| 151 | expect(fileKind("a.zip", z)).toBe("archive"); | |
| 152 | expect(fileKind("a.zip", enc("not a zip"))).toBe("text"); | |
| 153 | expect(fileKind("a.tar.gz", gzipSync(enc("x")))).toBe("archive"); | |
| 154 | expect(fileKind("a.tgz", gzipSync(enc("x")))).toBe("archive"); | |
| 155 | expect(fileKind("a.gz", gzipSync(enc("x")))).toBe("binary"); | |
| 156 | expect(fileKind("a.tar", tar([{ name: "a" }]))).toBe("archive"); | |
| 157 | }); | |
| 158 | test("a blob page reaches it, and not past the inline limit", () => { | |
| 159 | const z = zip([{ name: "a" }]); | |
| 160 | const hit = rendererFor("a.zip", { kind: "binary", body: z, size: z.length }); | |
| 161 | expect(hit?.render).toBe(RENDERERS.archive!); | |
| 162 | expect(rendererFor("a.zip", { kind: "binary", body: z, size: 60_000_000 })).toBeUndefined(); | |
| 163 | }); | |
| 164 | }); | |
| 165 | ||
| 166 | describe("wording and the terminal", () => { | |
| 167 | test("counts do not depend on the locale", async () => { | |
| 168 | const { count } = await import("../src/web/render/archive.ts"); | |
| 169 | expect(count(0)).toBe("0"); | |
| 170 | expect(count(2000)).toBe("2,000"); | |
| 171 | expect(count(1234567)).toBe("1,234,567"); | |
| 172 | }); | |
| 173 | ||
| 174 | test("the over-50 MB sentence", async () => { | |
| 175 | const { overSize } = await import("../src/web/render/archive.ts"); | |
| 176 | expect(overSize(50_000_000)).toBe("This archive is over 50 MB, too large to list here. Download it instead."); | |
| 177 | }); | |
| 178 | ||
| 179 | test("a name with control and bidi characters is shown replaced", () => { | |
| 180 | const z = zip([{ name: "ab\u0007c.txt", size: 1 }]); | |
| 181 | const out = archive({ path: "x.zip", body: z, size: z.length } as never).content.toString(); | |
| 182 | expect(out).not.toContain(""); | |
| 183 | expect(out).not.toContain("\u0007"); | |
| 184 | expect(out).toContain("a�b�c.txt"); | |
| 185 | }); | |
| 186 | ||
| 187 | test("the table is not inside a card", () => { | |
| 188 | const z = zip([{ name: "a.txt", size: 1 }]); | |
| 189 | const out = archive({ path: "x.zip", body: z, size: z.length } as never).content.toString(); | |
| 190 | expect(out).not.toContain("card"); | |
| 191 | expect(out).toContain('class="table archive-view"'); | |
| 192 | }); | |
| 193 | ||
| 194 | test("the terminal note gives count and unpacked size, gated like the page", async () => { | |
| 195 | const { archiveNote } = await import("../src/tui/ui/archive-note.ts"); | |
| 196 | const fmt = (n: number): string => `${n} B`; | |
| 197 | const z = zip([{ name: "a.txt", size: 10 }, { name: "b/c.txt", size: 5 }]); | |
| 198 | expect(archiveNote("x.zip", z, z.length, fmt)).toBe("zip, 2 entries, 15 B unpacked."); | |
| 199 | expect(archiveNote("x.txt", z, z.length, fmt)).toBeUndefined(); | |
| 200 | expect(archiveNote("x.zip", enc("not a zip at all"), 16, fmt)).toBeUndefined(); | |
| 201 | expect(archiveNote("x.zip", z, 60_000_000, fmt)).toContain("over 50 MB"); | |
| 202 | const bad = z.slice(0, z.length - 10); | |
| 203 | expect(archiveNote("x.zip", bad, bad.length, fmt)).toContain("damaged"); | |
| 204 | }); | |
| 205 | }); |
test/render-kinds.test.ts +3
| @@ -47,6 +47,9 @@ const SAMPLES: Record<string, [Uint8Array, FileKind, string]> = { | ||
| 47 | 47 | toml: [enc("a = 1\n"), "config", "application/octet-stream"], |
| 48 | 48 | ini: [enc("[a]\n"), "config", "application/octet-stream"], |
| 49 | 49 | conf: [enc("a 1\n"), "config", "application/octet-stream"], |
| 50 | zip: [of(0x50, 0x4b, 3, 4, 0), "archive", "application/octet-stream"], | |
| 51 | tar: [enc("x".repeat(257) + "ustar\0"), "archive", "application/octet-stream"], | |
| 52 | tgz: [of(0x1f, 0x8b, 8), "archive", "application/octet-stream"], | |
| 50 | 53 | md: [enc("# a\n"), "markdown", "application/octet-stream"], |
| 51 | 54 | markdown: [enc("# a\n"), "markdown", "application/octet-stream"], |
| 52 | 55 | mdx: [enc("# a\n"), "markdown", "application/octet-stream"], |
The reason is spliced in by lower-casing the first letter of a sentence, which is brittle. It also leaves the title as "Binary file" for a valid archive that is only too big or damaged. Pass the sentence in ready-made and let the card title say "Archive" in that case. Also, the em dash joins a size to a sentence that already ends in a full stop, which reads awkwardly.