trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRunsg rEventsFlows

A log file with terminal colours (.log, .ansi) renders in colour, with no escape codes shown #10335

Merged scoutcrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/10330-a-log-file-with-terminal-colours-log-ans → main 7 commits fork 3 threads · 3 unresolved

  • Merged. scout/10330-a-log-file-with-terminal-colours-log-ans was merged into main as b0b1a12.

Review

Sign in to review this change.
11 files changed +317 −11
docs/design/areas/render.md +9
@@ -9,6 +9,15 @@ No script, no pdf.js.
99
1010 The header shows the size (`1.2 MB · PDF`) through `Rendered.summary`. Both themes use tokens only.
1111
12 ## Terminal log (.log, .ansi)
13
14 `src/web/render/log.ts` draws a log that holds escape codes inside the existing `.card.code` table, so it looks like any code file. Colour comes only from fixed classes (`src/web/style/content.ts`), tokens only, both themes:
15
16 - `.ansi-red` `--danger`, `.ansi-green` `--success`, `.ansi-yellow` `--attn`, `.ansi-blue` `--tok-number`, `.ansi-magenta` `--merged`, `.ansi-cyan` `--tok-property`, `.ansi-gray` (black) `--tok-comment`. White is the page's own ink.
17 - `.ansi-b` bold, `.ansi-u` underline.
18
19 Backgrounds, 256-colour above 15 and truecolour are dropped; so are cursor moves and OSC. A log with no escape code keeps the plain source view.
20
1221 ## Archive listing
1322
1423 A .zip, .tar or .tar.gz is drawn by `src/web/render/archive-view.ts` as the existing `.table` alone, with no `.card` around it (path, size, modified; `data-dense`), and a `tfoot` for the count and unpacked size. No new class: `.archive-view` is only a hook on the table and has no rule. How it works: `docs/features/archive-listing.md`. A list that cannot be read falls back to the Download card with the reason as its one sentence.
docs/design/attention-budget.json +8 −8
@@ -13,25 +13,25 @@
1313 },
1414 "repository": {
1515 "390": {
16 "words": 189,
17 "blocks": 28,
16 "words": 197,
17 "blocks": 29,
1818 "colours": 2
1919 },
2020 "1280": {
21 "words": 189,
22 "blocks": 28,
21 "words": 197,
22 "blocks": 29,
2323 "colours": 2
2424 }
2525 },
2626 "app": {
2727 "390": {
28 "words": 189,
29 "blocks": 28,
28 "words": 197,
29 "blocks": 29,
3030 "colours": 2
3131 },
3232 "1280": {
33 "words": 189,
34 "blocks": 28,
33 "words": 197,
34 "blocks": 29,
3535 "colours": 2
3636 }
3737 },
docs/features/ansi-log.md added +7
@@ -0,0 +1,7 @@
1 # Terminal logs in colour
2
3 `.log` and `.ansi` files that contain terminal escapes open on the blob page in colour (`src/web/render/log.ts`, parser in `ansi.ts`). A log with no escape byte keeps the ordinary source view.
4 Only SGR is read: foreground colour (30-37, 90-97, and 256-colour 0-15), bold, underline, and the resets 0, 22, 24, 39. Each becomes a fixed class (`ansi-red`, `ansi-b`, `ansi-u`, …) coloured by design tokens; the input never reaches a `style` attribute or a class name. Background, truecolour and 256-colour above 15 are dropped.
5 Every other escape (cursor moves, OSC titles and links, an unterminated OSC up to its newline) and every other control character, a bare CR included, is removed, never acted on. A colour is closed at each line end and reopened on the next.
6 Drawn as the code view is: first 1,500 lines, `?all=1` ("Show all lines", no JavaScript) up to 10,000, with an "N more lines" note. `?view=source`, Raw and Download show the bytes untouched.
7 The terminal (SSH) file view shows a `.log`/`.ansi` as plain text: its escapes are stripped (`stripAnsi`, `src/tui/term/ansi.ts`) before the lines are built, so a log cannot move the cursor, set a title or print `[31m`. It has no colour there. The class list lives in `docs/design/areas/render.md` and `src/web/style/content.ts`.
src/tui/screens/workspace.ts +10 −2
@@ -54,6 +54,7 @@ import { MAX_COLUMNS, parseCsv, type Table } from "../../text/csv.ts";
5454 import { CUE_CAP, parseSubtitles } from "../../web/render/subtitles.ts";
5555 import { layoutTable, type Grid } from "../ui/grid.ts";
5656 import type { Ctx, Screen } from "../app/app.ts";
57 import { stripAnsi } from "../term/ansi.ts";
5758 import { binding, type Binding } from "../app/keys.ts";
5859 import type { KeyEvent } from "../term/input.ts";
5960 import {
@@ -252,12 +253,19 @@ function landing(nodes: readonly Node[]): Node | undefined {
252253 * spans: the pane draws line numbers itself, so they stay pinned while the
253254 * code scrolls sideways under them.
254255 */
255 function buildLines(
256 function isLogPath(path: string): boolean {
257 return /\.(log|ansi)$/i.test(path);
258 }
259
260 export function buildLines(
256261 text: string,
257262 path: string,
258263 c: Tokens,
259264 ): { plain: string[]; lines: Span[][] } {
260 const body = text.replace(/\n$/, "");
265 // A terminal log carries escapes; the viewer shows its text, never drives
266 // the terminal or prints "[31m" as if it were the log.
267 const clean = isLogPath(path) && text.includes("\x1b") ? stripAnsi(text) : text;
268 const body = clean.replace(/\n$/, "");
261269 const plain = body.split("\n").map((l) => expandTabs(l));
262270 // The first line is handed to the detector too: a `#!/usr/bin/env python3`
263271 // with no extension, a `.zshrc`, a vim modeline. The name is the strong
src/web/render/ansi.ts added +118
@@ -0,0 +1,118 @@
1 /**
2 * Terminal output as escaped HTML lines. Pure: text in, one string per line
3 * out. Only SGR (`ESC [ … m`) is interpreted — foreground colour, bold and
4 * underline — and each becomes a fixed class (`ansi-red`, `ansi-b`,
5 * `ansi-u`); the input never reaches a `style` attribute or a class name.
6 * Every other escape (cursor moves, OSC titles and hyperlinks, charset
7 * selects) and every other control character is removed, never acted on.
8 * Background, 256-colour (above 15) and truecolour requests are dropped.
9 */
10
11 import { escapeHtml } from "../html.ts";
12
13 /** Colour class by SGR number (30-37; 90-97 share them). White is the default ink. */
14 const HUES: readonly (string | undefined)[] = [
15 "gray", // 0 black: invisible on a dark page, so it is the quiet ink
16 "red",
17 "green",
18 "yellow",
19 "blue",
20 "magenta",
21 "cyan",
22 undefined, // 7 white: the page's own ink
23 ];
24
25 interface Pen {
26 hue: string | undefined;
27 bold: boolean;
28 underline: boolean;
29 }
30
31 const classOf = (pen: Pen): string => {
32 const parts: string[] = [];
33 if (pen.hue !== undefined) parts.push(`ansi-${pen.hue}`);
34 if (pen.bold) parts.push("ansi-b");
35 if (pen.underline) parts.push("ansi-u");
36 return parts.join(" ");
37 };
38
39 /** The pen after one SGR parameter list. */
40 function sgr(pen: Pen, params: string): void {
41 const codes = params === "" ? [0] : params.split(/[;:]/).map((p) => (p === "" ? 0 : Number(p)));
42 for (let i = 0; i < codes.length; i++) {
43 const c = codes[i]!;
junipercrew

Item 3: the 90-97 branch, 22, 24 and 39 have no test that fails without them.
44 if (!Number.isInteger(c)) continue;
45 if (c === 0) {
46 pen.hue = undefined;
47 pen.bold = false;
48 pen.underline = false;
49 } else if (c === 1) pen.bold = true;
50 else if (c === 22) pen.bold = false;
51 else if (c === 4) pen.underline = true;
52 else if (c === 24) pen.underline = false;
53 else if ((c >= 30 && c <= 37) || (c >= 90 && c <= 97)) pen.hue = HUES[c % 10];
54 else if (c === 39) pen.hue = undefined;
55 else if (c === 38 || c === 48) {
56 const mode = codes[i + 1];
57 if (c === 38 && mode === 5) {
58 const n = codes[i + 2];
59 if (n !== undefined && Number.isInteger(n) && n >= 0 && n <= 15) pen.hue = HUES[n % 8];
60 }
61 i += mode === 5 ? 2 : mode === 2 ? 4 : 0;
62 }
63 }
64 }
65
66 /**
67 * OSC (to BEL or ST), CSI, any other escape pair, then a lone control
68 * character that is neither a tab nor a newline (a bare CR included).
69 */
70 const TOKEN =
71 // eslint-disable-next-line no-control-regex
junipercrew

Item 2: [^\x07\x1b]* crosses newlines, so an unterminated OSC eats the rest of the log up to the next ESC. Exclude \n and test it.
72 /\x1b\][^\x07\x1b\n]*(?:\x07|\x1b\\)?|\x1b\[([0-?]*)[ -/]*([@-~])?|\x1b[ -/]*[0-~]?|[\x00-\x08\x0b-\x1f\x7f]/g;
73
74 export interface AnsiLines {
75 /** Escaped markup, one entry per line; a colour never spans two entries. */
76 readonly lines: string[];
77 /** Lines the whole text has, whether or not all were drawn. */
78 readonly total: number;
79 }
80
81 /** The first `limit` lines of `input` as markup, and the line count of all of it. */
82 export function ansiLines(input: string, limit: number): AnsiLines {
83 const text = input.replace(/\n$/, "");
84 let total = 1;
85 for (let i = text.indexOf("\n"); i !== -1; i = text.indexOf("\n", i + 1)) total++;
86
87 const pen: Pen = { hue: undefined, bold: false, underline: false };
88 const lines: string[] = [];
89 let line = "";
90 const put = (chunk: string): void => {
91 if (chunk === "") return;
92 const cls = classOf(pen);
93 line += cls === "" ? escapeHtml(chunk) : `<span class="${cls}">${escapeHtml(chunk)}</span>`;
94 };
95 const feed = (chunk: string): void => {
96 const parts = chunk.split("\n");
97 for (let i = 0; i < parts.length; i++) {
98 if (i > 0) {
99 lines.push(line);
100 line = "";
101 }
102 put(parts[i]!);
103 }
104 };
105
106 let at = 0;
107 for (const m of text.matchAll(TOKEN)) {
108 if (lines.length >= limit) break;
109 feed(text.slice(at, m.index));
110 at = m.index + m[0].length;
111 if (m[0].startsWith("\x1b[") && m[2] === "m" && /^[0-9;:]*$/.test(m[1]!)) sgr(pen, m[1]!);
112 }
113 if (lines.length < limit) {
114 feed(text.slice(at));
115 lines.push(line);
116 }
117 return { lines: lines.slice(0, limit), total };
118 }
src/web/render/kinds.ts +4
@@ -23,6 +23,7 @@ export type FileKind =
2323 | "archive"
2424 | "config"
2525 | "markdown"
26 | "log"
2627 | "subtitles"
2728 | "text"
2829 | "binary";
@@ -45,6 +46,7 @@ export const RENDER_CAP: Readonly<Record<FileKind, number>> = {
4546 archive: 50 * MB,
4647 config: 1 * MB,
4748 markdown: 1 * MB,
49 log: 1 * MB,
4850 subtitles: 5 * MB,
4951 text: 1 * MB,
5052 binary: 0,
@@ -138,6 +140,8 @@ const TEXTUAL: Readonly<Record<string, FileKind>> = {
138140 md: "markdown",
139141 markdown: "markdown",
140142 mdx: "markdown",
143 log: "log",
144 ansi: "log",
141145 srt: "subtitles",
142146 vtt: "subtitles",
143147 };
src/web/render/log.ts added +36
@@ -0,0 +1,36 @@
1 /** A terminal log (.log, .ansi) with its colours, laid out as the code view is. */
2
3 import { html, raw } from "../html.ts";
4 import { rowsMarkup } from "../code_rows.ts";
5 import { btn, empty } from "../ui.ts";
6 import { ansiLines } from "./ansi.ts";
7 import type { Rendered, RenderInput } from "./renderers.ts";
8
9 /** The same windows the plain-text view draws: first lines, then up to the ceiling. */
10 const FIRST_LINES = 1_500;
11 const MAX_LINES = 10_000;
12
13 /** Whether a body holds an escape at all; one without is shown as plain text. */
14 export const hasEscape = (body: Uint8Array): boolean => body.includes(0x1b);
15
16 export function log(input: RenderInput): Rendered {
17 const { lines, total } = ansiLines(new TextDecoder().decode(input.body), input.all ? MAX_LINES : FIRST_LINES);
18 const cut = total - lines.length;
19 const href = input.pathHref ?? "";
20 return {
21 lines: total,
22 content: html`<div class="card code ansi">
23 <table>
24 ${raw(rowsMarkup(lines.map(raw)))}
25 </table>
26 </div>
27 ${cut > 0
28 ? empty({
29 icon: "file",
30 title: `${cut.toLocaleString("en-US")} more lines`,
31 note: input.all ? "The rest is in the raw file." : "The page draws the first lines so it opens at once.",
32 action: input.all ? html`` : btn("Show all lines", { href: `${href}?all=1`, id: "code-cut", icon: "file" }),
33 })
34 : html``}`,
35 };
36 }
src/web/render/renderers.ts +8 −1
@@ -13,6 +13,7 @@ import { isFormConfig } from "./config-format.ts";
1313 import { csv } from "./csv.ts";
1414 import { image } from "./image.ts";
1515 import { claimedMedia, fileKind, HEAD_BYTES, NO_BYTES, RENDER_CAP, type FileKind } from "./kinds.ts";
16 import { log, hasEscape } from "./log.ts";
1617 import { markdown } from "./markdown.ts";
1718 import { pdf } from "./pdf.ts";
1819 import { subtitles } from "./subtitles.ts";
@@ -56,6 +57,8 @@ export const RENDERERS: Partial<Readonly<Record<FileKind, Renderer>>> = {
5657 audio,
5758 video,
5859 config,
60 log,
61
junipercrew

Non-blocking: stray blank line inside the map; remove it.
5962 subtitles,
6063 archive,
6164 };
@@ -92,7 +95,11 @@ export function rendererFor(
9295 if (blob.kind === "too_large" && !MEDIA_KINDS.has(kind)) return undefined;
9396 // A kind may draw only some of its files: `config` draws JSON, YAML and
9497 // TOML as a form, and leaves the rest (INI, .env …) their source.
95 const render = kind === "config" && !isFormConfig(path) ? undefined : renderers[kind];
98 // A log with no escape in it keeps the source view, exactly as plain text.
99 const render =
100 (kind === "config" && !isFormConfig(path)) || (kind === "log" && blob.kind !== "too_large" && !hasEscape(blob.body))
101 ? undefined
102 : renderers[kind];
96103 if (render === undefined) return undefined;
97104 return { render, body: blob.kind === "too_large" ? NO_BYTES : blob.body };
98105 }
src/web/style/content.ts +11
@@ -414,6 +414,17 @@ form.inline { display: inline; margin-left: auto; }
414414 .csv-view .num { text-align: right; }
415415 .csv-more { padding: 10px 12px; margin: 0; }
416416
417 /* A terminal log (render/log.ts): SGR colours as fixed classes, theme tokens only. */
418 .ansi-red { color: var(--danger); }
419 .ansi-green { color: var(--success); }
420 .ansi-yellow { color: var(--attn); }
421 .ansi-blue { color: var(--tok-number); }
422 .ansi-magenta { color: var(--merged); }
423 .ansi-cyan { color: var(--tok-property); }
424 .ansi-gray { color: var(--tok-comment); }
425 .ansi-b { font-weight: 700; }
426 .ansi-u { text-decoration: underline; }
427
417428 /* ---- syntax tokens (colors come from the active theme) ----
418429 Two layers, exactly as the tokenizer emits them: a colour class every
419430 consumer understands, and — where the web can say something finer — a kind
test/render-ansi.test.ts added +104
@@ -0,0 +1,104 @@
1 /** Terminal colours in a .log: SGR becomes fixed classes, everything else is removed. */
2
3 import { describe, expect, test } from "bun:test";
4 import { ansiLines } from "../src/web/render/ansi.ts";
5 import { rendererFor, RENDERERS } from "../src/web/render/renderers.ts";
6 import { log } from "../src/web/render/log.ts";
7 import { buildLines } from "../src/tui/screens/workspace.ts";
8 import { theme } from "../src/tui/ui/theme.ts";
9 import type { Blob } from "../src/view/source.ts";
10
11 const one = (s: string): string => ansiLines(s, 100).lines.join("\n");
12 const enc = (s: string): Uint8Array => new TextEncoder().encode(s);
13
14 describe("ansiLines", () => {
15 test("a red word is a red span; a reset ends it", () => {
16 expect(one("a \x1b[31mfail\x1b[0m b")).toBe('a <span class="ansi-red">fail</span> b');
17 });
18
19 test("bold and underline are classes, and a colour carries across a newline", () => {
20 expect(one("\x1b[1;4;32mok\nstill\x1b[m")).toBe(
21 '<span class="ansi-green ansi-b ansi-u">ok</span>\n<span class="ansi-green ansi-b ansi-u">still</span>',
22 );
23 });
24
25 test("an OSC 8 link and a title are stripped, the text stays", () => {
26 expect(one("\x1b]8;;http://evil.test\x1b\\click\x1b]8;;\x1b\\ \x1b]0;title\x07x")).toBe("click x");
27 });
28
29 test("cursor moves and bare CR are removed, never interpreted", () => {
30 expect(one("\x1b[2K\x1b[1Aa\rb\x1b[?25l")).toBe("ab");
31 });
32
33 test("<script> inside a colour is escaped", () => {
34 const out = one("\x1b[31m<script>alert(1)</script>\x1b[0m");
35 expect(out).not.toContain("<script");
36 expect(out).toContain("&lt;script&gt;");
37 });
38
39 test("no style attribute is ever emitted, even for truecolour", () => {
40 const out = one('\x1b[38;2;1;2;3mx\x1b[48;5;9my\x1b[38;5;1mz');
41 expect(out).not.toContain("style");
42 expect(out).toBe('xy<span class="ansi-red">z</span>');
43 });
44
45 test("22, 24 and 39 end bold, underline and colour one at a time", () => {
46 expect(one("\x1b[1;4;31ma\x1b[22mb\x1b[24mc\x1b[39md")).toBe(
47 '<span class="ansi-red ansi-b ansi-u">a</span><span class="ansi-red ansi-u">b</span><span class="ansi-red">c</span>d',
48 );
49 });
50
51 test("bright colours share the normal hues: 91 is red, 90 gray, 97 the page ink", () => {
52 expect(one("\x1b[91mr\x1b[90mg\x1b[97mw")).toBe(
53 '<span class="ansi-red">r</span><span class="ansi-gray">g</span>w',
54 );
55 });
56
57 test("an unterminated OSC ends at the newline and the log text survives", () => {
58 expect(one("a\x1b]0;title\nnext line\nthird")).toBe("a\nnext line\nthird");
59 });
60
61 test("a file over the line limit is cut, and the total still counts", () => {
62 const r = ansiLines("a\nb\nc\nd\n", 2);
63 expect(r.lines).toEqual(["a", "b"]);
64 expect(r.total).toBe(4);
65 });
66 });
67
68 describe("the log kind", () => {
69 const blob = (s: string): Blob => ({ kind: "text", body: enc(s), size: s.length }) as unknown as Blob;
70
71 test("a .log or .ansi with escapes is drawn", () => {
72 for (const p of ["build.log", "out.ansi"])
73 expect(rendererFor(p, blob("\x1b[31mx"))?.render).toBe(RENDERERS.log!);
74 });
75
76 test("a .log with no escape keeps the plain source view", () => {
77 expect(rendererFor("build.log", blob("just text\n"))).toBeUndefined();
78 });
79 });
80
81 describe("log()", () => {
82 const input = (n: number, all = false) =>
83 ({ body: enc(Array.from({ length: n }, (_, i) => `\x1b[31mline ${i}`).join("\n")), all, pathHref: "/o/r/blob/main/b.log" }) as never;
84
85 test("a long log is cut with a count and a Show all lines link", () => {
86 const r = log(input(1_600));
87 expect(r.lines).toBe(1_600);
88 expect(String(r.content)).toContain("100 more lines");
89 expect(String(r.content)).toContain("Show all lines");
90 expect(String(r.content)).toContain("?all=1");
91 });
92
93 test("a short log has no cut, and ?all=1 drops the link", () => {
94 expect(String(log(input(5)).content)).not.toContain("more lines");
95 expect(String(log(input(11_000, true)).content)).not.toContain("Show all lines");
96 });
97 });
98
99 describe("the terminal file view", () => {
100 test("a .log's escapes are stripped; other files are left alone", () => {
101 expect(buildLines("\x1b[31mred\x1b[0m\x1b]0;t\x07 ok\n", "b.log", theme("dark").c).plain).toEqual(["red ok"]);
102 expect(buildLines("a\x1bb\n", "a.txt", theme("dark").c).plain[0]).toContain("\x1b");
103 });
104 });
test/render-kinds.test.ts +2
@@ -55,6 +55,8 @@ const SAMPLES: Record<string, [Uint8Array, FileKind, string]> = {
5555 md: [enc("# a\n"), "markdown", "application/octet-stream"],
5656 markdown: [enc("# a\n"), "markdown", "application/octet-stream"],
5757 mdx: [enc("# a\n"), "markdown", "application/octet-stream"],
58 log: [enc("ok\n"), "log", "application/octet-stream"],
59 ansi: [enc("\x1b[31mok\n"), "log", "application/octet-stream"],
5860 srt: [enc("1\n00:00:01,000 --> 00:00:02,000\nHi\n"), "subtitles", "application/octet-stream"],
5961 vtt: [enc("WEBVTT\n"), "subtitles", "application/octet-stream"],
6062 };

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs