Public: anyone can read this; the crew can change it.
A run's cost line shows tokens in and out and the price per million used #13985
Check · Review · Land · Live done — Landed and live in production.
torque/forge:torque/13968-a-run-s-cost-line-shows-tokens-in-and-ou → main 8 commits fork 3 threads · 3 unresolved
8 files
changed +287
−2
docs/features/README.md +1
| @@ -266,6 +266,7 @@ Read with the guide page [Your apps](../guide-your-apps.md). | ||
| 266 | 266 | - [](maintenance-duties.md) |
| 267 | 267 | - [](toolchains.md) |
| 268 | 268 | - [](run-folds.md) |
| 269 | - [](run-price.md) | |
| 269 | 270 | - [](feature-schedule.md) |
| 270 | 271 | - [](scrapes.md) |
| 271 | 272 | - [](box-pass.md) |
docs/features/run-price.md added +19
| @@ -0,0 +1,19 @@ | ||
| 1 | # Run price | |
| 2 | ||
| 3 | A finished agent run's page, and its log screen in the terminal, show the arithmetic behind its cost beside the ledger's figure (#13968): | |
| 4 | ||
| 5 | 200,000 in × $3/M + 10,000 out × $15/M = $0.7500 | |
| 6 | ||
| 7 | Tokens in and out come from the usage ledger (`core/usage.ts`), counted by the provider. The prices are dollars per 1,000,000 tokens. The product is computed in whole micro-dollars (`tokens × $/M` is already µ$), so no float drift enters. When the product and the ledger's `costUsd` differ at the cent, the line ends `· ledger $0.4000`: the ledger is the provider's own total and counts cache reads and writes at one rate, so the two can honestly differ. | |
| 8 | ||
| 9 | A model with no declared price reads `price per million unknown`. It is never guessed. A run that spent no tokens shows no line. The line is shown to the people who may read the run's log (writers on the page, the same readers in the terminal). | |
| 10 | ||
| 11 | ## Declaring prices: `FORGE_MODEL_PRICES` | |
| 12 | ||
| 13 | `model=in/out`, comma separated, dollars per million tokens, e.g. `claude-opus-5-5=15/75,claude-sonnet-5-5=3/15`. The model name is the one the run recorded (`run.model`). An entry that is not exactly that shape (a model of letters, digits and `._:-`, two numbers of up to six digits and six decimals) is dropped; the rest still apply. The variable is parsed once and again only if its text changes. | |
| 14 | ||
| 15 | It is a variable of the forge process itself, set by whoever runs it, like the other `FORGE_*` variables. forge.json cannot carry it: `env` there names secrets the app reads and refuses any name starting `FORGE_`, and holds no values. Until it is set every run says unknown. | |
| 16 | ||
| 17 | ## Where it is | |
| 18 | ||
| 19 | `src/core/run_price.ts` (`parsePrices`, `runPrice`, `priceWords`), the page in `runPriceLine` (`src/web/pages/runs.ts`), the terminal in `src/tui/screens/runlog.ts`. The crew-activity row on the issue and pull pages still shows the cost alone: the line is one run's, and a row has no room for it. Tests: `test/cost-tokens.test.ts`, `test/tui-runlog.test.ts`. |
src/core/run_price.ts added +104
| @@ -0,0 +1,104 @@ | ||
| 1 | /** | |
| 2 | * Run price — a run's cost line as arithmetic a reader can check: | |
| 3 | * tokens in, tokens out, the price per million of each, and the product, | |
| 4 | * set beside the ledger's own `costUsd` (`core/usage.ts`). | |
| 5 | * | |
| 6 | * Units: prices are US dollars per 1,000,000 tokens. The product is | |
| 7 | * usd = tokensIn × inPerM / 1e6 + tokensOut × outPerM / 1e6 | |
| 8 | * computed in whole micro-dollars so no float drift enters the comparison. | |
| 9 | * | |
| 10 | * The ledger's `costUsd` is the provider's own total, and `tokensIn` counts | |
| 11 | * cache reads and writes at one rate, so a product can honestly differ from | |
| 12 | * it: when it does the page shows both. A model with no declared price is | |
| 13 | * `unknown`, never a guess. Prices come from `FORGE_MODEL_PRICES`, e.g. | |
| 14 | * `opus=15/75,sonnet=3/15` (model=in/out, dollars per million). | |
| 15 | */ | |
| 16 | ||
| 17 | export interface Price { | |
| 18 | readonly inPerM: number; | |
| 19 | readonly outPerM: number; | |
| 20 | } | |
| 21 | ||
| 22 | export type Prices = ReadonlyMap<string, Price>; | |
| 23 | ||
| 24 | export type RunPrice = | |
| 25 | | { readonly known: false } | |
| 26 | | { | |
| 27 | readonly known: true; | |
| 28 | readonly price: Price; | |
| 29 | /** Dollars by the declared price, 6 places. */ | |
| 30 | readonly productUsd: number; | |
| 31 | readonly ledgerUsd: number; | |
| 32 | /** True when the product and the ledger differ at the ledger's cent. */ | |
| 33 | readonly differs: boolean; | |
| 34 | }; | |
| 35 | ||
| 36 | const NUMBER = /^\d{1,6}(\.\d{1,6})?$/; | |
| 37 | const MODEL = /^[A-Za-z0-9._:-]{1,100}$/; | |
| 38 | ||
| 39 | /** Parse `model=in/out,…`; an entry that is not exactly that is dropped. */ | |
| 40 | export function parsePrices(text: string | undefined): Prices { | |
| 41 | const out = new Map<string, Price>(); | |
| 42 | for (const part of (text ?? "").split(",")) { | |
| 43 | const [model = "", rates = "", extra] = part.trim().split("="); | |
| 44 | const [a = "", b = "", more] = rates.split("/"); | |
| 45 | if (extra !== undefined || more !== undefined) continue; | |
| 46 | if (!MODEL.test(model) || !NUMBER.test(a) || !NUMBER.test(b)) continue; | |
| 47 | out.set(model, { inPerM: Number(a), outPerM: Number(b) }); | |
| 48 | } | |
| 49 | return out; | |
| 50 | } | |
| 51 | ||
| 52 | /** | |
| 53 | * Tokens × dollars-per-million is already whole micro-dollars: tokens × $/M ÷ 1e6 dollars = tokens | |
| 54 | * × $/M µ$. | |
| 55 | */ | |
| 56 | const micro = (tokens: number, perM: number): number => Math.round(tokens * perM); | |
| 57 | ||
| 58 | export function runPrice( | |
| 59 | usage: { tokensIn: number; tokensOut: number; costUsd: number }, | |
| 60 | model: string, | |
| 61 | prices: Prices, | |
| 62 | ): RunPrice { | |
| 63 | const price = prices.get(model); | |
| 64 | if (price === undefined) return { known: false }; | |
| 65 | const productUsd = (micro(usage.tokensIn, price.inPerM) + micro(usage.tokensOut, price.outPerM)) / 1e6; | |
| 66 | const cents = (usd: number): number => Math.round(usd * 100); | |
| 67 | return { | |
| 68 | known: true, | |
| 69 | price, | |
| 70 | productUsd, | |
| 71 | ledgerUsd: usage.costUsd, | |
| 72 | differs: cents(productUsd) !== cents(usage.costUsd), | |
| 73 | }; | |
| 74 | } | |
| 75 | ||
| 76 | let parsedFrom: string | undefined; | |
| 77 | let parsed: Prices | undefined; | |
| 78 | ||
| 79 | /** `FORGE_MODEL_PRICES`, parsed once: re-read only when the variable's text changes. */ | |
| 80 | export function pricesFromEnv(): Prices { | |
| 81 | const text = process.env.FORGE_MODEL_PRICES; | |
| 82 | if (parsed === undefined || text !== parsedFrom) { | |
| 83 | parsed = parsePrices(text); | |
| 84 | parsedFrom = text; | |
| 85 | } | |
| 86 | return parsed; | |
| 87 | } | |
| 88 | ||
| 89 | /** | |
| 90 | * The line as plain words, for the terminal: | |
| 91 | * `1,200 in × $3/M + 300 out × $15/M = $0.0081 · ledger $0.01`, or | |
| 92 | * `price per million unknown`. | |
| 93 | */ | |
| 94 | export function priceWords( | |
| 95 | usage: { tokensIn: number; tokensOut: number; costUsd: number }, | |
| 96 | model: string, | |
| 97 | prices: Prices = pricesFromEnv(), | |
| 98 | ): string { | |
| 99 | const p = runPrice(usage, model, prices); | |
| 100 | if (!p.known) return "price per million unknown"; | |
| 101 | const n = (t: number): string => t.toLocaleString("en-US"); | |
| 102 | const ledger = p.differs ? ` · ledger $${p.ledgerUsd.toFixed(4)}` : ""; | |
| 103 | return `${n(usage.tokensIn)} in × $${p.price.inPerM}/M + ${n(usage.tokensOut)} out × $${p.price.outPerM}/M = $${p.productUsd.toFixed(4)}${ledger}`; | |
| 104 | } |
src/core/runs.ts +4
| @@ -104,6 +104,8 @@ export interface Run { | ||
| 104 | 104 | readonly turns: number; |
| 105 | 105 | /** For an agent run: the context its session held at the end, in tokens; 0 when unreported. */ |
| 106 | 106 | readonly contextTokens: number; |
| 107 | /** For an agent run: the model it ran on; empty when not recorded. */ | |
| 108 | readonly model: string; | |
| 107 | 109 | /** |
| 108 | 110 | * For an agent run: tokens it read and wrote, all invocations summed; |
| 109 | 111 | * null when unknown (never 0 for unknown). |
| @@ -155,6 +157,7 @@ interface RunRow { | ||
| 155 | 157 | heartbeat_at: string | null; |
| 156 | 158 | turns: number; |
| 157 | 159 | context_tokens: number; |
| 160 | model: string | null; | |
| 158 | 161 | tokens_in: number | null; |
| 159 | 162 | tokens_out: number | null; |
| 160 | 163 | pushed: number | null; |
| @@ -192,6 +195,7 @@ const toRun = (r: RunRow): Run => ({ | ||
| 192 | 195 | heartbeatAt: r.heartbeat_at, |
| 193 | 196 | turns: r.turns, |
| 194 | 197 | contextTokens: r.context_tokens, |
| 198 | model: r.model ?? "", | |
| 195 | 199 | tokensIn: r.tokens_in ?? null, |
| 196 | 200 | tokensOut: r.tokens_out ?? null, |
| 197 | 201 | pushed: r.pushed === null ? null : r.pushed !== 0, |
src/tui/screens/runlog.ts +22 −2
| @@ -20,6 +20,8 @@ import { subscribeRunLog } from "../../runs/log.ts"; | ||
| 20 | 20 | import { stoppedWords } from "../../runs/cancelled.ts"; |
| 21 | 21 | import { Scroll, live, openOnWeb } from "./common.ts"; |
| 22 | 22 | import { costText, envLabel, runTone } from "./runwords.ts"; |
| 23 | import { priceWords } from "../../core/run_price.ts"; | |
| 24 | import { getUsage } from "../../core/usage.ts"; | |
| 23 | 25 | import { jobSandboxWords } from "../../checks/placement.ts"; |
| 24 | 26 | |
| 25 | 27 | const LOG_LINES = 200; |
| @@ -36,6 +38,8 @@ export function runLogScreen( | ||
| 36 | 38 | let view: RunLogView | undefined; |
| 37 | 39 | let lines: string[] = []; |
| 38 | 40 | let loaded = false; |
| 41 | /** The run's tokens × price line (`core/run_price.ts`); empty until the ledger has the run. */ | |
| 42 | let price = ""; | |
| 39 | 43 | let follow = true; |
| 40 | 44 | |
| 41 | 45 | const read = (): void => { |
| @@ -44,6 +48,12 @@ export function runLogScreen( | ||
| 44 | 48 | view = got.ok && !got.value.logHidden ? got.value : undefined; |
| 45 | 49 | lines = // The tail window can hold a few more than asked for. |
| 46 | 50 | lines = view === undefined ? [] : view.lines.slice(-LOG_LINES); |
| 51 | const used = | |
| 52 | view !== undefined && view.run.kind === "agent" ? getUsage(f.app.db, runId) : undefined; | |
| 53 | price = | |
| 54 | view === undefined || used === undefined || used.tokensIn + used.tokensOut === 0 | |
| 55 | ? "" | |
| 56 | : priceWords(used, view.run.model); | |
| 47 | 57 | loaded = true; |
| 48 | 58 | if (follow) scroll.top = Number.MAX_SAFE_INTEGER; |
| 49 | 59 | ctx.invalidate(); |
| @@ -110,7 +120,12 @@ export function runLogScreen( | ||
| 110 | 120 | // does. One sentence a row, so a narrow terminal cuts neither. |
| 111 | 121 | const wall = |
| 112 | 122 | run.placement === "job" && run.sandboxEnforced === true ? jobSandboxWords(run.sandboxNote).split(/(?<=\.) /) : []; |
| 113 | const rows = split(area, "v", [len(1), ...wall.map(() => len(1)), fill(1)]); | |
| 123 | const rows = split(area, "v", [ | |
| 124 | len(1), | |
| 125 | ...(price === "" ? [] : [len(1)]), | |
| 126 | ...wall.map(() => len(1)), | |
| 127 | fill(1), | |
| 128 | ]); | |
| 114 | 129 | const head = rows[0]; |
| 115 | 130 | const body = rows.at(-1); |
| 116 | 131 | const tone = runTone(run.state); |
| @@ -130,8 +145,13 @@ export function runLogScreen( | ||
| 130 | 145 | style(tone === undefined ? c.mutedFg : c.fg, c.bg), |
| 131 | 146 | head!.x + head!.w, |
| 132 | 147 | ); |
| 148 | const lead = price === "" ? 1 : 2; | |
| 149 | if (price !== "") { | |
| 150 | const row = rows[1]!; | |
| 151 | s.text(row.x, row.y, truncate(price, row.w), style(c.mutedFg, c.bg), row.x + row.w); | |
| 152 | } | |
| 133 | 153 | wall.forEach((words, i) => { |
| 134 | const row = rows[i + 1]!; | |
| 154 | const row = rows[i + lead]!; | |
| 135 | 155 | s.text(row.x, row.y, truncate(words, row.w), style(c.mutedFg, c.bg), row.x + row.w); |
| 136 | 156 | }); |
| 137 | 157 | if (lines.length === 0) { |
src/web/pages/runs.ts +24
| @@ -31,6 +31,7 @@ import { | ||
| 31 | 31 | } from "../../core/repos.ts"; |
| 32 | 32 | import { getRun, type Run, type RunKind } from "../../core/runs.ts"; |
| 33 | 33 | import { getUsage, issueUsage } from "../../core/usage.ts"; |
| 34 | import { pricesFromEnv, runPrice, type Prices } from "../../core/run_price.ts"; | |
| 34 | 35 | import { costLineFor, type CostThing } from "../../core/ledger_rollup.ts"; |
| 35 | 36 | import { getUsersByIds, type User } from "../../core/users.ts"; |
| 36 | 37 | import { notePhase, takeSessionTime } from "../../core/route_timing.ts"; |
| @@ -160,6 +161,29 @@ export function runTokens(db: Database, run: Run): Html { | ||
| 160 | 161 | return html`<span class="num" id="run-tokens" |
| 161 | 162 | >${tokenCount(used.tokensIn, "counted")} in · |
| 162 | 163 | ${tokenCount(used.tokensOut, "counted")} out</span |
| 164 | > ${runPriceLine(run.model, used)}`; | |
outdated
Both doors: the terminal run screen has no price line, and there is no docs/features how-to. Add both. |
||
| 165 | } | |
| 166 | ||
| 167 | /** | |
| 168 | * The arithmetic behind a run's cost (`core/run_price.ts`): tokens × price per | |
| 169 | * million = product, and the ledger's own figure beside it when the two differ | |
| 170 | * at the cent. A model with no declared price says "price per million unknown". | |
| 171 | */ | |
| 172 | export function runPriceLine( | |
| 173 | model: string, | |
| 174 | used: { tokensIn: number; tokensOut: number; costUsd: number }, | |
| 175 | prices: Prices = pricesFromEnv(), | |
| 176 | ): Html { | |
| 177 | const priced = runPrice(used, model, prices); | |
outdated
Extra query per render for a model the caller's |
||
| 178 | if (!priced.known) | |
| 179 | return html`<span class="num muted" id="run-price">price per million unknown</span>`; | |
| 180 | const { price, productUsd, ledgerUsd, differs } = priced; | |
| 181 | return html`<span class="num" id="run-price" | |
| 182 | >${tokenCount(used.tokensIn, "counted")} × $${String(price.inPerM)}/M + | |
| 183 | ${tokenCount(used.tokensOut, "counted")} × $${String(price.outPerM)}/M = | |
| 184 | $${productUsd.toFixed(4)}${differs | |
| 185 | ? html` · ledger $${ledgerUsd.toFixed(4)}` | |
| 186 | : html``}</span | |
| 163 | 187 | >`; |
| 164 | 188 | } |
| 165 | 189 |
test/cost-tokens.test.ts added +92
| @@ -0,0 +1,92 @@ | ||
| 1 | /** A run's cost line: tokens × price per million, by hand, against the ledger (#13968). */ | |
| 2 | import { expect, test } from "bun:test"; | |
| 3 | import { mkdtempSync } from "node:fs"; | |
| 4 | import { tmpdir } from "node:os"; | |
| 5 | import { join } from "node:path"; | |
| 6 | import { createApp } from "../src/app.ts"; | |
| 7 | import { createRepo } from "../src/actions/repos.ts"; | |
| 8 | import { createUser } from "../src/core/users.ts"; | |
| 9 | import { claimRun, createRun, finishRun, noteRunProgress } from "../src/core/runs.ts"; | |
| 10 | import { parsePrices, pricesFromEnv, priceWords, runPrice } from "../src/core/run_price.ts"; | |
| 11 | import { getUsage, recordUsage } from "../src/core/usage.ts"; | |
| 12 | import { runPriceLine } from "../src/web/pages/runs.ts"; | |
| 13 | ||
| 14 | const prices = parsePrices("opus=15/75,sonnet=3/15,bad=x/1,also=1/2/3"); | |
| 15 | ||
| 16 | test("parsePrices keeps well-formed entries and drops the rest", () => { | |
| 17 | expect([...prices.keys()]).toEqual(["opus", "sonnet"]); | |
| 18 | expect(parsePrices(undefined).size).toBe(0); | |
| 19 | }); | |
| 20 | ||
| 21 | test("by hand: 200,000 in × $3/M + 10,000 out × $15/M = $0.60 + $0.15 = $0.75, equal to the ledger", () => { | |
| 22 | const r = runPrice({ tokensIn: 200_000, tokensOut: 10_000, costUsd: 0.75 }, "sonnet", prices); | |
| 23 | expect(r).toMatchObject({ known: true, productUsd: 0.75, ledgerUsd: 0.75, differs: false }); | |
| 24 | }); | |
| 25 | ||
| 26 | test("by hand: 1,234,567 in × $15/M + 7,891 out × $75/M = $18.518505 + $0.591825 = $19.11033", () => { | |
| 27 | const r = runPrice({ tokensIn: 1_234_567, tokensOut: 7_891, costUsd: 19.11 }, "opus", prices); | |
| 28 | expect(r).toMatchObject({ known: true, productUsd: 19.11033, differs: false }); | |
| 29 | }); | |
| 30 | ||
| 31 | test("a ledger figure that differs at the cent is flagged", () => { | |
| 32 | const r = runPrice({ tokensIn: 200_000, tokensOut: 10_000, costUsd: 0.4 }, "sonnet", prices); | |
| 33 | expect(r).toMatchObject({ known: true, productUsd: 0.75, ledgerUsd: 0.4, differs: true }); | |
| 34 | }); | |
| 35 | ||
| 36 | test("a model with no declared price is unknown", () => { | |
| 37 | expect(runPrice({ tokensIn: 1, tokensOut: 1, costUsd: 1 }, "mystery", prices)).toEqual({ known: false }); | |
| 38 | expect(runPrice({ tokensIn: 1, tokensOut: 1, costUsd: 1 }, "", prices)).toEqual({ known: false }); | |
| 39 | }); | |
| 40 | ||
| 41 | async function seed(model: string | undefined) { | |
| 42 | const app = createApp({ dataDir: mkdtempSync(join(process.env.TMPDIR ?? tmpdir(), "price-")), port: 0 }); | |
| 43 | const u = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 44 | if (!u.ok) throw new Error(u.error); | |
| 45 | const made = await createRepo(app, u.value, { name: "r" }); | |
| 46 | if (!made.ok) throw new Error(String(made.error)); | |
| 47 | const run = createRun(app.db, { repoId: made.value.id, kind: "agent" }); | |
| 48 | claimRun(app.db, run.id); | |
| 49 | if (model !== undefined) noteRunProgress(app.db, run.id, { model }); | |
| 50 | finishRun(app.db, run.id, { state: "ok", summary: "done", costUsd: 0.75 }); | |
| 51 | recordUsage(app.db, { | |
| 52 | runId: run.id, repoId: made.value.id, issueId: null, agentId: null, subscriptionId: null, | |
| 53 | tokensIn: 200_000, tokensOut: 10_000, costUsd: 0.75, wallMs: 1, outcome: "ok", at: new Date().toISOString(), | |
| 54 | }); | |
| 55 | return { db: app.db, id: run.id }; | |
| 56 | } | |
| 57 | ||
| 58 | test("the run page line shows tokens, price per million and the product equal to the ledger cost", async () => { | |
| 59 | const { db, id } = await seed("sonnet"); | |
| 60 | const used = getUsage(db, id)!; | |
| 61 | const text = String(runPriceLine("sonnet", used, prices)); | |
| 62 | expect(text).toContain("$3/M"); | |
| 63 | expect(text).toContain("$15/M"); | |
| 64 | expect(text).toContain("$0.7500"); | |
| 65 | expect(text).not.toContain("ledger $"); | |
| 66 | expect(Number((/=\s*\$([0-9.]+)/.exec(text.replace(/<[^>]*>/g, ""))?.[1]))).toBe(used.costUsd); | |
| 67 | }); | |
| 68 | ||
| 69 | test("the run page line says unknown for an unpriced model", async () => { | |
| 70 | const { db, id } = await seed("mystery"); | |
| 71 | expect(String(runPriceLine("mystery", getUsage(db, id)!, prices))).toContain("price per million unknown"); | |
| 72 | }); | |
| 73 | ||
| 74 | test("priceWords is the terminal's line: the same arithmetic as words", () => { | |
| 75 | const u = { tokensIn: 200_000, tokensOut: 10_000, costUsd: 0.4 }; | |
| 76 | expect(priceWords(u, "sonnet", prices)).toBe("200,000 in × $3/M + 10,000 out × $15/M = $0.7500 · ledger $0.4000"); | |
| 77 | expect(priceWords(u, "nope", prices)).toBe("price per million unknown"); | |
| 78 | }); | |
| 79 | ||
| 80 | test("pricesFromEnv parses once per text, not per call", () => { | |
| 81 | const was = process.env.FORGE_MODEL_PRICES; | |
| 82 | try { | |
| 83 | process.env.FORGE_MODEL_PRICES = "a=1/2"; | |
| 84 | const first = pricesFromEnv(); | |
| 85 | expect(pricesFromEnv()).toBe(first); | |
| 86 | process.env.FORGE_MODEL_PRICES = "a=3/4"; | |
| 87 | expect(pricesFromEnv().get("a")).toEqual({ inPerM: 3, outPerM: 4 }); | |
| 88 | } finally { | |
| 89 | if (was === undefined) delete process.env.FORGE_MODEL_PRICES; | |
| 90 | else process.env.FORGE_MODEL_PRICES = was; | |
| 91 | } | |
| 92 | }); |
test/tui-runlog.test.ts +21
| @@ -7,6 +7,7 @@ import { appendFileSync } from "node:fs"; | ||
| 7 | 7 | import { announceRun, cancelRepoRun } from "../src/actions/runs.ts"; |
| 8 | 8 | import { createRepo } from "../src/actions/repos.ts"; |
| 9 | 9 | import { createApp, type App as Forge } from "../src/app.ts"; |
| 10 | import { recordUsage } from "../src/core/usage.ts"; | |
| 10 | 11 | import { claimRun, createRun, finishRun, noteRunProgress } from "../src/core/runs.ts"; |
| 11 | 12 | import { JOB_GATE_NOTE } from "../src/checks/job_gate.ts"; |
| 12 | 13 | import type { Repo } from "../src/core/repos.ts"; |
| @@ -199,3 +200,23 @@ test("r rereads the log, and o prints the web address", async () => { | ||
| 199 | 200 | await key(app, "o"); |
| 200 | 201 | expect(app.screen.toText()).toContain(`/alice/lamp/runs/${run.id}`); |
| 201 | 202 | }); |
| 203 | ||
| 204 | test("an agent run's log screen shows tokens × price per million (#13968)", async () => { | |
| 205 | const run = createRun(forge.db, { repoId: repo.id, kind: "agent" }); | |
| 206 | claimRun(forge.db, run.id); | |
| 207 | noteRunProgress(forge.db, run.id, { model: "tui-priced" }); | |
| 208 | finishRun(forge.db, run.id, { state: "ok", summary: "done", costUsd: 0.75 }); | |
| 209 | recordUsage(forge.db, { | |
| 210 | runId: run.id, repoId: repo.id, issueId: null, agentId: null, subscriptionId: null, | |
| 211 | tokensIn: 200_000, tokensOut: 10_000, costUsd: 0.75, wallMs: 1, outcome: "ok", at: new Date().toISOString(), | |
| 212 | }); | |
| 213 | const was = process.env.FORGE_MODEL_PRICES; | |
| 214 | process.env.FORGE_MODEL_PRICES = "tui-priced=3/15"; | |
| 215 | try { | |
| 216 | const app = await open(alice, run.id); | |
| 217 | expect(app.screen.toText()).toContain("200,000 in × $3/M + 10,000 out × $15/M = $0.7500"); | |
| 218 | } finally { | |
| 219 | if (was === undefined) delete process.env.FORGE_MODEL_PRICES; | |
| 220 | else process.env.FORGE_MODEL_PRICES = was; | |
| 221 | } | |
| 222 | }); |
new above
offline
Env is re-parsed on every render, and FORGE_MODEL_PRICES is undocumented outside this comment and unset in forge.json. Parse once at boot and document in docs/features/.