Public: anyone can read this; the crew can change it.
A talk answer shows what it cost in dollars beside it, from the cost ledger, and a conversation shows its total #13456
Check · Review · Land · Live done — Landed and live in production.
torque/forge:torque/13429-a-talk-answer-shows-what-it-cost-in-doll → main 5 commits fork 3 threads · 3 unresolved
10 files
changed +166
−17
docs/design/areas/talk.md +1 −1
| @@ -33,7 +33,7 @@ | ||
| 33 | 33 | - `.talk-files` — the line of file badges under a sent message; each links to the owner-only download. |
| 34 | 34 | - `.talk-attach` — the "Attach" file-picker label in the composer bar, a `.btn` that stays hidden until the script runs (the picker needs it). |
| 35 | 35 | |
| 36 | Under a reply, `.talk-cost` is one quiet line with the turn's dollars; above the log, `.talk-context` says how full the context is and, from 80 %, suggests a new conversation. Both are `small muted` text, no new colour. | |
| 36 | Under a reply, `.talk-cost` is one quiet line with the turn's dollars; above the log, `.talk-total` says what the conversation has cost in all and `.talk-context` says how full the context is and, from 80 %, suggests a new conversation. Both are `small muted` text, no new colour. | |
| 37 | 37 | - `.talk-queued` — a message waiting for the running turn: the person's words, dimmed, its "Queued" line and Cancel under it. |
| 38 | 38 | |
| 39 | 39 | Search (#8823) adds no component class: the form is a `.bar` with `.input` and `.btn`, and the results and the conversations list are `.list` of `row()`s (an "archived" `badge` as the trail, the match as the row's meta). The turn a result opens at is marked by a muted hairline on its left edge, `.talk-turn:target`, the one new rule, which replaces the two `.talk-chats li a` rules the rows made unnecessary. |
docs/features/talk.md +1 −1
| @@ -92,7 +92,7 @@ Prior art read: Vercel AI Elements (Apache-2.0 React on shadcn; anatomy and beha | ||
| 92 | 92 | |
| 93 | 93 | **Without the script** the page is a plain form: Enter adds a line, starters are links that prefill the box, and Copy and Latest are not drawn (the script draws them, along with the "Enter sends" hint). The log is `role="log"`; with the script, a new message while you are scrolled up is announced in the status line. |
| 94 | 94 | |
| 95 | **Cost and context.** Under a finished reply a quiet line gives the turn's dollars (`$0.12 this turn`); above the log, "Context 85% full (170k of 200k tokens)". Both are read from the run's record (`conversation_run`: `cost_usd`, and `context_tokens`/`context_window` from the last message's usage and the result's `modelUsage`), shown by `core/conversation_cost.ts`. When a run names no window, 200k is assumed and the line says so by showing it. From 80 % the line adds that a new conversation answers faster and costs less (information only: there is no way to start another yet, see "Many conversations"). A reply that streams in gets both when the stream ends; the terminal prints the last turn's cost and the share after its confirmation line. | |
| 95 | **Cost and context.** Above the log, "Spent $0.52 in this conversation" totals the finished runs' `cost_usd` (`conversationTotal`); it, and a turn's cost, show nothing when no run reported one, never `$0`. `GET /api/v1/conversation/turns` (the read the MCP `say` tool points to) carries `totalUsd` and each agent turn's `costUsd`, `null` when unknown. Under a finished reply a quiet line gives the turn's dollars (`$0.12 this turn`); above the log, "Context 85% full (170k of 200k tokens)". Both are read from the run's record (`conversation_run`: `cost_usd`, and `context_tokens`/`context_window` from the last message's usage and the result's `modelUsage`), shown by `core/conversation_cost.ts`. When a run names no window, 200k is assumed and the line says so by showing it. From 80 % the line adds that a new conversation answers faster and costs less (information only: there is no way to start another yet, see "Many conversations"). A reply that streams in gets both when the stream ends; the terminal (`forge talk`) prints the last turn's cost, the total ("Spent $0.52 in this conversation") and the share after its confirmation line. The MCP read of all this is the turns route above. | |
| 96 | 96 | |
| 97 | 97 | **Stopped is not failed.** A turn the person stopped is stored with `conversation_run.state = 'stopped'` (the `stopped` column is kept and set alongside, only for a release still in play that counts by it). The flow vitals line says it apart: `Talk: 2 turns failed, 1 stopped by the person` (`flow/talk_turns.ts`), and the failure rate leaves stopped turns out. The line is in the flow telemetry (`GET /api/v1/telemetry/flow` and the MCP flow tool) as `vitals.says`, with the counts at `vitals.talkTurns`. |
| 98 | 98 |
src/core/conversation_cost.ts +18 −1
| @@ -10,7 +10,10 @@ export const CONTEXT_TOKENS = 200_000; | ||
| 10 | 10 | /** From this share of the context a new conversation is suggested. */ |
| 11 | 11 | export const FULL_AT = 0.8; |
| 12 | 12 | |
| 13 | /** The dollars of the run that wrote each agent turn, by turn id: on the last agent turn inside the run's span. */ | |
| 13 | /** | |
| 14 | * The dollars of the run that wrote each agent turn, by turn id: on the last agent turn inside the | |
| 15 | * run's span. | |
| 16 | */ | |
| 14 | 17 | export function turnCosts(db: Database, conversationId: string): Map<string, number> { |
| 15 | 18 | const rows = db |
| 16 | 19 | .query<{ id: string; cost: number }, [string]>( |
| @@ -27,6 +30,20 @@ export function turnCosts(db: Database, conversationId: string): Map<string, num | ||
| 27 | 30 | return costs; |
| 28 | 31 | } |
| 29 | 32 | |
| 33 | /** | |
| 34 | * What the conversation's finished runs cost in all, from the same records; undefined when none | |
| 35 | * reported a cost (never $0). | |
| 36 | */ | |
| 37 | export function conversationTotal(db: Database, conversationId: string): number | undefined { | |
| 38 | const row = db | |
| 39 | .query<{ n: number | null }, [string]>("SELECT SUM(cost_usd) AS n FROM conversation_run WHERE conversation_id = ?1 AND finished_at IS NOT NULL AND cost_usd > 0") | |
| 40 | .get(conversationId); | |
| 41 | return row?.n == null || row.n <= 0 ? undefined : row.n; | |
| 42 | } | |
| 43 | ||
| 44 | /** The header's words: "Spent $0.52 in this conversation". */ | |
| 45 | export const totalText = (usd: number): string => `Spent ${dollars(usd)} in this conversation`; | |
| 46 | ||
| 30 | 47 | export interface Context { |
| 31 | 48 | readonly tokens: number; |
| 32 | 49 | /** The model's window as its run reported it, else `CONTEXT_TOKENS`. */ |
src/http/api/conversation_talk.ts +18 −7
| @@ -12,6 +12,7 @@ import { blockedFields } from "../../kernel/blocked.ts"; | ||
| 12 | 12 | import type { App } from "../../app.ts"; |
| 13 | 13 | import { RateLimiter } from "../../core/ratelimit.ts"; |
| 14 | 14 | import { cancelTalk, stopTalk, TALK_WORDS, talk } from "../../actions/talk.ts"; |
| 15 | import { conversationTotal, turnCosts } from "../../core/conversation_cost.ts"; | |
| 15 | 16 | import { |
| 16 | 17 | archiveConversation, |
| 17 | 18 | createConversation, |
| @@ -63,7 +64,10 @@ export function turnsLimit(raw: string | null): number { | ||
| 63 | 64 | return Math.min(MAX_PAGE, Math.max(1, Math.floor(n))); |
| 64 | 65 | } |
| 65 | 66 | |
| 66 | /** Files a person may upload in a minute; a refused file still costs a read, so this is checked first. */ | |
| 67 | /** | |
| 68 | * Files a person may upload in a minute; a refused file still costs a read, so this is checked | |
| 69 | * first. | |
| 70 | */ | |
| 67 | 71 | export const UPLOADS_PER_MINUTE = 20; |
| 68 | 72 | const limiters = new WeakMap<App, RateLimiter>(); |
| 69 | 73 | const uploadLimiter = (app: App): RateLimiter => { |
| @@ -94,7 +98,8 @@ export function conversationTalkRoutes(router: Router<Handler>): void { | ||
| 94 | 98 | return apiError("invalid_files", 422, `files is a list of at most ${FILES_PER_MESSAGE} attachment ids`); |
| 95 | 99 | const files = pendingAmong(ctx.app.db, user.id, ids as string[]); |
| 96 | 100 | if (!files.ok) return apiError("not_found", 404, FILE_WORDS.not_found[1]); |
| 97 | // What to do when a turn runs: absent hands the words to it; `queue` waits for it; `steer` stops it and sends these at once. | |
| 101 | // What to do when a turn runs: absent hands the words to it; `queue` waits for it; `steer` | |
| 102 | // stops it and sends these at once. | |
| 98 | 103 | const during = input["during"]; |
| 99 | 104 | if (during !== undefined && during !== "queue" && during !== "steer") return apiError("invalid_during", 422, "during is queue or steer"); |
| 100 | 105 | const said = talk(ctx.app, user, text, files.value, during === "queue" ? "queue" : during === "steer" ? "interrupt" : "inject", named, typeof input["about"] === "string" ? input["about"] : null); |
| @@ -125,7 +130,8 @@ export function conversationTalkRoutes(router: Router<Handler>): void { | ||
| 125 | 130 | return json({ stopped: true }, 202); |
| 126 | 131 | }); |
| 127 | 132 | |
| 128 | // One file, as the raw body; its name in the url-encoded `x-file-name`. Its type is what its bytes are. | |
| 133 | // One file, as the raw body; its name in the url-encoded `x-file-name`. Its type is what its | |
| 134 | // bytes are. | |
| 129 | 135 | router.add("POST", "/api/v1/conversation/files", async (ctx) => { |
| 130 | 136 | const user = ctx.auth.user; |
| 131 | 137 | if (user === undefined) return unauthorized(); |
| @@ -189,9 +195,13 @@ export function conversationTalkRoutes(router: Router<Handler>): void { | ||
| 189 | 195 | // An agent turn carries the steps line and links of the calls before it (`core/talk_work.ts`). |
| 190 | 196 | const all = turns.ok ? turns.value : []; |
| 191 | 197 | const works = worksOf(ctx.app.db, user.id, conversation.id, all); |
| 192 | const shown = all.map((t) => (t.role === "agent" ? { ...t, work: works.get(t.id) ?? null } : t)); | |
| 193 | // The tag is of the exact bytes sent (so `before` and `limit` give their own), made after the person's own access was settled above. | |
| 194 | const text = JSON.stringify({ running: turnRunning(ctx.app, user.id, conversation.id), conversation: conversation.id, queued: queuedOf(ctx.app, user.id, conversation.id).map((q) => ({ id: q.id, text: q.text })), turns: shown }); | |
| 198 | // Each agent turn carries what its run cost and the conversation its total, both from the usage | |
| 199 | // records; null when unknown, never 0. | |
| 200 | const costs = turnCosts(ctx.app.db, conversation.id); | |
| 201 | const shown = all.map((t) => (t.role === "agent" ? { ...t, work: works.get(t.id) ?? null, costUsd: costs.get(t.id) ?? null } : t)); | |
| 202 | // The tag is of the exact bytes sent (so `before` and `limit` give their own), made after the | |
| 203 | // person's own access was settled above. | |
| 204 | const text = JSON.stringify({ running: turnRunning(ctx.app, user.id, conversation.id), conversation: conversation.id, totalUsd: conversationTotal(ctx.app.db, conversation.id) ?? null, queued: queuedOf(ctx.app, user.id, conversation.id).map((q) => ({ id: q.id, text: q.text })), turns: shown }); | |
| 195 | 205 | const etag = weakEtag(text); |
| 196 | 206 | const headers = { etag, "cache-control": "private", vary: "authorization, cookie" }; |
| 197 | 207 | if (holds(ctx.request, etag)) return new Response(null, { status: 304, headers }); |
| @@ -201,7 +211,8 @@ export function conversationTalkRoutes(router: Router<Handler>): void { | ||
| 201 | 211 | router.add("GET", "/api/v1/conversations", (ctx) => { |
| 202 | 212 | const user = ctx.auth.user; |
| 203 | 213 | if (user === undefined) return unauthorized(); |
| 204 | // The tag is over the viewer and the rendered body, so any create, rename, archive or turn moves it; the 304 is after the access check. | |
| 214 | // The tag is over the viewer and the rendered body, so any create, rename, archive or turn | |
| 215 | // moves it; the 304 is after the access check. | |
| 205 | 216 | const text = JSON.stringify({ conversations: listConversations(ctx.app.db, user.id, { archived: ctx.url.searchParams.get("archived") === "1" }) }); |
| 206 | 217 | const etag = weakEtag(`${user.id}\n${text}`); |
| 207 | 218 | const headers = { etag, "cache-control": "private, no-cache", vary: "authorization, cookie" }; |
src/tui/ssh-talk.ts +14 −4
| @@ -29,7 +29,7 @@ | ||
| 29 | 29 | |
| 30 | 30 | import { cancelTalk, queuedTalk, TALK_WORDS, stopTalk, talk, talkAgain } from "../actions/talk.ts"; |
| 31 | 31 | import { FILE_WORDS, IMAGE_MAX, pendingAmong, removePending, storeFile } from "../core/conversation_files.ts"; |
| 32 | import { contextOf, contextText, dollars, turnCosts } from "../core/conversation_cost.ts"; | |
| 32 | import { contextOf, contextText, conversationTotal, dollars, totalText, turnCosts } from "../core/conversation_cost.ts"; | |
| 33 | 33 | import { openConversation } from "../core/conversations.ts"; |
| 34 | 34 | import { aboutLine } from "../core/talk_about.ts"; |
| 35 | 35 | import type { App } from "../app.ts"; |
| @@ -52,19 +52,27 @@ export function isStop(words: string): boolean { | ||
| 52 | 52 | return words.trim() === "/stop" || words.trim() === "--stop"; |
| 53 | 53 | } |
| 54 | 54 | |
| 55 | /** `/queue <words>` or `/steer <words>`: what to do with the words while an answer runs; undefined for plain words. */ | |
| 55 | /** | |
| 56 | * `/queue <words>` or `/steer <words>`: what to do with the words while an answer runs; undefined | |
| 57 | * for plain words. | |
| 58 | */ | |
| 56 | 59 | export function parseDuring(words: string): { during: "queue" | "interrupt"; words: string } | undefined { |
| 57 | 60 | const m = /^\/(queue|steer)(?:[ \t]+([\s\S]*))?$/.exec(words.trim()); |
| 58 | 61 | return m === null ? undefined : { during: m[1] === "queue" ? "queue" : "interrupt", words: m[2] ?? "" }; |
| 59 | 62 | } |
| 60 | 63 | |
| 61 | /** `--attach <name> <words>`: the name given to the file and the words after it; undefined without the flag. */ | |
| 64 | /** | |
| 65 | * `--attach <name> <words>`: the name given to the file and the words after it; undefined without | |
| 66 | * the flag. | |
| 67 | */ | |
| 62 | 68 | export function parseAttach(words: string): { name: string; words: string } | undefined { |
| 63 | 69 | const m = /^--attach[ \t]+(\S+)(?:[ \t]+([\s\S]*))?$/.exec(words.trim()); |
| 64 | 70 | return m === null ? undefined : { name: m[1]!, words: m[2] ?? "" }; |
| 65 | 71 | } |
| 66 | 72 | |
| 67 | /** `--about <ref> <words>`: the subject named and the words after it; undefined without the flag. */ | |
| 73 | /** | |
| 74 | * `--about <ref> <words>`: the subject named and the words after it; undefined without the flag. | |
| 75 | */ | |
| 68 | 76 | export function parseAbout(words: string): { about: string; words: string } | undefined { |
| 69 | 77 | const m = /^--about[ \t]+(\S+)(?:[ \t]+([\s\S]*))?$/.exec(words.trim()); |
| 70 | 78 | return m === null ? undefined : { about: m[1]!, words: m[2] ?? "" }; |
| @@ -153,6 +161,8 @@ export function serveTalk(app: App, session: SshSession, user: User | undefined, | ||
| 153 | 161 | const last = [...turnCosts(app.db, conversation.id).values()].at(-1); |
| 154 | 162 | const context = contextOf(app.db, conversation.id); |
| 155 | 163 | if (last !== undefined) session.write(`last turn ${dollars(last)}${CRLF}`); |
| 164 | const total = conversationTotal(app.db, conversation.id); | |
| 165 | if (total !== undefined) session.write(`${totalText(total)}${CRLF}`); | |
| 156 | 166 | if (context !== undefined) session.write(`${contextText(context)}${CRLF}`); |
| 157 | 167 | } |
| 158 | 168 | session.exit(0); |
src/web/pages/talk.ts +11 −1
| @@ -40,7 +40,7 @@ import { | ||
| 40 | 40 | type Turn, |
| 41 | 41 | } from "../../core/conversations.ts"; |
| 42 | 42 | import { conversationMarkdown } from "../../core/conversation_markdown.ts"; |
| 43 | import { contextOf, contextText, dollars, turnCosts, type Context } from "../../core/conversation_cost.ts"; | |
| 43 | import { contextOf, contextText, conversationTotal, dollars, totalText, turnCosts, type Context } from "../../core/conversation_cost.ts"; | |
| 44 | 44 | import { aboutLine as aboutText } from "../../core/talk_about.ts"; |
| 45 | 45 | import { WORDS_MAX } from "../../core/fence.ts"; |
| 46 | 46 | import { thinkingOf, worksOf, type Work } from "../../core/talk_work.ts"; |
| @@ -210,6 +210,13 @@ export const queuedHtml = (q: { id: string; text: string }): Html => | ||
| 210 | 210 | const contextLine = (c: Context | undefined): Html => |
| 211 | 211 | html`<p class="talk-context small muted" data-context ${c === undefined ? html`hidden` : null}>${c === undefined ? "" : contextText(c)}</p>`; |
| 212 | 212 | |
| 213 | /** | |
| 214 | * The running total in the header; always drawn so a finished run can fill it, hidden while | |
| 215 | * unknown. | |
| 216 | */ | |
| 217 | const totalLine = (usd: number | undefined): Html => | |
Principle 1: a second |
||
| 218 | html`<p class="talk-total small muted" data-total ${usd === undefined ? html`hidden` : null}>${usd === undefined ? "" : totalText(usd)}</p>`; | |
| 219 | ||
| 213 | 220 | /** The cost line under a reply. */ |
| 214 | 221 | const costLine = (usd: number): Html => html`<p class="talk-cost small muted">${dollars(usd)} this turn</p>`; |
| 215 | 222 | |
| @@ -379,6 +386,7 @@ async function talkPage(ctx: Ctx, conversation: Conversation, before0: string | | ||
| 379 | 386 | <nav class="seg talk-tabs" aria-label="Chat or preview"><a href="?pane=chat"${shown === "chat" ? raw(' aria-current="page"') : null}>Chat</a><a href="?pane=preview"${shown !== "chat" ? raw(' aria-current="page"') : null}>Preview</a></nav> |
| 380 | 387 | <div class="talk" data-talk data-user="${user.id}" data-c="${conversation.id}" data-here="${here}" data-paged="${paged ? "1" : "0"}" data-running="${running && !paged ? "1" : "0"}"> |
| 381 | 388 | <div> |
| 389 | ${totalLine(conversationTotal(ctx.app.db, conversation.id))} | |
| 382 | 390 | ${contextLine(context)} |
| 383 | 391 | ${turns.length === DEFAULT_PAGE && turns[0] !== undefined |
| 384 | 392 | ? html`<p class="talk-more small"><a href="${here}?before=${turns[0].id}">Earlier</a></p>` |
| @@ -634,8 +642,10 @@ export function talkStream(ctx: Ctx): Response { | ||
| 634 | 642 | // The run stored its figures before it stopped running: send them with the end. |
| 635 | 643 | const costs = [...turnCosts(app.db, conversation.id)].slice(-5).map(([id, usd]) => ({ id: `t-${id}`, html: costLine(usd).text })); |
| 636 | 644 | const context = contextOf(app.db, conversation.id); |
| 645 | const total = conversationTotal(app.db, conversation.id); | |
| 637 | 646 | emit("done", { |
| 638 | 647 | costs, |
| 648 | total: total === undefined ? "" : totalText(total), | |
| 639 | 649 | context: context === undefined ? "" : contextText(context), |
| 640 | 650 | acts: [...exchangeActs(app.db, user.id, conversation.id, false, ((c) => (c.id === conversation.id ? HERE : urlOf(conversation)))(openConversation(app.db, user.id)))].map(([id, act]) => ({ id: `t-${id}`, html: act.text })), |
| 641 | 651 | }); |
src/web/pages/talk_client.ts +3 −2
| @@ -51,7 +51,7 @@ export const talkCss = /* css */ ` | ||
| 51 | 51 | .talk-compose .bar { margin-top: 6px; gap: 8px; align-items: center; flex-wrap: wrap; } |
| 52 | 52 | .talk-compose .btn { min-height: 2.75rem; } |
| 53 | 53 | .talk-status { min-height: 1.5em; margin: 0; } |
| 54 | .talk-context, .talk-cost { margin: 0; } | |
| 54 | .talk-context, .talk-total, .talk-cost { margin: 0; } | |
| 55 | 55 | .talk-meta { margin: 4px 0 0; display: flex; gap: 6px; align-items: center; } |
| 56 | 56 | .talk-copy { margin-left: auto; background: none; border: 0; color: inherit; font: inherit; cursor: pointer; padding: 6px 8px; min-height: 2rem; opacity: 0.7; } |
| 57 | 57 | .talk-copy:hover { opacity: 1; text-decoration: underline; } |
| @@ -249,8 +249,9 @@ if (root) { | ||
| 249 | 249 | log.removeAttribute("aria-busy"); |
| 250 | 250 | mode(false); dropActs(); say("The forge replied."); source.close(); source = null; |
| 251 | 251 | document.querySelectorAll("[data-frame]").forEach((f) => { f.src = f.src; }); |
| 252 | const o = JSON.parse(e.data), c = root.querySelector("[data-context]"); | |
| 252 | const o = JSON.parse(e.data), c = root.querySelector("[data-context]"), t = root.querySelector("[data-total]"); | |
| 253 | 253 | for (const k of o.costs || []) { const li = document.getElementById(k.id); if (li && !li.querySelector(".talk-cost")) li.insertAdjacentHTML("beforeend", k.html); } |
| 254 | if (o.total && t) { t.hidden = false; t.textContent = o.total; } | |
outdated
|
||
| 254 | 255 | if (o.context && c) { c.hidden = false; c.textContent = o.context; } |
| 255 | 256 | for (const a of o.acts || []) { |
| 256 | 257 | const li = document.getElementById(a.id); |
test/conversation-talk.test.ts +1
| @@ -314,6 +314,7 @@ echo '{"type":"result","subtype":"success","is_error":false,"result":"Heard.","t | ||
| 314 | 314 | .run(conversation.id, id, at, new Date(Date.now() + 1000).toISOString()); |
| 315 | 315 | const r = await exec("talk and the figures"); |
| 316 | 316 | expect(r.out).toContain("last turn $0.50"); |
| 317 | expect(r.out).toMatch(/Spent \$0\.5\d in this conversation/); | |
| 317 | 318 | expect(r.out).toContain("Context 50% full (500k of 1M tokens)"); |
| 318 | 319 | app.db.query("DELETE FROM conversation_run WHERE id = 'rc1'").run(); |
| 319 | 320 | await settle(); |
test/talk-cost.test.ts added +98
| @@ -0,0 +1,98 @@ | ||
| 1 | /** | |
| 2 | * What an answer cost (#13429): each agent turn and the conversation's total are read from the | |
| 3 | * runs' own usage records (`conversation_run`), on the page and in the turns API the MCP points | |
| 4 | * to; an unknown cost shows nothing, never $0. | |
| 5 | */ | |
| 6 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 7 | import { mkdtemp, rm } from "node:fs/promises"; | |
| 8 | import { tmpdir } from "node:os"; | |
| 9 | import { join } from "node:path"; | |
| 10 | import { createApp, type App } from "../src/app.ts"; | |
| 11 | import { appendTurn, newConversation } from "../src/core/conversations.ts"; | |
| 12 | import { conversationTotal, totalText } from "../src/core/conversation_cost.ts"; | |
| 13 | import { createSession } from "../src/core/sessions.ts"; | |
| 14 | import { createUser, type User } from "../src/core/users.ts"; | |
| 15 | import { serve, type Servers } from "../src/http/server.ts"; | |
| 16 | import { quietLogs } from "./helpers/globals.ts"; | |
| 17 | ||
| 18 | quietLogs(); | |
| 19 | ||
| 20 | let dir: string; | |
| 21 | let app: App; | |
| 22 | let servers: Servers; | |
| 23 | let alice: User; | |
| 24 | ||
| 25 | const run = (c: string, id: string, cost: number, start: number) => | |
| 26 | app.db | |
| 27 | .query("INSERT INTO conversation_run (id, conversation_id, user_id, prompt, state, cost_usd, started_at, finished_at) VALUES (?1,?2,?3,'x','ok',?4,?5,?6)") | |
| 28 | .run(id, c, alice.id, cost, new Date(start).toISOString(), new Date(start + 5000).toISOString()); | |
| 29 | ||
| 30 | beforeAll(async () => { | |
| 31 | dir = await mkdtemp(join(tmpdir(), "forge-talk-cost-")); | |
| 32 | app = createApp({ dataDir: dir, port: 0 }); | |
| 33 | servers = await serve(app); | |
| 34 | alice = await (async () => { | |
| 35 | const u = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 36 | if (!u.ok) throw new Error("seed"); | |
| 37 | return u.value; | |
| 38 | })(); | |
| 39 | }); | |
| 40 | afterAll(async () => { | |
| 41 | await servers.stop(); | |
| 42 | await rm(dir, { recursive: true, force: true }); | |
| 43 | }); | |
| 44 | ||
| 45 | describe("talk cost", () => { | |
| 46 | test("known cost shows on the answer, in the header and in the turns API; unknown shows nothing", async () => { | |
| 47 | const conv = newConversation(app.db, alice.id); | |
| 48 | appendTurn(app.db, alice.id, conv.id, { role: "person", text: "one" }); | |
| 49 | const t0 = Date.now() - 1000; | |
| 50 | const known = appendTurn(app.db, alice.id, conv.id, { role: "agent", text: "costly answer" }); | |
| 51 | if (!known.ok) throw new Error(known.error); | |
| 52 | run(conv.id, "k1", 0.5, t0); | |
| 53 | expect(conversationTotal(app.db, conv.id)).toBe(0.5); | |
| 54 | expect(totalText(0.5)).toBe("Spent $0.50 in this conversation"); | |
| 55 | ||
| 56 | const headers = { cookie: `forge_session=${createSession(app.db, alice.id).token}` }; | |
| 57 | const page = await (await fetch(`${servers.url}/talk/${conv.id}`, { headers })).text(); | |
| 58 | expect(page).toContain("Spent $0.50 in this conversation"); | |
| 59 | expect(page).toContain("$0.50 this turn"); | |
| 60 | ||
| 61 | const api = (await (await fetch(`${servers.url}/api/v1/conversation/turns?conversation=${conv.id}`, { headers })).json()) as { | |
| 62 | totalUsd: number | null; | |
| 63 | turns: { role: string; costUsd?: number | null }[]; | |
| 64 | }; | |
| 65 | expect(api.totalUsd).toBe(0.5); | |
| 66 | expect(api.turns.find((t) => t.role === "agent")?.costUsd).toBe(0.5); | |
| 67 | ||
| 68 | // A conversation whose runs reported no cost: nothing, not $0. | |
| 69 | const other = newConversation(app.db, alice.id); | |
| 70 | appendTurn(app.db, alice.id, other.id, { role: "agent", text: "free answer" }); | |
| 71 | expect(conversationTotal(app.db, other.id)).toBeUndefined(); | |
| 72 | const page2 = await (await fetch(`${servers.url}/talk/${other.id}`, { headers })).text(); | |
| 73 | expect(page2).not.toContain("Spent $"); | |
| 74 | expect(page2).not.toContain("$0"); | |
| 75 | const api2 = (await (await fetch(`${servers.url}/api/v1/conversation/turns?conversation=${other.id}`, { headers })).json()) as { | |
| 76 | totalUsd: number | null; | |
| 77 | turns: { costUsd?: number | null }[]; | |
| 78 | }; | |
| 79 | expect(api2.totalUsd).toBeNull(); | |
| 80 | expect(api2.turns[0]?.costUsd).toBeNull(); | |
| 81 | }); | |
| 82 | }); | |
| 83 | ||
| 84 | describe("talk cost, mixed runs", () => { | |
| 85 | test("a run costing 0 beside a costed one adds nothing and never shows $0", async () => { | |
| 86 | const conv = newConversation(app.db, alice.id); | |
| 87 | appendTurn(app.db, alice.id, conv.id, { role: "agent", text: "mixed answer" }); | |
| 88 | const t0 = Date.now() - 20_000; | |
| 89 | run(conv.id, "m1", 0.25, t0); | |
| 90 | run(conv.id, "m2", 0, t0 + 6000); | |
| 91 | expect(conversationTotal(app.db, conv.id)).toBe(0.25); | |
| 92 | const headers = { cookie: `forge_session=${createSession(app.db, alice.id).token}` }; | |
| 93 | const page = await (await fetch(`${servers.url}/talk/${conv.id}`, { headers })).text(); | |
| 94 | expect(page).toContain("Spent $0.25 in this conversation"); | |
| 95 | expect(page).not.toContain("$0.00"); | |
| 96 | expect(page).not.toContain("Spent $0 "); | |
| 97 | }); | |
| 98 | }); |
test/talk-page.test.ts +1
| @@ -877,6 +877,7 @@ describe("cost and context (#8822)", () => { | ||
| 877 | 877 | const sent = JSON.parse(done?.data ?? "{}"); |
| 878 | 878 | expect(sent.costs[0].html).toContain("$0.12 this turn"); |
| 879 | 879 | expect(sent.context).toContain("Context 85% full"); |
| 880 | expect(sent.total).toBe("Spent $0.12 in this conversation"); | |
| 880 | 881 | // Under 80 %: no suggestion. |
| 881 | 882 | app.db.query("UPDATE conversation_run SET context_tokens = 40000").run(); |
| 882 | 883 | const calm = await (await get("/talk", erin.value)).text(); |
new above
offline
No test covers this terminal line. Add one that fails without it.