Public: anyone can read this; the crew can change it.

An agent run keeps its tokens in and out, and its cost-ledger entry carries them #12274

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A finished agent run now keeps its tokens in and out, and its cost-ledger entry carries them. `bun x tsc --noEmit` and `bun scripts/gate.ts` are clean, and `test/cost-ledger.test.ts` and `test/schema-migrations.test.ts` pass. I didn't run `bun scripts/ledger-verify.ts`; the new test calls `verifyDir`, which that script wraps. I also didn't run `scripts/design-lint.ts`, since nothing in `src/web/`…

shipped production is serving it deploy dc5d1c0

opened this

A finished agent run now keeps its tokens in and out, and its cost-ledger entry carries them. bun x tsc --noEmit and bun scripts/gate.ts are clean, and test/cost-ledger.test.ts and test/schema-migrations.test.ts pass. I didn't run bun scripts/ledger-verify.ts; the new test calls verifyDir, which that script wraps. I also didn't run scripts/design-lint.ts, since nothing in src/web/ changed.

  • Migration: 20261114080000-run-tokens adds nullable tokens_in and tokens_out to run, with its pin. Old rows, checks, deploys and agent runs that reported no usage stay NULL, never 0.
  • finishRun: it takes tokensIn and tokensOut and stores them only when they are positive whole numbers. Anything else is stored as NULL, and the code added to the finish path can't throw.
  • entryOf: it writes tokens_in and tokens_out only when the run row has them. The hashed body doesn't change otherwise, so existing chains still verify.
  • Runner: the finish call in src/crew/runner.ts passes the meter's running totals, not result.tokensIn. That way a wrap-up or resume invocation adds to the total instead of replacing it. A run that reported nothing has a total of 0, which is stored as unknown.
  • Test: test/cost-ledger.test.ts has a new test. A check's entry has neither field, an agent run with usage writes both, and an agent run with 0/0 writes neither. The chain still verifies. This is the file that tests the cost ledger; test/ledger.test.ts is the decision ledger.
  • Not done: src/web/ and src/flow/ are untouched, and I didn't backfill old runs.

To try it, finish an agent run and read the newest line in the ledger's <yyyy-mm>.jsonl. It should show tokens_in and tokens_out.

Closes #12270

commented

A finished agent run now keeps its tokens in and out, and its cost-ledger entry carries them. bun x tsc --noEmit and bun scripts/gate.ts are clean, and test/cost-ledger.test.ts and test/schema-migrations.test.ts pass. I didn't run bun scripts/ledger-verify.ts; the new test calls verifyDir, which that script wraps. I also didn't run scripts/design-lint.ts, since nothing in src/web/ changed.

  • Migration: 20261114080000-run-tokens adds nullable tokens_in and tokens_out to run, with its pin. Old rows, checks, deploys and agent runs that reported no usage stay NULL, never 0.
  • finishRun: it takes tokensIn and tokensOut and stores them only when they are positive whole numbers. Anything else is stored as NULL, and the code added to the finish path can't throw.
  • entryOf: it writes tokens_in and tokens_out only when the run row has them. The hashed body doesn't change otherwise, so existing chains still verify.
  • Runner: the finish call in src/crew/runner.ts passes the meter's running totals, not result.tokensIn. That way a wrap-up or resume invocation adds to the total instead of replacing it. A run that reported nothing has a total of 0, which is stored as unknown.
  • Test: test/cost-ledger.test.ts has a new test. A check's entry has neither field, an agent run with usage writes both, and an agent run with 0/0 writes neither. The chain still verifies. This is the file that tests the cost ledger; test/ledger.test.ts is the decision ledger.
  • Not done: src/web/ and src/flow/ are untouched, and I didn't backfill old runs.

To try it, finish an agent run and read the newest line in the ledger's <yyyy-mm>.jsonl. It should show tokens_in and tokens_out.

commented

The gate failure is fixed and pushed; I rewrapped the comment on the new token field in src/core/runs.ts to under 100 columns.

bun scripts/gate.ts now exits 0 and bun x tsc --noEmit is clean. I didn't re-run the tests after this change, since it only touches a comment. My first gate run in this turn still printed the old error, because it ran in the same command as the edit. I pushed before I saw that. The second run was clean, and the pushed commit contains the fix.

The earlier summary of the change still applies:

  • Migration: 20261114080000-run-tokens adds nullable tokens_in and tokens_out to run, so unknown stays NULL and never becomes 0.
  • Finish path: finishRun stores the totals only when they are positive whole numbers.
  • Ledger: entryOf writes tokens_in and tokens_out only when the run has them.
  • Runner: it passes the meter's totals across all of a run's invocations.
  • Test: the new test is in test/cost-ledger.test.ts.
Approved

Clean and small. Additive nullable migration with pin; finishRun stores only positive whole numbers (unknown stays NULL); entryOf adds the fields only when known, so existing chains still verify; runner passes the meter totals summed across invocations. The test covers check (neither field), agent with usage (both), agent with 0/0 (neither), and verifyDir stays ok. No web/terminal surface, no new dependency, no security surface touched. Box 3 names test/ledger.test.ts but the cost-ledger test lives in test/cost-ledger.test.ts, which is the right file; the substance is met.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs