trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns15g rEventsFlows

A forge's owner sets their own model key and their own budgets, and the crew picks the work back up without anyone else's help #11553

Merged scoutcrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/11515-a-forge-s-owner-sets-their-own-model-key → main 3 commits fork

  • Merged. scout/11515-a-forge-s-owner-sets-their-own-model-key was merged into main as a5a2512.

Review

Sign in to review this change.
16 files changed +867 −13
docs/features/own-forge-budgets.md added +11
@@ -0,0 +1,11 @@
1 # A forge's own key and budgets
2
3 A forge that is its owner's own (a hosted child) runs its crew on the key its owner added, and on limits its owner sets.
4
5 - **Key.** Add one from the first screen (or `/settings/crew/key`, or the terminal's crew-key screen). Every repository on the forge uses it; no grant on any other forge is consulted, and nothing says "ask an admin". Until the forge has a key, a run stops with "This forge has no model key: add yours at /settings/crew" and its issue is parked.
6 - **Budgets.** Settings → Crew → Budgets (`/settings/crew#budgets`, admin only), or the terminal's "Crew budgets" screen (palette entry, forge admins who are people only; j/k choose, enter types dollars), hold spend per run, per UTC day and per UTC month, with what was spent against the day and the month. Defaults: $5 a run, $50 a day, $500 a month. A run a limit stops says which one and links back to that setting. Day and month count the agent runs in the usage ledger (conversations are not counted).
7 - **A ceiling, not a hint.** A run reserves its cap against the day and the month when it starts (`ceiling_reserved_usd` on the run, the column the account ceilings use), so agents starting together share what is left; the reservation goes when the run leaves queued/running. A run cut off mid-way at a cap one of these limits set says which limit in its summary. One thing can still overshoot: a conversation is not counted, and a run's last call may finish a little past its cap, as with every cap on the forge.
8 - **Picked back up.** Adding a key comments once on each open issue whose newest park was for "no model key", "no model account" or "no model grant"; raising a limit does the same for issues a limit of the owner's stopped (`src/crew/wake.ts`, at most 200 issues a wake, found by a query over park notes and not by walking every repository). The comment is in the admin's name; a person's word restarts the park count, so the crew tries the issue again on its next pass. A sealed issue's comment is seen only where the issue is. The owner is told "3 issues picked back up".
9 - **Who pays.** With its own key the forge never draws on its host's included credit, and its home says so. A forge still on the included credit works as before.
10
11 The limits bind only a forge that has a `parent.json` in its data directory; the forge that hosts others sets spend through account ceilings.
src/actions/crew_budgets.ts added +33
@@ -0,0 +1,33 @@
1 /**
2 * The owner's spend limits, set from the Crew settings page (#11515): per
3 * run, per day, per month. A forge admin only; each amount is checked at the
4 * boundary; a limit raised wakes the issues a limit had parked.
5 */
6
7 import type { App } from "../app.ts";
8 import { BUDGET_KEYS, budgetLimit, setBudget, validBudget, type BudgetKey } from "../crew/forge_budget.ts";
9 import { wakeParked } from "../crew/wake.ts";
10 import type { User } from "../core/users.ts";
11
12 export type CrewBudgetsError = "not_found" | "invalid_amount";
13
14 /** `amounts` holds the dollars typed for each limit; a key left out stays as it is. */
15 export function saveCrewBudgets(
16 app: App,
17 user: User | undefined,
18 amounts: Partial<Record<BudgetKey, number>>,
19 ): { ok: true; woken: number } | { ok: false; error: CrewBudgetsError } {
20 if (user === undefined || !user.isAdmin || user.kind === "agent") return { ok: false, error: "not_found" };
21 for (const key of BUDGET_KEYS) {
22 const n = amounts[key];
23 if (n !== undefined && !validBudget(n)) return { ok: false, error: "invalid_amount" };
24 }
25 let raised = false;
26 for (const key of BUDGET_KEYS) {
27 const n = amounts[key];
28 if (n === undefined) continue;
29 if (n > budgetLimit(app.db, key)) raised = true;
30 setBudget(app.db, key, n, user.id);
31 }
32 return { ok: true, woken: raised ? wakeParked(app, user, "limit") : 0 };
33 }
src/actions/crew_key.ts +4 −2
@@ -10,6 +10,7 @@ import type { App } from "../app.ts";
1010 import { addSubscription, getSubscriptionByName, setGrants } from "../core/subscriptions.ts";
1111 import type { User } from "../core/users.ts";
1212 import { spawnedOrg } from "../spawn/claim.ts";
13 import { wakeParked } from "../crew/wake.ts";
1314
1415 const KEY_NAME = "my-key";
1516 /** The most bytes a key may be; the longest real one is well under it. */
@@ -27,7 +28,7 @@ export async function saveCrewKey(
2728 app: App,
2829 user: User | undefined,
2930 raw: string,
30 ): Promise<{ ok: true } | { ok: false; error: CrewKeyError }> {
31 ): Promise<{ ok: true; woken: number } | { ok: false; error: CrewKeyError }> {
3132 if (user === undefined || !user.isAdmin || user.kind === "agent") return { ok: false, error: "not_found" };
3233 if (!keyShapeOk(raw)) return { ok: false, error: "invalid_token" };
3334 const name = getSubscriptionByName(app.db, KEY_NAME) === undefined ? KEY_NAME : `${KEY_NAME}-${Date.now().toString(36)}`;
@@ -41,5 +42,6 @@ export async function saveCrewKey(
4142 .map((r) => r.subscription_id);
4243 setGrants(app.db, scope, "", [...held, added.value.id]);
4344 }
44 return { ok: true };
45 // Issues parked for want of a key are tried again (`crew/wake.ts`).
46 return { ok: true, woken: wakeParked(app, user, "key") };
4547 }
src/crew/forge_budget.ts added +136
@@ -0,0 +1,136 @@
1 /**
2 * What a hosted forge's owner lets the crew spend (#11515): a limit per run,
3 * per UTC day and per UTC month, in dollars and cents, kept as forge settings
4 * and counted from the usage ledger. They bind only a forge that is somebody's
5 * own (a hosted child, `parent.json` in its data directory); the forge that
6 * hosts others sets its spend through the account ceilings instead. Pure
7 * data: who may set is the action's to decide.
8 */
9
10 import type { Database } from "bun:sqlite";
11 import { existsSync } from "node:fs";
12 import { join } from "node:path";
13 import type { App } from "../app.ts";
14 import { monthStart } from "../core/model_grant.ts";
15 import { dayStart } from "../core/account_ceiling.ts";
16 import { MIN_RUN_USD } from "../core/subscriptions.ts";
17 import { writeSetting } from "../settings/limits.ts";
18
19 export type BudgetKey = "run" | "day" | "month";
20 export const BUDGET_KEYS: readonly BudgetKey[] = ["run", "day", "month"];
21
22 /** Sensible for a key the owner pays for: a run is never cut off under a dollar. */
23 export const BUDGET_DEFAULT_USD: Record<BudgetKey, number> = { run: 5, day: 50, month: 500 };
24 export const BUDGET_MAX_USD = 100_000;
25 export const BUDGET_LABEL: Record<BudgetKey, string> = { run: "per run", day: "per day", month: "per month" };
26 /** Where the limits are set, and where a stop names them. */
27 export const BUDGETS_AT = "/settings/crew#budgets";
28
29 const storeKey = (key: BudgetKey): string => `crew.${key}_usd`;
30
31 /** Whole cents, above zero, within the maximum. */
32 export const validBudget = (usd: number): boolean =>
33 Number.isFinite(usd) && usd >= 0.01 && usd <= BUDGET_MAX_USD && Math.abs(usd * 100 - Math.round(usd * 100)) < 1e-6;
34
35 /** Whether this forge is somebody's own: its host's file is in its data directory. */
36 export const isOwnForge = (app: App): boolean => existsSync(join(app.config.dataDir, "parent.json"));
37
38 export function budgetLimit(db: Database, key: BudgetKey): number {
39 const row = db.query<{ value: string }, [string]>("SELECT value FROM forge_setting WHERE key = ?1").get(storeKey(key));
40 const n = row === null ? NaN : Number(row.value);
41 return validBudget(n) ? n : BUDGET_DEFAULT_USD[key];
42 }
43
44 /** Set one limit; false when the amount is not valid. */
45 export function setBudget(db: Database, key: BudgetKey, usd: number, actorId: string): boolean {
46 if (!validBudget(usd)) return false;
47 const from = budgetLimit(db, key);
48 const to = String(Math.round(usd * 100) / 100);
49 writeSetting(db, storeKey(key), to, String(from), to, actorId);
50 return true;
51 }
52
53 /** What the forge's agent runs have spent since `since`, from the usage ledger (`usage_at`). Conversations are not counted. */
54 export function spentSince(db: Database, since: string): number {
55 return db.query<{ n: number }, [string]>("SELECT COALESCE(SUM(cost_usd), 0) AS n FROM usage WHERE at >= ?1").get(since)?.n ?? 0;
56 }
57
58 export interface BudgetLine {
59 readonly key: BudgetKey;
60 readonly limitUsd: number;
61 /** Spent against it: nothing for the per-run limit, which is not a total. */
62 readonly spentUsd: number | undefined;
63 }
64
65 export function budgetLines(db: Database, now: Date): BudgetLine[] {
66 return [
67 { key: "run", limitUsd: budgetLimit(db, "run"), spentUsd: undefined },
68 { key: "day", limitUsd: budgetLimit(db, "day"), spentUsd: spentSince(db, dayStart(now)) },
69 { key: "month", limitUsd: budgetLimit(db, "month"), spentUsd: spentSince(db, monthStart(now)) },
70 ];
71 }
72
73 export type BudgetVerdict =
74 | { readonly ok: true; readonly capUsd: number; readonly by: BudgetKey | undefined }
75 | { readonly ok: false; readonly said: string; readonly limit: BudgetKey };
76
77 const dollars = (n: number): string => `$${n.toFixed(2)}`;
78
79 /** The words a run is stopped with for want of a key; the runner says them and `wake.ts` reads them. */
80 export const NO_KEY_SAID = "This forge has no model key: add yours at /settings/crew";
81 export const NO_ACCOUNT_SAID = "This repository has no model account: ask an admin to grant one";
82 /** A stop by one of the owner's limits, as `stoppedBy` writes it. */
83 const LIMIT_STOP = /This forge's per (run|day|month) limit \(\$[\d.]+\) stopped the run/;
84 export const isLimitStop = (reason: string): boolean => LIMIT_STOP.test(reason);
85
86 /** The words for a stop by `key`, naming it and where to change it. */
87 export function stoppedBy(db: Database, key: BudgetKey): string {
88 return `This forge's ${BUDGET_LABEL[key]} limit (${dollars(budgetLimit(db, key))}) stopped the run: raise it at ${BUDGETS_AT}`;
89 }
90
91 /** What a run's summary ends with when it was cut off at the cap and a limit of the owner's set that cap; nothing otherwise. */
92 export function cutOffBy(db: Database, own: BudgetVerdict | undefined, cutOff: boolean): string {
93 return cutOff && own?.ok === true && own.by !== undefined ? ` — ${stoppedBy(db, own.by)}` : "";
94 }
95
96 /** What the other runs queued or running have reserved of the day and month and not spent yet. */
97 function reservedByOthers(db: Database, runId: string): number {
98 return (
99 db
100 .query<{ n: number }, [string]>(
101 `SELECT COALESCE(SUM(MAX(0, r.ceiling_reserved_usd -
102 (SELECT COALESCE(SUM(u.cost_usd), 0) FROM usage u WHERE u.run_id = r.id))), 0) AS n
103 FROM run r WHERE r.state IN ('queued', 'running') AND r.id <> ?1 AND r.ceiling_reserved_usd IS NOT NULL`,
104 )
105 .get(runId)?.n ?? 0
106 );
107 }
108
109 /**
110 * May a run start under the owner's limits, and with what cap. Undecided on
111 * a forge that hosts others. With `runId`, the cap is reserved for that run
112 * in the same transaction, and runs under way count against the day and the
113 * month, so agents that start together share what is left rather than each
114 * being handed all of it.
115 */
116 export function budgetVerdict(app: App, wantUsd: number, now: Date, runId?: string): BudgetVerdict | undefined {
117 if (!isOwnForge(app)) return undefined;
118 return app.db.transaction((): BudgetVerdict => verdict(app, wantUsd, now, runId))();
119 }
120
121 function verdict(app: App, wantUsd: number, now: Date, runId: string | undefined): BudgetVerdict {
122 const lines = budgetLines(app.db, now);
123 const others = runId === undefined ? 0 : reservedByOthers(app.db, runId);
124 const run = lines[0]!;
125 if (run.limitUsd < MIN_RUN_USD) return { ok: false, limit: "run", said: stoppedBy(app.db, "run") };
126 let cap = Math.min(wantUsd, run.limitUsd);
127 let by: BudgetKey | undefined = run.limitUsd < wantUsd ? "run" : undefined;
128 for (const line of lines.slice(1)) {
129 const left = line.limitUsd - (line.spentUsd ?? 0) - others;
130 if (left < MIN_RUN_USD) return { ok: false, limit: line.key, said: stoppedBy(app.db, line.key) };
131 if (left < cap) by = line.key;
132 cap = Math.min(cap, left);
133 }
134 if (runId !== undefined) app.db.query("UPDATE run SET ceiling_reserved_usd = ?1 WHERE id = ?2").run(cap, runId);
135 return { ok: true, capUsd: cap, by };
136 }
src/crew/runner.ts +25 −7
@@ -11,7 +11,6 @@
1111 * what the forge has.
1212 */
1313
14 import type { Database } from "bun:sqlite";
1514 import { isStrangerPull } from "./stranger.ts";
1615 import { noteMeasure } from "../core/measure.ts";
1716 import { capacityEnv } from "../core/capacity.ts";
@@ -156,6 +155,7 @@ import { laterGate } from "./later.ts";
156155 import { MERGES_CLEAN_NOW, mergesCleanNow } from "./clean_now.ts";
157156 import { handedLimbs } from "./handed_limbs.ts";
158157 import { parkedTwice } from "./parks.ts";
158 import { budgetLimit, budgetVerdict, cutOffBy, isOwnForge, NO_ACCOUNT_SAID, NO_KEY_SAID } from "./forge_budget.ts";
159159 import { boxesOf, landedBoxes } from "./box_pass.ts";
160160 import { reviewRange } from "./review_range.ts";
161161 import { heldNow, stuckOf } from "./held.ts";
@@ -269,13 +269,18 @@ export function steerRun(app: App, runId: string, text: string): boolean {
269269 }
270270
271271 /** What a refused run says: an org is named with the page where an admin grants it; a person keeps the plain words. */
272 function noGrantSaid(db: Database, ownerId: string, ownerName: string): string {
272 function noGrantSaid(app: App, ownerId: string, ownerName: string): string {
273 const db = app.db;
274 // A forge that is its owner's own, or a repository its admin owns, has no
275 // one else to ask: the key is theirs to add (#11515).
276 if (isOwnForge(app) || getUserById(db, ownerId)?.isAdmin === true)
277 return NO_KEY_SAID;
273278 const org = db
274279 .query<{ kind: string }, [string]>("SELECT kind FROM user WHERE id = ?1")
275280 .get(ownerId)?.kind === "org";
276281 return org
277282 ? `the account \`${ownerName}\` has no model grant; an admin can grant one at /settings/accounts/${ownerName}`
278 : "This repository has no model account: ask an admin to grant one";
283 : NO_ACCOUNT_SAID;
279284 }
280285
281286 /**
@@ -1332,7 +1337,12 @@ async function work(
13321337 };
13331338 // What this run may spend at most; on a budgeted account the pick
13341339 // reserves it (or what is left, if less) against the account's budget.
1335 const ownBudget = agent.budgetUsd > 0 ? agent.budgetUsd : options.budgetUsd;
1340 const ownBudget =
1341 agent.budgetUsd > 0
1342 ? agent.budgetUsd
1343 : isOwnForge(app)
1344 ? budgetLimit(app.db, "run")
1345 : options.budgetUsd;
13361346 const picked = pickSubscription(
13371347 app.db,
13381348 app.config.dataDir,
@@ -1373,7 +1383,7 @@ async function work(
13731383 ) {
13741384 const said =
13751385 grantCap === undefined
1376 ? noGrantSaid(app.db, repo.ownerId, repo.ownerName)
1386 ? noGrantSaid(app, repo.ownerId, repo.ownerName)
13771387 : "This month's model grant is spent: an admin can raise it";
13781388 write(said);
13791389 return { state: "failed", summary: said };
@@ -1401,6 +1411,12 @@ async function work(
14011411 write(ceiling.said);
14021412 return { state: "failed", summary: ceiling.said };
14031413 }
1414 // The owner's own limits on a forge that is theirs (`forge_budget.ts`).
1415 const own = budgetVerdict(app, ceiling.capUsd ?? wanted, new Date(), run.id);
1416 if (own !== undefined && !own.ok) {
1417 write(own.said);
1418 return { state: "failed", summary: own.said };
1419 }
14041420 meter.ceilingOwner = repo.ownerId;
14051421 if (picked.kind === "ready") {
14061422 write(`subscription: ${picked.subscription.name}`);
@@ -1437,7 +1453,7 @@ async function work(
14371453 // The budget in two parts: the work, and a reserve for landing it. An
14381454 // agent cut off mid-task otherwise leaves a tree the forge commits blind
14391455 // and a pull whose description is the error.
1440 const budget = Math.min(ceiling.capUsd ?? Infinity, wanted);
1456 const budget = Math.min(ceiling.capUsd ?? Infinity, wanted, own?.ok === true ? own.capUsd : Infinity);
14411457 const reserve = wrapUpReserve(budget);
14421458 // Asked again for every invocation (`crew/sandbox.ts`): an agent made
14431459 // since the last one is denied from this one on.
@@ -1876,7 +1892,9 @@ async function work(
18761892 state: result.ok || landed.pushed ? "ok" : "failed",
18771893 summary: outage
18781894 ? `network unreachable after ${MAX_PAUSES} pauses; nothing to land (${result.reason})`
1879 : landed.summary + (result.ok ? "" : ` (${result.reason})`),
1895 : landed.summary +
1896 (result.ok ? "" : ` (${result.reason})`) +
1897 cutOffBy(app.db, own, stoppedAt !== undefined),
18801898 resumesAt: limitLifts,
18811899 sessionId: result.sessionId,
18821900 costUsd: result.costUsd,
src/crew/wake.ts added +72
@@ -0,0 +1,72 @@
1 /**
2 * Waking the work (#11515). A run that could not start for want of a model
3 * key, or that a spend limit stopped, parks its issue; the owner then adds a
4 * key or raises a limit, and the issue should be tried again without anyone
5 * asking. A person's word on an issue starts its strike count from zero
6 * (`parks.ts`), so waking is that word: one comment, in the acting admin's
7 * name, on each issue whose newest park since the last word was for the
8 * means. Once per park: the comment is the word, so a second wake finds
9 * nothing left to wake.
10 */
11
12 import type { App } from "../app.ts";
13 import { addComment } from "../actions/issues.ts";
14 import { PARK_LIKE } from "../core/done_on_main.ts";
15 import { getIssueById } from "../core/issues.ts";
16 import { getRepoById } from "../core/repos.ts";
17 import type { User } from "../core/users.ts";
18 import { isLimitStop, NO_ACCOUNT_SAID, NO_KEY_SAID } from "./forge_budget.ts";
19 import { parkReasons } from "./parks.ts";
20
21 /** Why the issues were woken, for the comment they get. */
22 export type WakeCause = "key" | "limit";
23
24 const WORDS: Record<WakeCause, string> = {
25 key: "A model key was added",
26 limit: "A spend limit was raised",
27 };
28
29 /** Park reasons a new key can fix: the exact words the runner writes for want of a key, an account or a grant. */
30 const NO_GRANT = /the account `[^`]+` has no model grant; an admin can grant one/;
31 const MEANS: Record<WakeCause, (reason: string) => boolean> = {
32 key: (r) => r.includes(NO_KEY_SAID) || r.includes(NO_ACCOUNT_SAID) || NO_GRANT.test(r),
33 limit: isLimitStop,
34 };
35
36 /** What a wake looks at, at most: issues carrying a park for the cause, newest first. A request stays small. */
37 export const WAKE_MAX = 200;
38
39 /** SQL pre-filter on the park text, so only issues that have such a park are read. */
40 const CANDIDATES: Record<WakeCause, string> = {
41 key: "(c.body LIKE '%has no model key%' OR c.body LIKE '%has no model account%' OR c.body LIKE '%has no model grant%')",
42 limit: "c.body LIKE '%limit ($%stopped the run%'",
43 };
44
45 /** Un-park every issue parked for the owner's means; how many were picked back up. */
46 export function wakeParked(app: App, by: User, cause: WakeCause): number {
47 const rows = app.db
48 .query<{ id: string }, [number]>(
49 `SELECT i.id, MAX(c.created_at) AS at
50 FROM comment c JOIN issue i ON i.id = c.issue_id JOIN repo r ON r.id = i.repo_id
51 WHERE i.state = 'open' AND i.kind = 'issue' AND r.autonomy = 'autonomous'
52 AND c.body LIKE '${PARK_LIKE}' AND ${CANDIDATES[cause]}
53 GROUP BY i.id ORDER BY at DESC LIMIT ?1`,
54 )
55 .all(WAKE_MAX);
56 let woken = 0;
57 for (const { id } of rows) {
58 const issue = getIssueById(app.db, id);
59 const repo = issue === undefined ? undefined : getRepoById(app.db, issue.repoId);
60 if (issue === undefined || repo === undefined) continue;
61 const reasons = parkReasons(app.db, repo, issue.id);
62 const last = reasons[reasons.length - 1];
63 if (last === undefined || !MEANS[cause](last)) continue;
64 // A sealed issue's comment is seen only where the issue is (`core/sealed.ts`).
65 const said = addComment(app, by, repo, issue, `${WORDS[cause]}, so the crew picks this back up and tries it again.`);
66 if (said.ok) woken += 1;
67 }
68 return woken;
69 }
70
71 /** The owner's line after a wake: nothing when nothing woke. */
72 export const wokeWords = (n: number): string => (n === 0 ? "" : `${n} ${n === 1 ? "issue" : "issues"} picked back up.`);
src/http/settings_routes.ts +2
@@ -8,6 +8,7 @@
88 */
99
1010 import { crewKeyAdd } from "../web/studio/crew_key.ts";
11 import { crewBudgetsSave } from "../web/studio/crew_budgets.ts";
1112 import type { Handler } from "./context.ts";
1213 import type { Router } from "./router.ts";
1314 import { parentLinkStart } from "./rp.ts";
@@ -177,6 +178,7 @@ export function settingsRoutes(router: Router<Handler>, knownPatterns: () => rea
177178 router.add("GET", "/settings/subscriptions", subscriptionsPage);
178179 router.add("POST", "/settings/subscriptions", subscriptionAdd);
179180 router.add("POST", "/settings/crew/key", crewKeyAdd);
181 router.add("POST", "/settings/crew/budgets", crewBudgetsSave);
180182 router.add("POST", "/settings/subscriptions/:id/delete", subscriptionDelete);
181183 router.add("POST", "/settings/subscriptions/:id/wake", subscriptionWake);
182184 router.add(
src/tui/screens/crewbudgets.ts added +92
@@ -0,0 +1,92 @@
1 /**
2 * The crew's spend limits in the terminal (#11515), the second door of
3 * `POST /settings/crew/budgets`: per run, per day and per month, with what
4 * was spent against the day and the month. j/k choose a limit, enter types
5 * its dollars; saved through the same `saveCrewBudgets` the page uses, so a
6 * raised limit wakes the parked work here too. A forge admin who is a
7 * person, on a forge that is its owner's own; anyone else sees nothing.
8 */
9
10 import { saveCrewBudgets } from "../../actions/crew_budgets.ts";
11 import { BUDGET_KEYS, BUDGET_LABEL, BUDGET_MAX_USD, budgetLines, isOwnForge } from "../../crew/forge_budget.ts";
12 import { wokeWords } from "../../crew/wake.ts";
13 import type { Ctx, Screen } from "../app/app.ts";
14 import { binding } from "../app/keys.ts";
15 import type { Session } from "../session.ts";
16 import { style } from "../term/style.ts";
17 import { drawLines, promptScreen, Scroll } from "./common.ts";
18 import type { PaletteEntry } from "./entries.ts";
19
20 const mayAct = (f: Session): boolean => f.user?.isAdmin === true && f.user.kind === "user" && isOwnForge(f.app);
21
22 const usd = (n: number): string => `$${n.toFixed(2)}`;
23 const AMOUNT = new RegExp(`^\\d{1,${String(BUDGET_MAX_USD).length}}(\\.\\d{1,2})?$`);
24 const BAD_AMOUNT = `Dollars and cents, from $0.01 to $${BUDGET_MAX_USD.toLocaleString("en-US")}.`;
25
26 /** One limit a line, the chosen one marked, with what is spent against it; empty for anyone the gate refuses. */
27 export function budgetRows(f: Session, chosen: number): string[] {
28 if (!mayAct(f)) return [];
29 return budgetLines(f.app.db, new Date()).map((l, i) => {
30 const spent = l.spentUsd === undefined ? "" : ` (${usd(l.spentUsd)} spent this ${l.key === "day" ? "day" : "month"}, UTC)`;
31 return `${i === chosen ? ">" : " "} Spend ${BUDGET_LABEL[l.key]}: ${usd(l.limitUsd)}${spent}`;
32 });
33 }
34
35 /** Set the chosen limit from what was typed: what it says, and whether it was saved. Nothing is saved for anyone the gate refuses. */
36 export function setChosen(f: Session, chosen: number, raw: string): { said: string; saved: boolean } {
37 const key = BUDGET_KEYS[chosen];
38 if (!mayAct(f) || key === undefined) return { said: "Only a forge admin sets a budget.", saved: false };
39 const text = raw.trim();
40 if (!AMOUNT.test(text)) return { said: BAD_AMOUNT, saved: false };
41 const saved = saveCrewBudgets(f.app, f.user, { [key]: Number(text) });
42 if (!saved.ok) return { said: BAD_AMOUNT, saved: false };
43 return { said: `Spend ${BUDGET_LABEL[key]} is ${usd(Number(text))}. ${wokeWords(saved.woken)}`.trim(), saved: true };
44 }
45
46 export function crewBudgetsScreen(f: Session, ctx: Ctx): Screen {
47 let chosen = 0;
48 let note = "";
49 const screen: Screen = {
50 title: "Crew budgets",
51 keys: [binding(["j", "k"], "choose"), binding(["enter"], "set"), binding(["esc", "q"], "back")],
52 render(area, s) {
53 const c = ctx.theme.c;
54 const lines = [...budgetRows(f, chosen), ...(note === "" ? [] : ["", note])];
55 drawLines(s, area, c, new Scroll(), lines.map((text) => [{ text, style: style(c.fg, c.bg) }]));
56 },
57 key(e) {
58 if (e.ctrl || e.alt) return true;
59 if (e.key === "esc" || e.key === "q") ctx.pop();
60 else if (e.key === "j" || e.key === "down") chosen = Math.min(chosen + 1, BUDGET_KEYS.length - 1);
61 else if (e.key === "k" || e.key === "up") chosen = Math.max(chosen - 1, 0);
62 else if (e.key === "enter" && mayAct(f))
63 ctx.push(
64 promptScreen(ctx, screen, {
65 title: `Spend ${BUDGET_LABEL[BUDGET_KEYS[chosen]!]} (dollars)`,
66 label: "Dollars and cents",
67 onSubmit: (value) => {
68 const r = setChosen(f, chosen, value);
69 if (!r.saved) return r.said;
70 note = r.said;
71 ctx.invalidate();
72 return undefined;
73 },
74 }),
75 );
76 else return false;
77 ctx.invalidate();
78 return true;
79 },
80 };
81 return screen;
82 }
83
84 /** The crew's spend limits: per run, per day, per month. */
85 export const entry: PaletteEntry = {
86 id: "crewbudgets",
87 title: "Crew budgets",
88 hint: "what the crew may spend per run, day and month on your key",
89 after: ["crewkey", "subscriptions"],
90 offered: mayAct,
91 run: (session, ctx) => ctx.push(crewBudgetsScreen(session, ctx)),
92 };
src/tui/screens/entries.ts +2
@@ -89,6 +89,7 @@ import { entry as secrets } from "./secrets.ts";
8989 import { entry as sentinel } from "./sentinel.ts";
9090 import { entry as subscriptions } from "./subscriptions.ts";
9191 import { entry as crewKey } from "./crewkey.ts";
92 import { entry as crewBudgets } from "./crewbudgets.ts";
9293 import { entry as system } from "./system.ts";
9394 import { entry as teams } from "./teams.ts";
9495 import { entry as telemetry } from "./telemetry.ts";
@@ -173,6 +174,7 @@ export const ENTRIES: readonly PaletteEntry[] = [
173174 bulk,
174175 subscriptions,
175176 crewKey,
177 crewBudgets,
176178 cloud,
177179 pinned,
178180 bets,
src/web/pages/crew.ts +2
@@ -6,6 +6,7 @@
66 * the crew is actually there, in words, before anyone assigns anything.
77 */
88
9 import { budgetsCard } from "../studio/crew_budgets.ts";
910 import { fundingOf, fundingWords } from "../../spawn/funding.ts";
1011 import {
1112 createAgent,
@@ -661,6 +662,7 @@ function crewView(
661662 ["Scorecard", `Over the pulls each agent opened, and each team's members pooled (an agent on two teams counts in both), in the last ${RECENT_DAYS} days and over all time: review rounds are change requests per landed pull, red checks the share of finished checks that failed; reverts (the landed pulls a merged pull took back) and time to land (the mean from opening to merge) are all time.`],
662663 ["Model review", `At most one change of model per agent a week, to the cheapest that holds its quality. A step up always asks; a step down ${review.limits.auto ? "applies on its own" : "asks too"} (Settings → Limits). Under ${review.limits.minLanded} landed changes nothing. A week after a change the result is reported, and the change undone if quality fell. An ask nobody answers lapses after two weeks. Weekly, from the last ${review.windowDays} days of the ledger.`],
663664 ], helpAsked(ctx.request.url))}
665 ${budgetsCard(ctx)}
664666 ${knobsCard(ctx)}
665667 ${when(spending.length > 0, () =>
666668 spendingCard(spending, forgeCeiling(ctx.app.config) ?? undefined),
src/web/studio/crew_budgets.ts added +68
@@ -0,0 +1,68 @@
1 /**
2 * Settings → Crew → Budgets (#11515): the three numbers a forge's owner
3 * changes, with what was spent against the day and the month, and the POST
4 * that saves them. Shown on a forge that is its owner's own; the forge that
5 * hosts others sets spend through account ceilings.
6 */
7
8 import { saveCrewBudgets } from "../../actions/crew_budgets.ts";
9 import { BUDGET_KEYS, BUDGET_LABEL, BUDGET_MAX_USD, budgetLines, isOwnForge, type BudgetKey } from "../../crew/forge_budget.ts";
10 import { wokeWords } from "../../crew/wake.ts";
11 import type { Ctx } from "../../http/context.ts";
12 import { notFoundPage, signInFirst } from "../chrome.ts";
13 import { field, readForm } from "../form.ts";
14 import { html, type Html } from "../html.ts";
15 import { flashTo } from "../layout.ts";
16 import { btn } from "../ui.ts";
17
18 export const BUDGETS_ACTION = "/settings/crew/budgets";
19 const usd = (n: number): string => `$${n.toFixed(2)}`;
20 /** Digits as wide as the maximum, and at most cents. */
21 const AMOUNT = new RegExp(`^\\d{1,${String(BUDGET_MAX_USD).length}}(\\.\\d{1,2})?$`);
22
23 const fact = (label: string, value: string): Html => html`<div class="status-row"><dt>${label}</dt><dd>${value}</dd></div>`;
24
25 /** The card: one field per limit, what has been spent against the day and the month, one save. Nothing on a forge that hosts others. */
26 export function budgetsCard(ctx: Ctx): Html | null {
27 if (!isOwnForge(ctx.app)) return null;
28 const lines = budgetLines(ctx.app.db, new Date());
29 return html`<section class="settings-block" id="budgets">
30 <h2>Budgets</h2>
31 <p class="small muted">What the crew may spend on your model key; a run a limit stops says which one.</p>
32 <form class="stack" method="post" action="${BUDGETS_ACTION}">
33 ${lines.map(
34 (l) => html`<div class="field">
35 <label for="budget-${l.key}">Spend ${BUDGET_LABEL[l.key]} (dollars)</label>
36 <input class="input" id="budget-${l.key}" name="${l.key}" type="number" min="0.01" max="${BUDGET_MAX_USD}" step="0.01" required value="${l.limitUsd.toFixed(2)}" inputmode="decimal" />
37 </div>`,
38 )}
39 <div>${btn("Save budgets", { variant: "primary", size: "sm" })}</div>
40 </form>
41 <dl class="status-rows">${lines.map((l) =>
42 l.spentUsd === undefined ? null : fact(`Spent this ${l.key === "day" ? "day (UTC)" : "month (UTC)"}`, `${usd(l.spentUsd)} of ${usd(l.limitUsd)}`),
43 )}</dl>
44 </section>`;
45 }
46
47 // POST /settings/crew/budgets
48 export async function crewBudgetsSave(ctx: Ctx): Promise<Response> {
49 const user = ctx.auth.user;
50 if (!user) return signInFirst(ctx);
51 if (!user.isAdmin) return notFoundPage(ctx);
52 const form = await readForm(ctx, "admin", 1024);
53 if (form instanceof Response) return form;
54 const amounts: Partial<Record<BudgetKey, number>> = {};
55 for (const key of BUDGET_KEYS) {
56 const raw = field(form, key).trim();
57 if (raw !== "") amounts[key] = AMOUNT.test(raw) ? Number(raw) : NaN;
58 }
59 const saved = saveCrewBudgets(ctx.app, user, amounts);
60 if (!saved.ok)
61 return saved.error === "not_found"
62 ? notFoundPage(ctx)
63 : flashTo(ctx, "/settings/crew#budgets", {
64 tone: "danger",
65 message: `Each limit is dollars and cents, from $0.01 to $${BUDGET_MAX_USD.toLocaleString("en-US")}.`,
66 });
67 return flashTo(ctx, "/settings/crew#budgets", { message: `Budgets saved. ${wokeWords(saved.woken)}`.trim() });
68 }
src/web/studio/crew_key.ts +14 −3
@@ -12,6 +12,7 @@ import { join } from "node:path";
1212 import type { App } from "../../app.ts";
1313 import { listAgents } from "../../core/agents.ts";
1414 import { saveCrewKey } from "../../actions/crew_key.ts";
15 import { wokeWords } from "../../crew/wake.ts";
1516 import { crewHasCredential } from "../../core/subscriptions.ts";
1617 import { leftUsd, usdWords } from "../../spawn/credit.ts";
1718 import { fundingOf } from "../../spawn/funding.ts";
@@ -52,9 +53,13 @@ export function crewCanRun(app: App): boolean {
5253 /** Whether the forge has a crew at all, so a line about it is true. */
5354 export const hasCrew = (app: App): boolean => listAgents(app.db).length > 0;
5455
56 /** Said once where a forge of its own runs on its own key. */
57 const OWN_KEY_NOTE = "The crew works on this forge's own model key; the host's included credit is not drawn on.";
58
5559 /** What the crew runs on, in one sentence and no field: the first screen's line, said the same on the page and in the terminal. */
5660 export function crewNote(app: App): string {
57 if (crewHasCredential(app.db)) return "The crew works on this forge's included credit.";
61 if (crewHasCredential(app.db))
62 return isHostedForge(app) ? OWN_KEY_NOTE : "The crew works on this forge's own model key.";
5863 const credit = includedCredit(app);
5964 if (credit !== undefined && credit.leftUsd > 0)
6065 return `The crew is building on ${usdWords(credit.leftUsd)} of ${usdWords(credit.capUsd)} included credit.`;
@@ -85,10 +90,16 @@ const keyField = (): Html => html`<form class="stack" method="post" action="${KE
8590 <div>${btn("Save key", { variant: "primary", size: "sm", icon: "key" })}</div>
8691 </form>`;
8792
93 /** On a forge of its own that has its own key: which one is paying, so nobody wonders whether the host's credit is drawn on. */
94 function payingCard(app: App, compact: boolean): Html | null {
95 if (compact || !isHostedForge(app)) return null;
96 return html`<p class="small muted" id="crew-key-note" role="status">${icon("key")} ${OWN_KEY_NOTE}</p>`;
97 }
98
8899 /** What the crew runs on, said once, with the field when a key would change anything (`compact`: a line and a link, no field). */
89100 export function crewKeyCard(ctx: Ctx, compact = false): Html | null {
90101 // Credit left is something to run on: the card never asks for a key then, in any form (#11508).
91 if (crewHasCredential(ctx.app.db)) return null;
102 if (crewHasCredential(ctx.app.db)) return payingCard(ctx.app, compact);
92103 const admin = ctx.auth.user?.isAdmin === true;
93104 // On a repository page: one line and a link, the field lives on the home page.
94105 if (compact) {
@@ -125,5 +136,5 @@ export async function crewKeyAdd(ctx: Ctx): Promise<Response> {
125136 return saved.error === "not_found"
126137 ? notFoundPage(ctx)
127138 : flashTo(ctx, "/", { tone: "danger", message: "That key could not be saved. Check that it is the whole key and try again." });
128 return flashTo(ctx, "/", { message: "Key saved. The crew can work now." });
139 return flashTo(ctx, "/", { message: `Key saved. The crew can work now. ${wokeWords(saved.woken)}`.trim() });
129140 }
test/crew-budgets-page.test.ts added +104
@@ -0,0 +1,104 @@
1 /**
2 * Settings → Crew → Budgets (#11515): the card on a forge of its own, the
3 * POST that saves the three limits (admin only, same-origin, validated), and
4 * the home's line saying whose key is paying.
5 */
6 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
7 import { mkdtemp, rm } from "node:fs/promises";
8 import { writeFileSync } from "node:fs";
9 import { tmpdir } from "node:os";
10 import { join } from "node:path";
11 import { saveCrewKey } from "../src/actions/crew_key.ts";
12 import { createApp, type App } from "../src/app.ts";
13 import { budgetLimit } from "../src/crew/forge_budget.ts";
14 import { createSession } from "../src/core/sessions.ts";
15 import { createUser, getUserByName } from "../src/core/users.ts";
16 import { serve, type Servers } from "../src/http/server.ts";
17 import { crewFirstText, crewNote } from "../src/web/studio/crew_key.ts";
18 import { quietLogs } from "./helpers/globals.ts";
19
20 quietLogs();
21
22 describe("the owner's budgets page", () => {
23 let dir = "";
24 let app: App;
25 let servers: Servers;
26 let admin = "";
27 let plain = "";
28 const post = (cookie: string, body: Record<string, string>, origin = servers.url) =>
29 fetch(`${servers.url}/settings/crew/budgets`, {
30 method: "POST",
31 redirect: "manual",
32 headers: { "content-type": "application/x-www-form-urlencoded", origin, cookie },
33 body: new URLSearchParams(body),
34 });
35
36 beforeAll(async () => {
37 dir = await mkdtemp(join(tmpdir(), "forge-budgets-"));
38 writeFileSync(join(dir, "parent.json"), "{}");
39 app = createApp({ dataDir: dir, port: 0 });
40 for (const [username, adminIfFirst] of [["alice", true], ["bob", false]] as const) {
41 const made = await createUser(app.db, { origin: "seed", trusted: true, username, email: `${username}@x.test`, password: "a-long-starter-passphrase", adminIfFirst });
42 if (!made.ok) throw new Error(made.error);
43 }
44 admin = `forge_session=${createSession(app.db, getUserByName(app.db, "alice")!.id).token}`;
45 plain = `forge_session=${createSession(app.db, getUserByName(app.db, "bob")!.id).token}`;
46 servers = await serve(app);
47 });
48 afterAll(async () => {
49 servers?.stop();
50 app?.close();
51 await rm(dir, { recursive: true, force: true });
52 });
53
54 test("the page shows three limits with their defaults and what is spent", async () => {
55 const res = await fetch(`${servers.url}/settings/crew`, { headers: { cookie: admin } });
56 const page = await res.text();
57 expect(page).toContain('id="budgets"');
58 expect(page).toContain('name="run" type="number"');
59 expect(page).toContain('value="5.00"');
60 expect(page).toContain('value="50.00"');
61 expect(page).toContain('value="500.00"');
62 expect(page).toContain("$0.00 of $50.00");
63 expect(page).toContain("Spent this day (UTC)");
64 expect(page).toContain(`max="100000"`);
65 });
66
67 test("an admin saves them; a bad amount, a stranger and a cross-origin post change nothing", async () => {
68 const saved = await post(admin, { run: "12.50", day: "40", month: "300" });
69 expect(saved.status).toBeLessThan(400);
70 expect([budgetLimit(app.db, "run"), budgetLimit(app.db, "day"), budgetLimit(app.db, "month")]).toEqual([12.5, 40, 300]);
71 await post(admin, { run: "abc", day: "1", month: "1" });
72 await post(plain, { run: "1" });
73 expect((await post(admin, { run: "1" }, "https://evil.example")).status).toBe(403);
74 expect(budgetLimit(app.db, "run")).toBe(12.5);
75 expect(budgetLimit(app.db, "day")).toBe(40);
76 });
77
78 test("a post that raises a limit says the parked work was picked back up", async () => {
79 const { addComment, openIssue } = await import("../src/actions/issues.ts");
80 const { createRepo } = await import("../src/actions/repos.ts");
81 const { PARK_NOTE } = await import("../src/crew/parks.ts");
82 const { stoppedBy, setBudget } = await import("../src/crew/forge_budget.ts");
83 const alice = getUserByName(app.db, "alice")!;
84 const made = await createRepo(app, alice, { name: "beds", visibility: "public" });
85 if (!made.ok) throw new Error("repo");
86 app.db.query("UPDATE repo SET autonomy = 'autonomous' WHERE id = ?1").run(made.value.id);
87 setBudget(app.db, "run", 0.1, alice.id);
88 const issue = openIssue(app, alice, made.value, { title: "Stopped by a limit", body: "x" });
89 if (!issue.ok) throw new Error("issue");
90 addComment(app, alice, made.value, issue.value, `${PARK_NOTE}${stoppedBy(app.db, "run")}`);
91 const res = await post(admin, { run: "8" });
92 expect(res.status).toBeLessThan(400);
93 const flash = res.headers.getSetCookie().map((c) => c.split(";")[0]).join("; ");
94 const page = await (await fetch(new URL(res.headers.get("location")!, servers.url), { headers: { cookie: `${admin}; ${flash}` } })).text();
95 expect(page).toContain("1 issue picked back up.");
96 });
97
98 test("with its own key the home line says whose key is paying, and the host's credit is not drawn on", async () => {
99 expect(crewNote(app)).not.toContain("own model key");
100 expect((await saveCrewKey(app, { ...getUserByName(app.db, "alice")!, isAdmin: true }, "sk-ant-api03-abcdefghijklmnop")).ok).toBe(true);
101 expect(crewNote(app)).toBe("The crew works on this forge's own model key; the host's included credit is not drawn on.");
102 expect(crewFirstText(app)).toContain("not drawn on");
103 });
104 });
test/crew-own-forge.test.ts added +123
@@ -0,0 +1,123 @@
1 /**
2 * A forge that is its owner's own (#11515): with no key the crew stops with
3 * the add-a-key words and parks the issue; adding a key, or raising a limit,
4 * picks the issue back up once; the host is never asked for model access.
5 */
6
7 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
8 import { existsSync, rmSync, writeFileSync } from "node:fs";
9 import { join } from "node:path";
10 import type { App } from "../src/app.ts";
11 import type { Repo } from "../src/core/repos.ts";
12 import type { User } from "../src/core/users.ts";
13 import { saveCrewBudgets } from "../src/actions/crew_budgets.ts";
14 import { saveCrewKey } from "../src/actions/crew_key.ts";
15 import { assignIssue, openIssue } from "../src/actions/issues.ts";
16 import { updateRepoSettings } from "../src/actions/repos.ts";
17 import { listRuns, type Run } from "../src/core/runs.ts";
18 import { PLATFORM } from "../src/core/subscriptions.ts";
19 import { budgetLimit, setBudget } from "../src/crew/forge_budget.ts";
20 import { parkedCount } from "../src/crew/parks.ts";
21 import { wokeWords } from "../src/crew/wake.ts";
22 import { readRunLog } from "../src/runs/log.ts";
23 import { promote } from "./helpers/members.ts";
24 import { agentRunSettled, startStandInCrew, stopStandInCrew, type StandInCrew } from "./helpers/crew-standin.ts";
25
26 describe("a forge of its own: its key, its limits, and the work picked back up", () => {
27 let standIn: StandInCrew;
28 let app: App;
29 let alice: User;
30 let bot: User;
31 let repo: Repo;
32
33 beforeAll(async () => {
34 standIn = await startStandInCrew();
35 ({ app, alice, bot, repo } = standIn);
36 // A hosted child: its host's file is in the data directory. No host grant, no key, no env.
37 writeFileSync(join(standIn.dataDir, "parent.json"), "{}");
38 delete process.env.ANTHROPIC_API_KEY;
39 const on = await updateRepoSettings(app, alice, repo, {
40 autonomy: "autonomous",
41 autonomyDirection: "Keep the garden small.",
42 autonomyCeilingUsd: 100,
43 autonomyBy: alice.id,
44 });
45 expect(on.ok).toBe(true);
46 // Not an admin of the forge until the key is added: the grant path has nothing for it.
47 app.db.query("UPDATE user SET is_admin = 0 WHERE id = ?1").run(alice.id);
48 promote(app.db, alice);
49 }, 30_000);
50 afterAll(() => stopStandInCrew(standIn));
51
52 const runsOf = (issueId: string): Run[] => listRuns(app.db, repo.id, { kind: "agent", limit: 50 }).filter((r) => r.issueId === issueId);
53
54 async function settledNth(issueId: string, n: number): Promise<Run> {
55 for (let i = 0; i < 700; i++) {
56 const done = runsOf(issueId).filter((r) => r.state === "ok" || r.state === "failed");
57 if (done.length >= n) return done.sort((a, b) => a.createdAt.localeCompare(b.createdAt))[n - 1]!;
58 await Bun.sleep(100);
59 }
60 throw new Error(`run ${n} did not settle`);
61 }
62
63 test("no key: the run stops with the add-a-key words and the issue is parked; a key picks it back up", async () => {
64 const issue = openIssue(app, alice, repo, { title: "Prune the roses", body: "Before spring." });
65 if (!issue.ok) throw new Error(String(issue.error));
66 expect(assignIssue(app, alice, repo, issue.value, bot).ok).toBe(true);
67 const first = await agentRunSettled(app, repo, (r) => r.issueId === issue.value.id, 60_000);
68 expect(first.state).toBe("failed");
69 expect(first.summary).toContain("This forge has no model key");
70 expect(first.summary).not.toContain("ask an admin");
71 expect(existsSync(standIn.argsFile)).toBe(false);
72 for (let i = 0; i < 100 && parkedCount(app.db, repo, issue.value.id) === 0; i++) await Bun.sleep(100);
73 expect(parkedCount(app.db, repo, issue.value.id)).toBe(1);
74
75 app.db.query("UPDATE user SET is_admin = 1 WHERE id = ?1").run(alice.id);
76 const admin = { ...alice, isAdmin: true };
77 const saved = await saveCrewKey(app, admin, "sk-ant-api03-ownerskey");
78 expect(saved).toEqual({ ok: true, woken: 1 });
79 expect(wokeWords(1)).toBe("1 issue picked back up.");
80 expect(parkedCount(app.db, repo, issue.value.id)).toBe(0);
81
82 // Within a minute a second run starts, on the owner's key and never the host's.
83 const second = await settledNth(issue.value.id, 2);
84 expect(existsSync(standIn.argsFile)).toBe(true);
85 const log = readRunLog(standIn.dataDir, second.id).lines.join("\n");
86 expect(log).toContain("subscription: my-key");
87 expect(log).not.toContain("platform");
88 // The forge's spend never appears against the host's credit.
89 const hostRows = app.db.query<{ n: number }, [string]>("SELECT COUNT(*) AS n FROM usage WHERE subscription_id = ?1").get(PLATFORM);
90 expect(hostRows?.n).toBe(0);
91 expect(existsSync(join(standIn.dataDir, "funding.json"))).toBe(false);
92 }, 120_000);
93
94 test("a per-run limit of $0.10 stops a run naming it; raising it runs the issue again", async () => {
95 rmSync(standIn.argsFile, { force: true });
96 const admin = { ...alice, isAdmin: true };
97 expect(saveCrewBudgets(app, admin, { run: 0.1 })).toEqual({ ok: true, woken: 0 });
98 expect(budgetLimit(app.db, "run")).toBe(0.1);
99 const issue = openIssue(app, alice, repo, { title: "Mow the lawn", body: "Soon." });
100 if (!issue.ok) throw new Error(String(issue.error));
101 expect(assignIssue(app, alice, repo, issue.value, bot).ok).toBe(true);
102 const stopped = await agentRunSettled(app, repo, (r) => r.issueId === issue.value.id, 60_000);
103 expect(stopped.state).toBe("failed");
104 expect(stopped.summary).toContain("per run limit ($0.10)");
105 expect(stopped.summary).toContain("/settings/crew#budgets");
106 expect(existsSync(standIn.argsFile)).toBe(false);
107 for (let i = 0; i < 100 && parkedCount(app.db, repo, issue.value.id) === 0; i++) await Bun.sleep(100);
108
109 expect(saveCrewBudgets(app, admin, { run: 5 })).toEqual({ ok: true, woken: 1 });
110 const again = await settledNth(issue.value.id, 2);
111 expect(again.summary).not.toContain("limit");
112 expect(existsSync(standIn.argsFile)).toBe(true);
113 }, 120_000);
114
115 test("the limits refuse what is not dollars and cents, and only an admin sets them (a spent day or month: forge-budget-verdict.test.ts)", () => {
116 const admin = { ...alice, isAdmin: true };
117 expect(saveCrewBudgets(app, admin, { day: -1 })).toEqual({ ok: false, error: "invalid_amount" });
118 expect(saveCrewBudgets(app, admin, { month: 1.234 })).toEqual({ ok: false, error: "invalid_amount" });
119 expect(saveCrewBudgets(app, { ...alice, isAdmin: false }, { day: 5 })).toEqual({ ok: false, error: "not_found" });
120 expect(setBudget(app.db, "day", 0.01, alice.id)).toBe(true);
121 expect(budgetLimit(app.db, "day")).toBe(0.01);
122 });
123 });
test/forge-budget-verdict.test.ts added +178
@@ -0,0 +1,178 @@
1 /**
2 * The owner's limits as a verdict (#11515): a spent day or month stops a run
3 * naming it; the cap is the smallest remainder; runs started together share
4 * what is left; a host forge is left undecided; the mid-run stop says which
5 * limit; a wake wakes only what its cause can fix; the terminal door is
6 * gated like the page.
7 */
8 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
9 import { mkdtemp, rm } from "node:fs/promises";
10 import { writeFileSync } from "node:fs";
11 import { tmpdir } from "node:os";
12 import { join } from "node:path";
13 import { saveCrewBudgets } from "../src/actions/crew_budgets.ts";
14 import { createRepo } from "../src/actions/repos.ts";
15 import { addComment, openIssue } from "../src/actions/issues.ts";
16 import { createApp, type App } from "../src/app.ts";
17 import { createRun } from "../src/core/runs.ts";
18 import type { Repo } from "../src/core/repos.ts";
19 import { createUser, type User } from "../src/core/users.ts";
20 import { budgetVerdict, cutOffBy, isLimitStop, NO_KEY_SAID, setBudget, stoppedBy } from "../src/crew/forge_budget.ts";
21 import { PARK_NOTE, parkedCount } from "../src/crew/parks.ts";
22 import { wakeParked } from "../src/crew/wake.ts";
23 import { budgetRows, setChosen } from "../src/tui/screens/crewbudgets.ts";
24 import { entry } from "../src/tui/screens/crewbudgets.ts";
25 import { quietLogs } from "./helpers/globals.ts";
26
27 quietLogs();
28
29 describe("the owner's limits as a verdict", () => {
30 let dir = "";
31 let app: App;
32 let alice: User;
33 let bob: User;
34 let repo: Repo;
35 const now = new Date();
36
37 const spend = (usd: number, at: Date) => {
38 const run = createRun(app.db, { repoId: repo.id, kind: "agent" });
39 app.db
40 .query(`INSERT INTO usage (run_id, repo_id, kind, cost_usd, outcome, at) VALUES (?1, ?2, 'build', ?3, 'ok', ?4)`)
41 .run(run.id, repo.id, usd, at.toISOString());
42 };
43 const clear = () => {
44 app.db.query("DELETE FROM usage").run();
45 app.db.query("DELETE FROM run").run();
46 };
47
48 beforeAll(async () => {
49 dir = await mkdtemp(join(tmpdir(), "forge-verdict-"));
50 app = createApp({ dataDir: dir, port: 0 });
51 const a = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-long-starter-passphrase", adminIfFirst: true });
52 const b = await createUser(app.db, { origin: "seed", trusted: true, username: "bob", email: "[email protected]", password: "a-long-starter-passphrase" });
53 if (!a.ok || !b.ok) throw new Error("user");
54 alice = a.value;
55 bob = b.value;
56 const r = await createRepo(app, alice, { name: "garden", visibility: "public" });
57 if (!r.ok) throw new Error("repo");
58 repo = r.value;
59 app.db.query("UPDATE repo SET autonomy = 'autonomous' WHERE id = ?1").run(repo.id);
60 });
61 afterAll(async () => {
62 app?.close();
63 await rm(dir, { recursive: true, force: true });
64 });
65
66 test("a forge that hosts others is left undecided", () => {
67 expect(budgetVerdict(app, 5, now)).toBeUndefined();
68 expect(entry.offered({ app, db: app.db, user: alice } as never)).toBe(false);
69 });
70
71 describe("on a forge of its own", () => {
72 beforeAll(() => writeFileSync(join(dir, "parent.json"), "{}"));
73
74 test("a spent day stops the run and names the day limit", () => {
75 clear();
76 setBudget(app.db, "run", 5, alice.id);
77 setBudget(app.db, "day", 10, alice.id);
78 setBudget(app.db, "month", 500, alice.id);
79 spend(9.99, now);
80 const v = budgetVerdict(app, 5, now);
81 expect(v).toEqual({ ok: false, limit: "day", said: stoppedBy(app.db, "day") });
82 expect(stoppedBy(app.db, "day")).toContain("per day limit ($10.00)");
83 });
84
85 test("a spent month stops the run and names the month limit", () => {
86 clear();
87 setBudget(app.db, "day", 1000, alice.id);
88 setBudget(app.db, "month", 20, alice.id);
89 // Yesterday's spend, in this month: the day is clear, the month is not.
90 const earlier = new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), 1, 0, 0, 1));
91 spend(19.995, earlier);
92 const v = budgetVerdict(app, 5, new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), 1, 12)));
93 expect(v?.ok).toBe(false);
94 expect(v && !v.ok && v.limit).toBe("month");
95 expect(v && !v.ok && v.said).toContain("per month limit ($20.00)");
96 });
97
98 test("the cap is the smallest remainder and `by` names it", () => {
99 clear();
100 setBudget(app.db, "run", 5, alice.id);
101 setBudget(app.db, "day", 50, alice.id);
102 setBudget(app.db, "month", 500, alice.id);
103 expect(budgetVerdict(app, 3, now)).toEqual({ ok: true, capUsd: 3, by: undefined });
104 expect(budgetVerdict(app, 20, now)).toEqual({ ok: true, capUsd: 5, by: "run" });
105 spend(47, now);
106 expect(budgetVerdict(app, 20, now)).toEqual({ ok: true, capUsd: 3, by: "day" });
107 setBudget(app.db, "month", 48.5, alice.id);
108 expect(budgetVerdict(app, 20, now)).toEqual({ ok: true, capUsd: 1.5, by: "month" });
109 });
110
111 test("runs started together share what is left, and a finished run's reservation goes", () => {
112 clear();
113 setBudget(app.db, "day", 8, alice.id);
114 setBudget(app.db, "month", 500, alice.id);
115 const first = createRun(app.db, { repoId: repo.id, kind: "agent" });
116 const second = createRun(app.db, { repoId: repo.id, kind: "agent" });
117 app.db.query("UPDATE run SET state = 'running' WHERE id IN (?1, ?2)").run(first.id, second.id);
118 expect(budgetVerdict(app, 5, now, first.id)).toEqual({ ok: true, capUsd: 5, by: undefined });
119 // The first holds $5 of the $8: the second gets $3, not $5.
120 expect(budgetVerdict(app, 5, now, second.id)).toEqual({ ok: true, capUsd: 3, by: "day" });
121 const third = createRun(app.db, { repoId: repo.id, kind: "agent" });
122 app.db.query("UPDATE run SET state = 'running' WHERE id = ?1").run(third.id);
123 expect(budgetVerdict(app, 5, now, third.id)).toEqual({ ok: false, limit: "day", said: stoppedBy(app.db, "day") });
124 app.db.query("UPDATE run SET state = 'ok' WHERE id = ?1").run(first.id);
125 expect(budgetVerdict(app, 5, now, third.id)).toEqual({ ok: true, capUsd: 5, by: undefined });
126 });
127
128 test("a run cut off at a cap an owner's limit set says which limit", () => {
129 expect(cutOffBy(app.db, { ok: true, capUsd: 3, by: "day" }, true)).toBe(` — ${stoppedBy(app.db, "day")}`);
130 expect(cutOffBy(app.db, { ok: true, capUsd: 3, by: "day" }, false)).toBe("");
131 expect(cutOffBy(app.db, { ok: true, capUsd: 3, by: undefined }, true)).toBe("");
132 expect(cutOffBy(app.db, undefined, true)).toBe("");
133 });
134
135 test("a wake wakes only what its cause can fix", () => {
136 setBudget(app.db, "run", 0.1, alice.id);
137 const park = (title: string, reason: string) => {
138 const issue = openIssue(app, alice, repo, { title, body: "x" });
139 if (!issue.ok) throw new Error("issue");
140 addComment(app, alice, repo, issue.value, `${PARK_NOTE}${reason}`);
141 return issue.value.id;
142 };
143 const noKey = park("needs a key", NO_KEY_SAID);
144 const limited = park("needs room", stoppedBy(app.db, "run"));
145 const chatter = park("mentions a key", "the tests have no key for the fixture");
146 expect([noKey, limited, chatter].map((id) => parkedCount(app.db, repo, id))).toEqual([1, 1, 1]);
147
148 expect(wakeParked(app, alice, "limit")).toBe(1);
149 expect([noKey, limited, chatter].map((id) => parkedCount(app.db, repo, id))).toEqual([1, 0, 1]);
150 expect(wakeParked(app, alice, "key")).toBe(1);
151 expect([noKey, limited, chatter].map((id) => parkedCount(app.db, repo, id))).toEqual([0, 0, 1]);
152 expect(wakeParked(app, alice, "key")).toBe(0);
153 expect(isLimitStop("the per run limit stopped it")).toBe(false);
154 });
155
156 test("raising a limit through the action reports the work picked back up", () => {
157 setBudget(app.db, "run", 0.1, alice.id);
158 const issue = openIssue(app, alice, repo, { title: "parked by a limit", body: "x" });
159 if (!issue.ok) throw new Error("issue");
160 addComment(app, alice, repo, issue.value, `${PARK_NOTE}${stoppedBy(app.db, "run")}`);
161 expect(saveCrewBudgets(app, alice, { run: 6 })).toEqual({ ok: true, woken: 1 });
162 });
163
164 test("the terminal door shows and sets the limits for an admin, and nothing for anyone else", () => {
165 setBudget(app.db, "run", 5, alice.id);
166 const as = (user: User) => ({ app, db: app.db, user }) as never;
167 expect(budgetRows(as(alice), 0)[0]).toBe("> Spend per run: $5.00");
168 expect(budgetRows(as(alice), 0)).toHaveLength(3);
169 expect(budgetRows(as(bob), 0)).toEqual([]);
170 expect(entry.offered(as(alice))).toBe(true);
171 expect(entry.offered(as(bob))).toBe(false);
172 expect(setChosen(as(bob), 0, "9")).toEqual({ said: "Only a forge admin sets a budget.", saved: false });
173 expect(setChosen(as(alice), 0, "abc").saved).toBe(false);
174 expect(setChosen(as(alice), 0, "7.50")).toEqual({ said: "Spend per run is $7.50.", saved: true });
175 expect(budgetRows(as(alice), 0)[0]).toBe("> Spend per run: $7.50");
176 });
177 });
178 });
test/starter-crew.test.ts +1 −1
@@ -174,7 +174,7 @@ describe("a new forge's starter crew", () => {
174174 const owners = listTeams(child.db, getUserByName(child.db, "maria")!.id).find((t) => t.name === OWNERS_TEAM)!;
175175 const founder = listTeamMembers(child.db, owners.id)[0]!;
176176 const saved = await saveCrewKey(child, { ...founder, isAdmin: true, kind: "user" }, " sk-ant-api03-abcdefghij ");
177 expect(saved).toEqual({ ok: true });
177 expect(saved).toEqual({ ok: true, woken: 0 });
178178 const subs = listSubscriptions(child.db);
179179 expect(subs.map((x) => x.name)).toEqual(["my-key"]);
180180 const org = getUserByName(child.db, "maria")!;

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs