Public: anyone can read this; the crew can change it.

A feature's scheduled tool is called the way its hand-run tool is: the unified timer broke the morning paper's print with a header its app refuses #14218

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

mason/forge:mason/14216-a-feature-s-scheduled-tool-is-called-the → main 7 commits fork

  • Merged. mason/14216-a-feature-s-scheduled-tool-is-called-the was merged into main as 5967d5f.

Review

Sign in to review this change.
12 files changed +321 −22
docs/features/feature-schedule.md +21
@@ -30,6 +30,27 @@ A feature's `schedule` is sugar for top-level `schedules` (#13533): `src/apps/fe
3030
3131 A name used by both a `schedules` entry and a feature tool is refused by the manifest. A run due while the forge was away fires within an hour (`catchUp`) and says it was late. **Only while the app is absorbed:** the forge calls a feature's tool only for an app an admin has absorbed. The timer exists from the commit (so its row shows), but when it comes due for an app that is not absorbed it is dropped unfired, with no run recorded and no call made (`permits` in `timer_fire.ts`, set by `timer_service.ts`); Run now on the features page and the terminal's `r` refuse it in words, arming nothing. Un-absorbing therefore stops the schedule at its next due time; absorbing again starts it again, at the next due time, never at once.
3232
33 ## Calling a tool: one contract (#14216)
34
35 A call of `POST /forge/tools/<tool>` carries the same headers whoever starts it: the clock, Run now, a person or an agent running the tool by hand (the `features` tool), or a probe. Both paths build them with `callHeaders` in `src/apps/timer_fire.ts`; `test/feature-call-contract.test.ts` holds it.
36
37 | Header | Is |
38 |---|---|
39 | `x-forge-schedule` | the tool's name |
40 | `x-forge-scheduled-at` | the instant the call is for (a hand's call: now) |
41 | `x-forge-attempt` | `1`, or the try count of a delivery |
42 | `idempotency-key` | `<app>:<name>:<scheduled time>` for the clock, `manual:<tool>:<uuid>` for a hand |
43 | `x-forge-identity` | `forge-timer\|<idempotency key>\|timer\|<expiry ms>` |
44 | `x-forge-signature` | base64url HMAC-SHA256 of `x-forge-identity` under the app's identity key |
45
46 **A hand's call is signed as a timer's, deliberately.** The identity of Run now, of a person's or an agent's call and of a probe is `forge-timer|manual:<tool>:<uuid>|timer|<expiry>`: one check (`fromForge`) accepts every call the forge makes to a tool, and the app does not branch on who started it. The `manual:` prefix of the idempotency key is what tells a hand's call from the clock's, if an app wants to know. Only the forge's key signs it; a visitor cannot.
47
48 **Verifying "this is my forge".** The key is `FORGE_APP_IDENTITY_KEY` in the app's environment, the same key that verifies a visitor's `x-forge-identity`. A request is the forge's own when `x-forge-identity` starts with `forge-timer|`, has not expired and its signature verifies. Do not guess from the absence of headers: the proxy puts `x-forge-identity` on a visitor's relayed request too, and a guard that treats any such header as a visitor's refuses the clock's call. The app template's `fromForge` (`src/once.ts`) is this check; a tool route should call it.
49
50 **Probe.** A tool declared `"probe": true` promises to answer 200 to a call carrying `x-forge-probe: 1` without doing its work. When the forge reads a commit's schedules (once per commit, so also when a mechanism change re-reads every app), it sends each such scheduled tool one probe, after every app's schedules are read and within a 30 s budget for the pass. Only production is probed: a preview's schedules do not fire. An app that refuses it gets a failed run on the schedule, with the answer's status, and its owner is asked at once. A tool without `probe` is never called in advance: a call would do the work.
51
52 **Proven.** `/api/v1/health` `schedules.ran` counts a schedule only once a fired run was answered with a 2xx (200, 202, 204 alike); a refused probe never counts.
53
3354 ## What the forge records
3455
3556 The runs are the timers' (`timer_run`, `src/core/timers.ts`): when each started, how many attempts, the HTTP status, the first 2 KB of the app's answer, the error sentence. A hand's call (the `features` tool, `mcp`) is the one thing still kept in `feature_fire` and `feature_run` (`src/core/feature_fires.ts`); un-absorbing or deleting the app clears those.
src/actions/feature_tools.ts +4 −2
@@ -8,6 +8,7 @@
88
99 import type { App } from "../app.ts";
1010 import { runRecorded } from "../apps/feature_schedule.ts";
11 import { identityKey } from "../apps/proxy.ts";
1112 import { callTool } from "../apps/feature_tools.ts";
1213 import { appsRuntime } from "../apps/runtime.ts";
1314 import { getAbsorbed } from "../core/features.ts";
@@ -36,6 +37,7 @@ export async function callFeatureTool(
3637 if (spec.writes && !canWrite(app.db, found.repo, actor)) return refuse("forbidden", "A tool that changes things is for someone who can write this app.");
3738 const port = appsRuntime(app)?.envStatus(found.record, "production")?.port;
3839 if (port === undefined) return refuse("not_running", "The app is not running.");
40 const key = identityKey(app.config.dataDir, found.record.id);
3941 // A plan is told only once the call could be made: allowed, absorbed, running.
4042 if (spec.writes && dryRun) return ok({ planned: `Would call ${tool} on ${found.record.slug}: ${spec.description} Send dryRun: false to do it.` });
4143 // A tool the schedule names is the same recorded run as the schedule's and the page's, so it
@@ -43,10 +45,10 @@ export async function callFeatureTool(
4345 // recorded.
4446 const scheduled = declared.ok && declared.value?.schedule.some((e) => e.tool === tool) === true;
4547 if (!scheduled) {
46 const answer = await callTool(port, tool, args);
48 const answer = await callTool(port, tool, args, { key });
4749 return answer.ok ? ok({ answer: answer.value }) : refuse("failed", answer.error);
4850 }
49 const ran = await runRecorded(app.db, found.record.id, tool, port, args, "mcp");
51 const ran = await runRecorded(app.db, found.record.id, tool, port, args, key, "mcp");
5052 if (!ran.ran) return refuse("failed", `${tool} is already running.`);
5153 return ran.answer.ok ? ok({ answer: ran.answer.value }) : refuse("failed", ran.answer.error);
5254 }
src/apps/feature_schedule.ts +2 −1
@@ -29,6 +29,7 @@ export async function runRecorded(
2929 tool: string,
3030 port: number,
3131 args: Record<string, unknown>,
32 identity: string,
3233 how: How,
3334 now: () => number = Date.now,
3435 ): Promise<{ ran: true; answer: Result<unknown, string> } | { ran: false }> {
@@ -39,7 +40,7 @@ export async function runRecorded(
3940 const started = now();
4041 // callTool is bounded (WAIT_MS) and answers an error rather than throwing; a throw is still a
4142 // recorded failure.
42 const answer = await callTool(port, tool, args).catch((): Result<unknown, string> => err("the app did not answer"));
43 const answer = await callTool(port, tool, args, { key: identity, now }).catch((): Result<unknown, string> => err("the app did not answer"));
4344 recordFire(db, appId, { tool, at: started, ok: answer.ok, error: answer.ok ? null : answer.error, durationMs: Math.max(0, now() - started) }, how);
4445 return { ran: true, answer };
4546 } finally {
src/apps/feature_tools.ts +22 −4
@@ -12,12 +12,18 @@
1212 */
1313
1414 import { err, ok, type Result } from "../kernel/result.ts";
15 import { callHeaders } from "./timer_fire.ts";
1516
1617 export interface ToolSpec {
1718 readonly name: string;
1819 readonly description: string;
1920 /** Whether a call changes anything: a write plans first, like the forge's own. */
2021 readonly writes: boolean;
22 /**
23 * Whether the tool answers a probe: a call carrying `x-forge-probe: 1` that checks the forge's
24 * headers are accepted and does none of the work. Only such a tool is probed.
25 */
26 readonly probe?: true;
2127 }
2228
2329 const NAME = /^[a-z][a-z0-9_]{0,39}$/;
@@ -37,22 +43,34 @@ export function parseTools(raw: unknown): Result<readonly ToolSpec[], string> {
3743 const t = (raw as Record<string, unknown>)[name];
3844 if (!NAME.test(name)) return err(`feature: tool ${JSON.stringify(name)}: names are lowercase words with _, at most 40 characters`);
3945 if (typeof t !== "object" || t === null || Array.isArray(t)) return err(`feature: tool ${name} must be an object`);
40 const { description, writes, ...extra } = t as Record<string, unknown>;
46 const { description, writes, probe, ...extra } = t as Record<string, unknown>;
4147 if (typeof description !== "string" || description.trim() === "" || description.length > DESCRIPTION_MAX)
4248 return err(`feature: tool ${name}: description is text of 1 to ${DESCRIPTION_MAX} characters`);
4349 if (writes !== undefined && typeof writes !== "boolean") return err(`feature: tool ${name}: writes is true or false`);
50 if (probe !== undefined && typeof probe !== "boolean") return err(`feature: tool ${name}: probe is true or false`);
4451 if (Object.keys(extra).length > 0) return err(`feature: tool ${name}: unknown field ${Object.keys(extra)[0]}`);
45 out.push({ name, description: description.trim(), writes: writes !== false });
52 out.push({ name, description: description.trim(), writes: writes !== false, ...(probe === true ? { probe: true as const } : {}) });
4653 }
4754 return ok(out);
4855 }
4956
5057 /** Call a declared tool on the app listening on `port`: its JSON answer, or why not. */
51 export async function callTool(port: number, tool: string, args: Record<string, unknown>): Promise<Result<unknown, string>> {
58 export async function callTool(
59 port: number,
60 tool: string,
61 args: Record<string, unknown>,
62 opts: { readonly key: string; readonly probe?: boolean; readonly now?: () => number },
63 ): Promise<Result<unknown, string>> {
64 const now = (opts.now ?? Date.now)();
5265 try {
5366 const res = await fetch(`http://127.0.0.1:${port}/forge/tools/${tool}`, {
5467 method: "POST",
55 headers: { "content-type": "application/json", accept: "application/json" },
68 headers: {
69 "content-type": "application/json",
70 accept: "application/json",
71 ...callHeaders(opts.key, tool, now, 1, now + WAIT_MS, `manual:${tool}:${crypto.randomUUID()}`),
72 ...(opts.probe === true ? { "x-forge-probe": "1" } : {}),
73 },
5674 body: JSON.stringify({ args }),
5775 redirect: "manual",
5876 signal: AbortSignal.timeout(WAIT_MS),
src/apps/schedule_probe.ts added +51
@@ -0,0 +1,51 @@
1 /**
2 * A dry run of a feature's scheduled tools, once, when the forge reads them from a commit
3 * (#14216): a schedule that moved to a new mechanism, or a new commit that changed what an app's
4 * guard accepts, is found refused the same hour, not at the next 07:15.
5 *
6 * The call is a tool call like any other (`callTool`: the headers of a delivery) plus
7 * `x-forge-probe: 1`. Only a tool that declares `"probe": true` is probed: it promises to answer
8 * 200 to a probe and do none of its work, so a probe never prints the paper. A refusal is recorded
9 * as a failed run of the schedule, which the app's owner is asked about like any failed run;
10 * an answer of 200 records nothing, because a schedule is proven only by a fired run the app
11 * answered 200 (`overdueSchedules`).
12 */
13
14 import type { Database } from "bun:sqlite";
15 import type { AppRecord } from "../core/apps.ts";
16 import { recordRun, scheduleTimers, type TimerEnv } from "../core/timers.ts";
17 import { callTool, type ToolSpec } from "./feature_tools.ts";
18 import { toolPath } from "./feature.ts";
19 import type { Reach } from "./timer_fire.ts";
20
21 /** Probe each scheduled tool of `app` in `env` that answers probes; the tools refused. */
22 export async function probeSchedules(
23 db: Database,
24 reach: Reach,
25 app: AppRecord,
26 env: TimerEnv,
27 tools: readonly ToolSpec[],
28 now: () => number = Date.now,
29 ): Promise<string[]> {
30 const refused: string[] = [];
31 for (const t of scheduleTimers(db, app.id)) {
32 if (t.env !== env || !t.path.startsWith(toolPath(""))) continue;
33 const name = t.path.slice(toolPath("").length);
34 if (tools.find((x) => x.name === name)?.probe !== true) continue;
35 if (reach.permits?.(app, t) === false) continue;
36 const port = await reach.ensure(app, env).catch(() => undefined);
37 if (port === undefined) continue;
38 const started = now();
39 const answer = await callTool(port, name, {}, { key: reach.identityKey(app.id), probe: true, now });
40 if (answer.ok) continue;
41 refused.push(name);
42 const status = /^the app answered (\d{3})$/.exec(answer.error)?.[1];
43 recordRun(db, {
44 appId: app.id, env, name: t.name, idemKey: `probe:${t.name}:${started}`, scheduledAt: started, startedAt: started, finishedAt: now(),
45 attempts: 1, status: "failed", httpStatus: status === undefined ? null : Number(status), response: null,
46 error: `A probe of ${name} was refused: ${answer.error}. The schedule would fail the same way; the app's guard must accept the forge's signed headers (docs/features/feature-schedule.md, "Calling a tool").`,
47 lateMs: 0, lateReason: null, durationMs: Math.max(0, now() - started),
48 });
49 }
50 return refused;
51 }
src/apps/timer_fire.ts +15 −3
@@ -59,15 +59,27 @@ export function idempotencyKey(slug: string, t: Pick<Timer, "name" | "key" | "sc
5959 * `forge-timer|<idempotency key>|timer|<expiry ms>`.
6060 */
6161 export function deliveryHeaders(key: string, t: Timer, attempt: number, expires: number, idemKey: string): Record<string, string> {
62 return {
63 ...callHeaders(key, t.name === "" ? t.key : t.name, t.scheduledAt, attempt, expires, idemKey),
64 ...(t.body === null ? {} : { "content-type": t.contentType ?? "application/octet-stream" }),
65 };
66 }
67
68 /**
69 * The one set of headers the forge sends when it calls an app's route, whoever starts the call: the
70 * clock, Run now, a person or an agent running a feature's tool by hand, a probe (docs/features/
71 * feature-schedule.md, "Calling a tool"). The signature covers
72 * `forge-timer|<idempotency key>|timer|<expiry ms>`.
73 */
74 export function callHeaders(key: string, name: string, scheduledAt: number, attempt: number, expires: number, idemKey: string): Record<string, string> {
6275 const identity = `forge-timer|${idemKey}|timer|${expires}`;
6376 return {
64 "x-forge-schedule": t.name === "" ? t.key : t.name,
65 "x-forge-scheduled-at": iso(t.scheduledAt),
77 "x-forge-schedule": name,
78 "x-forge-scheduled-at": iso(scheduledAt),
6679 "x-forge-attempt": String(attempt),
6780 "idempotency-key": idemKey,
6881 "x-forge-identity": identity,
6982 "x-forge-signature": createHmac("sha256", key).update(identity).digest("base64url"),
70 ...(t.body === null ? {} : { "content-type": t.contentType ?? "application/octet-stream" }),
7183 };
7284 }
7385
src/apps/timer_service.ts +39 −4
@@ -15,7 +15,8 @@ import { getRepoById } from "../core/repos.ts";
1515 import { dropGoneEnvs, syncSchedules, syncedCommit, type Timer } from "../core/timers.ts";
1616 import { readFileAt } from "./checkout.ts";
1717 import { identityKey } from "./proxy.ts";
18 import { toolPath } from "./feature.ts";
18 import { parseFeature, toolPath } from "./feature.ts";
19 import { probeSchedules } from "./schedule_probe.ts";
1920 import { parseManifest } from "./manifest.ts";
2021 import { appsRuntime } from "./runtime.ts";
2122 import type { EnvName } from "./data.ts";
@@ -25,9 +26,12 @@ import { during } from "../perf/jobs.ts";
2526
2627 /**
2728 * Bumped when what a commit's manifest means for schedules changes, so every environment is read
28 * again (2: a feature's `schedule` is a schedule, #13533).
29 * again (2: a feature's `schedule` is a schedule, #13533; 3: each is probed once, #14216).
2930 */
30 const SYNC_REVISION = "#2";
31 /** The most one pass of the sync spends probing, all apps together. */
32 const PROBE_BUDGET_MS = 30_000;
33
34 const SYNC_REVISION = "#3";
3135
3236 /**
3337 * A feature's tool is called only while an admin has absorbed the app (docs/features/absorb.md).
@@ -52,9 +56,12 @@ export function runtimeReach(app: App): Reach {
5256 }
5357
5458 /** Read each app's declared schedules at the commit each environment serves, once per commit. */
55 export async function syncAll(app: App, now: () => number = Date.now): Promise<void> {
59 export async function syncAll(app: App, now: () => number = Date.now, reach: Reach = runtimeReach(app)): Promise<void> {
5660 const rt = appsRuntime(app);
5761 if (rt === undefined) return;
62 // Probes call apps (waking one, waiting on its answer), so they run after every app's schedules
63 // are read: a slow app delays no other app's read.
64 const toProbe: { record: AppRecord; text: string }[] = [];
5865 for (const record of listApps(app.db)) {
5966 const live = new Set<string>();
6067 for (const st of rt.status(record)) {
@@ -69,9 +76,19 @@ export async function syncAll(app: App, now: () => number = Date.now): Promise<v
6976 // A forge.json that does not parse is the deploy's to report; it declares no schedule, so the
7077 // rows are dropped.
7178 syncSchedules(app.db, record.id, st.env, manifest?.ok ? (manifest.value.schedules ?? []) : [], sync, now());
79 // Only production is probed: a preview's schedules do not fire (they are read for the page),
80 // so a refusal there would be reported for a run that never comes.
81 if (st.env === "production" && text !== undefined) toProbe.push({ record, text });
7282 }
7383 dropGoneEnvs(app.db, record.id, live);
7484 }
85 // Bounded in all: probes still waiting when the budget is spent are left; they are not retried,
86 // for a schedule is proven by a run, not by its probe.
87 const deadline = Date.now() + PROBE_BUDGET_MS;
88 for (const { record, text } of toProbe) {
89 if (Date.now() >= deadline) break;
90 await Promise.race([probeFeature(app.db, reach, record, text, now), Bun.sleep(deadline - Date.now())]);
91 }
7592 }
7693
7794 export function startTimers(app: App, tickMs = 1000): { stop(): void } | undefined {
@@ -103,3 +120,21 @@ export function startTimers(app: App, tickMs = 1000): { stop(): void } | undefin
103120 tick.unref();
104121 return { stop: () => clearInterval(tick) };
105122 }
123
124 /**
125 * Probe the scheduled tools a commit's `forge.json` declares, once; a text that does not parse has
126 * none.
127 */
128 async function probeFeature(db: Database, reach: Reach, record: AppRecord, text: string, now: () => number): Promise<void> {
129 let raw: unknown;
130 try {
131 raw = JSON.parse(text);
132 } catch {
133 return;
134 }
135 if (typeof raw !== "object" || raw === null || Array.isArray(raw)) return;
136 const feature = parseFeature(raw as Record<string, unknown>);
137 if (feature.ok && feature.value !== undefined && feature.value.schedule.length > 0) {
138 await probeSchedules(db, reach, record, "production", feature.value.tools, now).catch(() => undefined);
139 }
140 }
src/core/timers.ts +5 −2
@@ -119,7 +119,10 @@ export interface ScheduleCounts {
119119 readonly oldestSeconds: number;
120120 /** Unpaused schedules in all. */
121121 readonly total: number;
122 /** Of those, how many have at least one recorded run. */
122 /**
123 * Of those, how many have a fired run the app answered 2xx (a refused probe, or a run it answered
124 * otherwise, proves nothing).
125 */
123126 readonly ran: number;
124127 }
125128
@@ -137,7 +140,7 @@ export function overdueSchedules(db: Database, now: number): ScheduleCounts {
137140 .get(now - OVERDUE_AFTER_MS);
138141 const all = db
139142 .query<{ total: number; ran: number | null }, []>(
140 `SELECT COUNT(*) AS total, SUM(EXISTS (SELECT 1 FROM timer_run r WHERE r.app_id = t.app_id AND r.name = t.name AND r.env = t.env)) AS ran
143 `SELECT COUNT(*) AS total, SUM(EXISTS (SELECT 1 FROM timer_run r WHERE r.app_id = t.app_id AND r.name = t.name AND r.env = t.env AND r.status = 'ok' AND r.http_status BETWEEN 200 AND 299)) AS ran
141144 FROM timer t WHERE t.kind = 'schedule' AND t.paused = 0`,
142145 )
143146 .get();
test/feature-call-contract.test.ts added +123
@@ -0,0 +1,123 @@
1 /**
2 * One contract for calling a feature's tool (#14216): the hand's call and the clock's carry the
3 * same headers, an app can verify them by signature, and a schedule whose app refuses them is
4 * flagged at once by a probe, not at its next firing. The app here refuses anything it cannot
5 * verify.
6 */
7 import { afterAll, beforeAll, expect, test } from "bun:test";
8 import { createHmac } from "node:crypto";
9 import { mkdtemp, rm } from "node:fs/promises";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12 import { createApp, type App } from "../src/app.ts";
13 import { createApp as createAppRecord, getAppById, type AppRecord } from "../src/core/apps.ts";
14 import { createRepo } from "../src/core/repos.ts";
15 import { createUser } from "../src/core/users.ts";
16 import { overdueSchedules, recordRun, runsOf, failingNow, scheduleTimer, syncSchedules } from "../src/core/timers.ts";
17 import { scheduleSpecs } from "../src/apps/feature.ts";
18 import { callTool } from "../src/apps/feature_tools.ts";
19 import { probeSchedules } from "../src/apps/schedule_probe.ts";
20 import { deliveryHeaders, type Reach } from "../src/apps/timer_fire.ts";
21 import type { ToolSpec } from "../src/apps/feature_tools.ts";
22
23 const KEY = "the-apps-identity-key";
24 let dataDir: string;
25 let app: App;
26 let record: AppRecord;
27 const seen: Record<string, string>[] = [];
28 let probes = 0;
29 let works = 0;
30
31 /**
32 * The documented verification: a valid signature over x-forge-identity under the app's key, not yet
33 * expired.
34 */
35 function fromForge(h: Headers): boolean {
36 const identity = h.get("x-forge-identity") ?? "";
37 const sig = h.get("x-forge-signature") ?? "";
38 const [kind, , , expires] = identity.split("|");
39 return kind === "forge-timer" && Number(expires) > Date.now() && sig === createHmac("sha256", KEY).update(identity).digest("base64url");
40 }
41
42 const server = Bun.serve({
43 port: 0,
44 hostname: "127.0.0.1",
45 fetch(req) {
46 // An app that refuses what it cannot verify, and any header it does not know.
47 if (!fromForge(req.headers)) return new Response("nope", { status: 404 });
48 seen.push(Object.fromEntries(req.headers));
49 if (req.headers.get("x-forge-probe") === "1") {
50 probes++;
51 return Response.json({ probe: true });
52 }
53 works++;
54 return Response.json({ printed: true });
55 },
56 });
57
58 beforeAll(async () => {
59 dataDir = await mkdtemp(join(tmpdir(), "forge-call-contract-"));
60 app = createApp({ dataDir, port: 0, disk: () => ({ freeBytes: 95, totalBytes: 100 }) });
61 const owner = await createUser(app.db, { origin: "seed", trusted: true, username: "paper-owner", email: "[email protected]", password: "a-good-passphrase" });
62 if (!owner.ok) throw new Error(owner.error);
63 const repo = await createRepo(app.db, app.reposRoot, { ownerId: owner.value.id, name: "paper" });
64 if (!repo.ok) throw new Error(repo.error);
65 const made = createAppRecord(app.db, { repoId: repo.value.id, slug: "paper", access: "private" });
66 if (!made.ok) throw new Error(made.error);
67 record = getAppById(app.db, made.value.id)!;
68 const specs = scheduleSpecs([{ tool: "print_today", at: "07:15", tz: "America/Chicago" }]);
69 syncSchedules(app.db, record.id, "production", specs, "c1", Date.now());
70 });
71 afterAll(async () => {
72 server.stop(true);
73 app.close();
74 await rm(dataDir, { recursive: true, force: true });
75 });
76
77 const reach = (key: string): Reach => ({ ensure: async () => server.port, identityKey: () => key, fetch });
78 const tool = (probe: boolean): ToolSpec => ({ name: "print_today", description: "Print", writes: true, ...(probe ? { probe: true as const } : {}) });
79
80 test("a person's call carries the same headers as the clock's, and the app verifies them", async () => {
81 const out = await callTool(server.port!, "print_today", {}, { key: KEY });
82 expect(out).toEqual({ ok: true, value: { printed: true } });
83 const timer = scheduleTimer(app.db, record.id, "print_today", "production")!;
84 const clock = Object.keys(deliveryHeaders(KEY, timer, 1, Date.now() + 1000, "k")).filter((h) => h !== "content-type");
85 const hand = seen.at(-1)!;
86 for (const h of clock) expect(hand[h]).toBeDefined();
87 // Signed under another key, the app refuses it as it would a visitor's relayed request.
88 expect(await callTool(server.port!, "print_today", {}, { key: "someone-else" })).toEqual({ ok: false, error: "the app answered 404" });
89 });
90
91 test("a probe of a tool the app refuses is a failed run the owner is asked about; the schedule is not proven", async () => {
92 const before = overdueSchedules(app.db, Date.now()).ran;
93 const refused = await probeSchedules(app.db, reach("a-wrong-key"), record, "production", [tool(true)]);
94 expect(refused).toEqual(["print_today"]);
95 const run = runsOf(app.db, record.id, "print_today", 5)[0]!;
96 expect(run.status).toBe("failed");
97 expect(run.httpStatus).toBe(404);
98 expect(run.error).toContain("refused");
99 expect(failingNow(app.db, record.id).map((r) => r.name)).toEqual(["print_today"]);
100 expect(overdueSchedules(app.db, Date.now()).ran).toBe(before);
101 });
102
103 test("a probe the app accepts records nothing and does none of the work; a tool that does not answer probes is never called", async () => {
104 const runs = runsOf(app.db, record.id).length;
105 const worked = works;
106 expect(await probeSchedules(app.db, reach(KEY), record, "production", [tool(true)])).toEqual([]);
107 expect(probes).toBe(1);
108 expect(works).toBe(worked);
109 expect(runsOf(app.db, record.id).length).toBe(runs);
110 expect(await probeSchedules(app.db, reach("a-wrong-key"), record, "production", [tool(false)])).toEqual([]);
111 expect(runsOf(app.db, record.id).length).toBe(runs);
112 });
113
114 test("a schedule whose app answers 204 to its run is proven; a 404 is not", () => {
115 const timer = scheduleTimer(app.db, record.id, "print_today", "production")!;
116 const base = { appId: record.id, env: "production" as const, name: timer.name, scheduledAt: 1, startedAt: 1, finishedAt: 2, attempts: 1, response: null, error: null, lateMs: 0, lateReason: null, durationMs: 1 };
117 const proven = () => overdueSchedules(app.db, Date.now()).ran;
118 const before = proven();
119 recordRun(app.db, { ...base, idemKey: "r-404", status: "failed", httpStatus: 404 });
120 expect(proven()).toBe(before);
121 recordRun(app.db, { ...base, idemKey: "r-204", status: "ok", httpStatus: 204 });
122 expect(proven()).toBe(1);
123 });
test/feature-schedule-one.test.ts +2 −1
@@ -134,6 +134,7 @@ describe("the morning paper's print", () => {
134134 const runs = runsOf(db, id, "print_today");
135135 expect(runs.map((r) => r.status)).toEqual(["failed"]);
136136 expect(runs[0]?.error).toContain("the app did not answer");
137 expect(overdueSchedules(db, now)).toMatchObject({ total: 1, ran: 1 });
137 // A failed run proves nothing (#14216): only a fired run the app answered 200 counts as ran.
138 expect(overdueSchedules(db, now)).toMatchObject({ total: 1, ran: 0 });
138139 });
139140 });
test/feature-tools.test.ts +5 −5
@@ -47,7 +47,7 @@ describe("calling a tool on the app", () => {
4747 return Response.json({ found: 3 });
4848 });
4949 try {
50 const out = await callTool(s.port!, "scan", { depth: 2 });
50 const out = await callTool(s.port!, "scan", { depth: 2 }, { key: "k" });
5151 expect(out).toEqual({ ok: true, value: { found: 3 } });
5252 expect(seen).toBe('POST /forge/tools/scan {"args":{"depth":2}}');
5353 } finally {
@@ -63,13 +63,13 @@ describe("calling a tool on the app", () => {
6363 return new Response("x".repeat(70 * 1024));
6464 });
6565 try {
66 expect((await callTool(bad.port!, "boom", {})).ok).toBe(false);
67 expect((await callTool(bad.port!, "text", {})).ok).toBe(false);
68 const big = await callTool(bad.port!, "big", {});
66 expect((await callTool(bad.port!, "boom", {}, { key: "k" })).ok).toBe(false);
67 expect((await callTool(bad.port!, "text", {}, { key: "k" })).ok).toBe(false);
68 const big = await callTool(bad.port!, "big", {}, { key: "k" });
6969 expect(!big.ok && big.error).toContain("64 KB");
7070 } finally {
7171 bad.stop(true);
7272 }
73 expect((await callTool(1, "scan", {})).ok).toBe(false);
73 expect((await callTool(1, "scan", {}, { key: "k" })).ok).toBe(false);
7474 });
7575 });
test/timers-sync.test.ts +32
@@ -10,6 +10,7 @@ import { createApp, type App } from "../src/app.ts";
1010 import { AppsRuntime } from "../src/apps/runtime.ts";
1111 import { newApp } from "../src/actions/apps.ts";
1212 import { commitToBranch } from "../src/actions/repos.ts";
13 import type { Reach } from "../src/apps/timer_fire.ts";
1314 import { syncAll } from "../src/apps/timer_service.ts";
1415 import { getAppByRepo } from "../src/core/apps.ts";
1516 import { getRepo } from "../src/core/repos.ts";
@@ -66,3 +67,34 @@ test("the schedules a production commit declares become rows once per commit; a
6667 await syncAll(app);
6768 expect(scheduleTimers(app.db, record.id)[0]!.paused).toBe(true);
6869 }, 30_000);
70
71 test("a new commit's scheduled tool is probed once by the sync pass, and not again on the next", async () => {
72 const alice = getUserByName(app.db, "alice")!;
73 const made = await newApp(app, alice, { name: "gazette", visibility: "private" });
74 if (!made.ok) throw new Error(String(made.error));
75 const repo = getRepo(app.db, "alice", "gazette")!;
76 const record = getAppByRepo(app.db, repo.id)!;
77 const forgeJson = JSON.stringify({
78 app: { run: "bun run src/server.ts" },
79 feature: { name: "Gazette", tools: { print_today: { description: "Print", probe: true } }, schedule: [{ tool: "print_today", at: "07:15", tz: "America/Chicago" }] },
80 });
81 const committed = await commitToBranch(app, alice, repo, { message: "feature", files: [{ path: "forge.json", content: forgeJson }] });
82 if (!committed.ok) throw new Error(JSON.stringify(committed.error));
83 const tip = (await resolveOid(repoDir(app.reposRoot, "alice", "gazette"), `refs/heads/${repo.defaultBranch}`))!;
84 const production = () => runtime.status(record).find((s) => s.env === "production");
85 for (let i = 0; i < 100 && production()?.commitOid !== tip; i++) {
86 await Bun.sleep(100); // real-timer: waits for the runtime to deploy the commit
87 }
88 expect(production()?.commitOid).toBe(tip);
89 let probes = 0;
90 const stub = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: (req) => (req.headers.get("x-forge-probe") === "1" ? (probes++, new Response(null, { status: 204 })) : new Response("no", { status: 404 })) });
91 try {
92 const reach: Reach = { ensure: async () => stub.port, identityKey: () => "k", fetch };
93 await syncAll(app, Date.now, reach);
94 expect(probes).toBe(1);
95 await syncAll(app, Date.now, reach);
96 expect(probes).toBe(1);
97 } finally {
98 stub.stop(true);
99 }
100 }, 30_000);

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs