Public: anyone can read this; the crew can change it.

The manifest a page serves lists only the tools its caller's role may use, and a test holds it under a stated size #13501

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/13486-the-manifest-a-page-serves-lists-only-th → main 2 commits fork 3 threads · 3 unresolved

  • Merged. scout/13486-the-manifest-a-page-serves-lists-only-th was merged into main as b85d599.

Review

Sign in to review this change.
1 file changed +104 −0
test/manifest-roles.test.ts added +104
@@ -0,0 +1,104 @@
1 /**
2 * An agent that reads a page learns what it may do, cheaply (#13486): the
3 * manifest omits every tool its caller cannot call, and its size per role
4 * is held under a ceiling written here. Raise a ceiling only with a reason
5 * in the pull; the cheap answer is the point. It calls `manifestFor` directly:
6 * the served route and the per-role lists are held by here-roles.test.ts.
7 */
8 import { afterAll, beforeAll, expect, test } from "bun:test";
9 import { mkdtemp, rm } from "node:fs/promises";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12 import { newApp } from "../src/actions/apps.ts";
13 import { manifestFor } from "../src/actions/here.ts";
14 import { createApp, type App } from "../src/app.ts";
15 import { createUser, type User } from "../src/core/users.ts";
16 import { addTeamMember, createOrg, createTeam } from "../src/core/orgs.ts";
17 import { promote } from "./helpers/members.ts";
18
19 /** Characters of the manifest's JSON, per role, on one public app's page. */
20 const CEILINGS = { outsider: 1_400, writer: 1_500, admin: 1_600 } as const;
21 /** What an outsider may be offered on a repository: read, report, offer, talk. */
22 const OUTSIDER_MAY = new Set([
23 "read",
24 "report a problem",
25 "open an issue",
26 "comment",
27 "offer a change",
28 "open the app",
29 "make it yours",
30 ]);
31 /** Verbs that act on the repository as a member. */
32 const MEMBER_ONLY = ["tell the crew", "approve a pull", "merge a pull"];
33 const WRITER_MAY = new Set([...OUTSIDER_MAY, ...MEMBER_ONLY]);
34 const ADMIN_MAY = new Set([...WRITER_MAY, "change settings"]);
35
36 let dataDir = "";
37 let app: App;
38 const who: Record<string, User> = {};
39
40 beforeAll(async () => {
41 dataDir = await mkdtemp(join(tmpdir(), "forge-manifest-roles-"));
42 app = createApp({ dataDir, port: 0 });
43 const mk = async (name: string, first = false) => {
44 const made = await createUser(app.db, {
45 origin: "seed", trusted: true, username: name, email: `${name}@forge.local`,
46 password: "a-good-passphrase", adminIfFirst: first,
47 });
48 if (!made.ok) throw new Error(String(made.error));
49 return made.value;
50 };
51 who["boss"] = await mk("boss", true);
52 who["wren"] = await mk("wren");
53 who["wes"] = await mk("wes");
54 who["nina"] = await mk("nina");
55 promote(app.db, ...Object.values(who));
56 const org = createOrg(app.db, "acme", who["wren"]!.id);
57 if (!org.ok) throw new Error(String(org.error));
58 const team = createTeam(app.db, org.value.id, "Writers", "write");
59 if (!team.ok) throw new Error(String(team.error));
60 addTeamMember(app.db, team.value, who["wes"]!.id);
61 const made = await newApp(app, who["wren"]!, { name: "pub", owner: "acme", visibility: "public" });
62 if (!made.ok) throw new Error(String(made.error));
63 }, 60_000);
64
65 afterAll(async () => {
66 app.close();
67 await rm(dataDir, { recursive: true, force: true });
68 });
69
70 function manifest(viewer: User | undefined) {
71 const m = manifestFor(app, viewer, undefined, "https://forge.test", "/acme/pub");
72 if (!m.ok) throw new Error("not_found");
73 return m.value;
74 }
75 const size = (viewer: User | undefined) => JSON.stringify(manifest(viewer)).length;
76 const verbs = (viewer: User | undefined) => manifest(viewer).can.map((a) => a.do);
77
78 test("a caller is offered only the tools its role may call", () => {
79 const within = (viewer: User | undefined, may: Set<string>) =>
80 verbs(viewer).filter((v) => !may.has(v));
81 // anonymous and signed-in outsiders: an allow-list, so an extra tool fails
82 expect(within(undefined, OUTSIDER_MAY)).toEqual([]);
83 expect(within(who["nina"], OUTSIDER_MAY)).toEqual([]);
outdated

Allow-list is applied to the anonymous caller only. Loop over [undefined, who["nina"]] so a signed-in outsider is held to OUTSIDER_MAY too.

84 expect(within(who["wes"], WRITER_MAY)).toEqual([]);
85 expect(within(who["wren"], ADMIN_MAY)).toEqual([]);
86 expect(verbs(undefined)).toContain("report a problem");
87 expect(verbs(undefined)).toContain("offer a change");
outdated

The writer has only a spot check here. Assert the writer's verbs are a subset of an explicit WRITER_MAY set so an unwanted tool fails.

88 expect(verbs(who["wes"])).toContain("merge a pull");
89 expect(verbs(who["wes"])).not.toContain("change settings");
90 expect(verbs(who["wren"])).toContain("change settings");
91 });
92
93 test("the manifest stays under its ceiling for each role", () => {
94 const sizes = {
95 outsider: size(undefined),
96 writer: size(who["wes"]),
97 admin: size(who["wren"]),
98 };
99 for (const role of Object.keys(CEILINGS) as (keyof typeof CEILINGS)[])
100 expect(
101 sizes[role],
102 `the ${role} manifest is ${sizes[role]} characters, over its ${CEILINGS[role]} ceiling`,
103 ).toBeLessThanOrEqual(CEILINGS[role]);
outdated

Strict size ordering is brittle and not what the issue asks. Drop it, or justify it.

104 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs