023e5e1 · d0c218c · d7ef01e · 91cdf0f
@@ -1248,35 +1248,31 @@
given up.
### The sentinel
The forge hunts its own faults (`docs/CREW.md`): a hunt an hour on the
autonomy clock, sooner when a held slot or a setting left away from its
default is new. Each finding is one issue with its evidence, filed in
`FORGE_SENTINEL_REPO` (default `trevato/forge`) as the `sentinel` agent, else
the first admin. Evidence is scrubbed of anything credential-shaped before it
is filed.
autonomy clock, sooner when a held slot is new. Each finding is one issue with
its evidence, filed in `FORGE_SENTINEL_REPO` (default `trevato/forge`) as the
`sentinel` agent, else the first admin. Evidence is scrubbed of anything
credential-shaped before it is filed. Other findings (`held`, `ledger`, `red`)
are filed anew when they return.
A setting kept on purpose is a decision, not drift. Closing a `setting:<key>`
finding's issue, or commenting on it (a person, not the filer and not an agent), records the value
and its `updated_at` in `sentinel_kept`; while both match, the sentinel files
nothing and comments nothing for that key. Changing the value on the capacity
page ends the keeping, and the setting is reported again under the usual
one-day rule. The rule is mechanical: any close counts, with or without a
reason, so a bare close silences the setting until its value changes. Other
findings (`held`, `ledger`, `red`) are filed anew when they return.
A capacity setting away from its default is not a finding: the sentinel files
nothing for it (#3269). On `/settings/capacity` the row of each such setting
carries one muted line, "changed from X N days ago by who", and its **Put back**
button (in the row's own form, beside Save) restores the default through
`changeCapacity` (same validation, record, event and reversal). The line stays
until the value is back at the default; nothing else reminds anyone. The
JSON API and MCP carry the same fact: a setting away from its default has a
`changed` field (`at`, `by`) in the capacity listing (`listCapacity`).
A setting finding is the site admin's ask, never a crew run: it is not
assigned, and the autonomy clock leaves it out of the crew's ready list (`crew/setting_finding.ts`). It shows on the home (kind `setting`,
one card per key: the value, its default, who set it and when) and in the
terminal's On you card, with two answers — **Keep** and **Put back**. On the
web they are `POST /asks/setting` (`key`, `answer=keep|put`, same-origin only);
in the terminal, `K` keeps and `B` puts back the row under the cursor. Only a
signed-in site admin can answer, whatever they can write: anyone else, and any
agent, gets `not_found`/`unauthorized` and nothing changes. Keep closes the
finding and records the value in `sentinel_kept`; Put back goes through the
capacity page's own door (`changeCapacity`: same validation, record, event and
reversal). The close is the gate: Put back changes the setting, and Keep records
the value, only after a finding has closed. If none can (every one is on a
mirrored repository, or the close is refused), nothing changes and the page and
the terminal say so (`unclosable`). Only a person's close keeps a setting.
A `setting:<key>` finding filed before that is closed on the next pass with a
comment pointing at the capacity page (`closeSettingFindings` in
`crew/setting_finding.ts`). Until it closes, it is the site admin's ask, never a
crew run: not assigned, left out of the crew's ready list, and shown on the home
(kind `setting`) and in the terminal's On you card with **Keep** and **Put
back** (`POST /asks/setting`, `key`, `answer=keep|put`, same-origin; `K` and `B`
in the terminal). Only a signed-in site admin can answer. Keep and Put back act
only after the finding has closed; if none can (a mirrored repository, or the
close is refused) nothing changes and the page says so (`unclosable`). This is
the legacy path, kept until no open setting finding is left.
- **Board:** a site admin's forge strip shows the open findings and the last
  hunt; a guest or member sees nothing.
@@ -39,10 +39,9 @@
the `denyRead` list) and not the boundary itself.
The forge hunts its own faults. `src/crew/sentinel.ts` runs on the autonomy
clock (a hunt an hour; a signal may bring one sooner, never within ten
minutes). It reads what the forge already holds — the run table, the decision
ledger (`retry stop` streaks, `guard` parks, a direction at its cap), capacity
settings away from their defaults for more than a day, `sqlite_master` for
triggers the schema does not create, and the error ring — and names each fault
with the evidence quoted (secret-shaped text scrubbed).
ledger (`retry stop` streaks, `guard` parks, a direction at its cap),
`sqlite_master` for triggers the schema does not create, and the error ring —
and names each fault with the evidence quoted (secret-shaped text scrubbed).
Each finding is exactly one open issue, keyed so it is never filed twice; a
finding still there is a comment, and only when the evidence changed. The board
@@ -5,7 +5,8 @@
the forge already holds and files what is wrong, with the evidence quoted.
**When.** On the autonomy clock: a hunt an hour, and one sooner (never within
ten minutes of the last) when a signal is new — queued checks waiting while no
run uses a slot, or a capacity setting away from its default for over a day.
run uses a slot. A capacity setting away from its default is no signal and
no finding: it is a line on `/settings/capacity` (#3269).
**What it reads.**
@@ -14,7 +15,6 @@
run uses a slot, or a capacity setting away from its default for over a day.
| a slot held by a finished run | `run`: queued work older than 30 min, nothing running or every lease lapsed |
| a check never green; pushes that cannot land | the last 20 `check` / `agent` run summaries per repository |
| `retry` stops, guard parks | the decision ledger (`decisionsSince`) |
| a setting left away from default | `forge_setting` against `capacityDefault` |
| a hand-installed trigger | `sqlite_master` against a scratch database migrated from the schema |
| the same ERROR line again and again | the error ring (`kernel/log.ts`) |
@@ -7,6 +7,7 @@
 * an event: the key, from what to what, and who.
 */
import type { Database } from "bun:sqlite";
import type { App } from "../app.ts";
import {
  CAPACITY,
@@ -34,6 +35,8 @@
export interface CapacityRow {
  readonly source: "setting" | "default";
  readonly default: string;
  readonly env: string;
  /** For a setting away from its default: when and by whom it was set. */
  readonly changed?: { readonly at: string; readonly by: string };
}
function admin(actor: User | undefined): Result<User, "not_found"> {
@@ -42,6 +45,18 @@
function admin(actor: User | undefined): Result<User, "not_found"> {
    : err("not_found");
}
/** Who set `key` and when, only while its stored value is not the default. */
export function changedOf(db: Database, key: CapacityKey): { changed?: { at: string; by: string } } {
  const value = storedCapacity(db, key);
  if (value === undefined || value === bootDefault(db, key)) return {};
  const row = db
    .query<{ updated_at: string; username: string | null }, [string]>(
      "SELECT s.updated_at, u.username FROM forge_setting s LEFT JOIN user u ON u.id = s.updated_by WHERE s.key = ?1",
    )
    .get(key);
  return row === null ? {} : { changed: { at: row.updated_at, by: row.username ?? "someone" } };
}
export function listCapacity(
  app: App,
  actor: User | undefined,
@@ -57,6 +72,7 @@
export function listCapacity(
      source: storedCapacity(app.db, key) === undefined ? "default" : "setting",
      default: bootDefault(app.db, key),
      env: CAPACITY[key].env,
      ...changedOf(app.db, key),
    })),
    changes: capacityChanges(app.db),
  });
@@ -16,11 +16,6 @@
import { Database } from "bun:sqlite";
import type { App } from "../app.ts";
import { addComment, openIssue, setIssueState } from "../actions/issues.ts";
import {
  CAPACITY_KEYS,
  capacityDefault,
  storedCapacity,
} from "../core/capacity.ts";
import { decisionsSince } from "../core/decisions.ts";
import { getAgentByName } from "../core/agents.ts";
import { getIssueById, getLabel, setIssueLabels as writeLabels, type Issue } from "../core/issues.ts";
@@ -34,6 +29,7 @@
import { spans } from "../kernel/degraded_history.ts";
import { QUEUE_WAIT_MAX_MS } from "../kernel/vitals.ts";
import { STALE_AFTER_MS } from "../runs/lease.ts";
import { isOutageSummary } from "./network.ts";
import { closeSettingFindings } from "./setting_finding.ts";
const log = createLog("sentinel");
@@ -42,8 +38,6 @@
const DAY_MS = 24 * HOUR_MS;
/** A hunt an hour; a signal may bring one sooner, but never within this. */
export const HUNT_EVERY_MS = HOUR_MS;
export const SIGNAL_GAP_MS = 10 * 60_000;
/** A setting away from its default this long is a finding. */
export const SETTING_AWAY_MS = DAY_MS;
/** A cause degraded longer than this in its day is a finding. */
export const DEGRADED_DAY_MS = 10 * 60_000;
/** How many alike make a streak. */
@@ -63,8 +57,6 @@
export interface Finding {
  readonly urgent?: true;
  /** What a person or a run would do, when it is mechanical and reversible. */
  readonly fix?: string;
  /** A setting finding: the value and `updated_at` a person may have kept (#2001). */
  readonly kept?: { readonly value: string; readonly updatedAt: string };
  /** A degraded finding: when its cause first opened. Same cause and time, same incident (#2553). */
  readonly incident?: string;
}
@@ -168,37 +160,6 @@
export function heldSlots(db: Database, now: number): Finding[] {
  return out;
}
/** A capacity setting that is not its default, for more than a day. */
export function settingsAway(db: Database, now: number): Finding[] {
  const out: Finding[] = [];
  for (const key of CAPACITY_KEYS) {
    const value = storedCapacity(db, key);
    if (value === undefined || value === capacityDefault(key)) continue;
    const row = db
      .query<
        { updated_at: string; username: string | null },
        [string]
      >("SELECT s.updated_at, u.username FROM forge_setting s LEFT JOIN user u ON u.id = s.updated_by WHERE s.key = ?1")
      .get(key);
    if (row === null || now - new Date(row.updated_at).getTime() < SETTING_AWAY_MS)
      continue;
    const days = Math.floor((now - new Date(row.updated_at).getTime()) / DAY_MS);
    out.push({
      key: `setting:${key}`,
      title: `Setting ${key} has been away from its default for ${days} days`,
      evidence: [
        `${key} = "${value}", default "${capacityDefault(key)}"`,
        `set by ${row.username ?? "unknown"} at ${row.updated_at}`,
      ],
      pillar: "Trust: a setting is a decision, not a leftover",
      lookFor: `${key} is back at "${capacityDefault(key)}", or a person says why it stays: the owner is asked in the inbox, Keep or Put back, and a person's close or comment keeps it, reported again only if its value changes. No agent is assigned; a run cannot change a setting.`,
      kept: { value, updatedAt: row.updated_at },
      fix: `Put it back: clear ${key} on the capacity page (it is recorded and reversible).`,
    });
  }
  return out;
}
/** The crew's stops and guard parks, read from the decision ledger. */
export function ledgerStreaks(db: Database, now: number): Finding[] {
  const out: Finding[] = [];
@@ -486,7 +447,6 @@
export function findFaults(db: Database, now = Date.now()): Finding[] {
  const found = [
    ...budgetBreaches(db, now),
    ...heldSlots(db, now),
    ...settingsAway(db, now),
    ...strayTriggers(db),
    ...ledgerStreaks(db, now),
    ...runStreaks(db),
@@ -583,43 +543,6 @@
function labelBreach(app: App, repo: Repo, actor: User, issue: Issue): void {
  }
}
/** The finding is a setting whose kept value and `updated_at` are on record. */
function keptRecorded(db: Database, f: Finding): boolean {
  if (f.kept === undefined) return false;
  const kept = db
    .query<{ value: string; updated_at: string }, [string]>("SELECT value, updated_at FROM sentinel_kept WHERE key = ?1")
    .get(f.key);
  return kept !== null && kept.value === f.kept.value && kept.updated_at === f.kept.updatedAt;
}
/**
 * Whether a person decided to keep this setting as it now stands: its
 * finding was closed, or explained in a comment by a person (not the sentinel, not an agent),
 * after the value was last set. A decision is recorded with the value and
 * `updated_at` it was made at; the same pair is kept, any other is drift.
 */
function settingKept(db: Database, f: Finding, actorId: string, at: string): boolean {
  if (f.kept === undefined) return false;
  if (keptRecorded(db, f)) return true;
  const { value, updatedAt } = f.kept;
  const decided = db
    .query<{ n: number }, [string, string, string]>(
      `SELECT COUNT(*) AS n FROM sentinel_finding t JOIN issue i ON i.id = t.issue_id
        WHERE t.key = ?1 AND ((i.state = 'closed' AND i.closed_at >= ?2
            AND (SELECT e.actor_id FROM event e
                  WHERE e.kind = 'issue.closed' AND e.repo_id = i.repo_id
                    AND json_extract(e.payload, '$.number') = i.number
                  ORDER BY e.id DESC LIMIT 1)
                IN (SELECT id FROM user WHERE is_agent = 0 AND kind = 'user'))
          OR EXISTS (SELECT 1 FROM comment c WHERE c.issue_id = i.id AND c.author_id <> ?3 AND c.created_at >= ?2
            AND EXISTS (SELECT 1 FROM user u WHERE u.id = c.author_id AND u.is_agent = 0 AND u.kind = 'user')))`,
    )
    .get(f.key, updatedAt, actorId);
  if (decided === null || decided.n === 0) return false;
  recordKept(db, f.key, value, updatedAt, at);
  return true;
}
/** Remember that a person kept `key` as it stands: this value, set at `updatedAt`. */
export function recordKept(db: Database, key: string, value: string, updatedAt: string, at: string): void {
  db.query(
@@ -720,8 +643,8 @@
function closeHeld(app: App, repo: Repo, actor: User, active: ReadonlySet<string
/**
 * Read, file, record. One open issue per finding: a tracked finding whose
 * issue is open is commented on only when its evidence changed; one whose
 * issue is closed and returns is a new issue — except a setting a person
 * kept (`settingKept`), which stays quiet until its value changes.
 * issue is closed and returns is a new issue. A setting away from its default
 * is no finding: the capacity page says so beside it (#3269).
 */
export function hunt(
  app: App,
@@ -738,7 +661,6 @@
export function hunt(
  if (repo !== undefined && actor !== undefined) {
    for (const f of findings) {
      try {
        if (settingKept(app.db, f, actor.id, at)) continue;
        const evidence = f.evidence.join("\n");
        const tracked = app.db
          .query<
@@ -787,7 +709,7 @@
export function hunt(
          }
          issueId = made.value.id;
          filed++;
          if (f.urgent && f.kept === undefined) labelBreach(app, repo, actor, made.value);
          if (f.urgent) labelBreach(app, repo, actor, made.value);
        }
        app.db
          .query("INSERT INTO sentinel_finding (key, issue_id, evidence, first_seen, last_seen, incident) VALUES (?1, ?2, ?3, ?4, ?4, ?5)")
@@ -797,8 +719,10 @@
export function hunt(
      }
    }
  }
  if (repo !== undefined && actor !== undefined)
  if (repo !== undefined && actor !== undefined) {
    closeHeld(app, repo, actor, new Set(findings.map((f) => f.key)), now);
    closeSettingFindings(app, repo, actor);
  }
  // A hunt with nowhere to file is recorded as such, never as a quiet one.
  const unfiled = repo === undefined || actor === undefined;
  app.db
@@ -856,8 +780,7 @@
export function sentinelSummary(db: Database): string | null {
}
/**
 * Hunt when it is time: an hour since the last, or a signal (a slot held, a
 * setting away) not yet tracked, and not within ten minutes of the last.
 * Hunt when it is time: an hour since the last, or a signal (a slot held) not yet tracked, and not within ten minutes of the last.
      
        junipercrew
        24d ago
        
      
      Docs (README sentinel section, docs/SENTINEL.md, docs/CREW.md) still describe the removed setting finding and its sooner hunt; update them.
    
 */
export function sentinelTick(app: App, now = Date.now()): HuntResult | null {
  const repoPath = sentinelRepoPath();
@@ -869,7 +792,7 @@
export function sentinelTick(app: App, now = Date.now()): HuntResult | null {
    // A breach is not held back by the gap: the crew hears within a tick.
    const signals = [
      ...breaches,
      ...(since < SIGNAL_GAP_MS ? [] : [...heldSlots(app.db, now), ...settingsAway(app.db, now)]),
      ...(since < SIGNAL_GAP_MS ? [] : heldSlots(app.db, now)),
    ];
    const tracked = new Set(
      app.db
@@ -879,7 +802,7 @@
export function sentinelTick(app: App, now = Date.now()): HuntResult | null {
        .all()
        .map((r) => r.key),
    );
    if (!signals.some((s) => !tracked.has(s.key) && !keptRecorded(app.db, s))) return null;
    if (!signals.some((s) => !tracked.has(s.key))) return null;
  }
  return hunt(app, repoPath, now);
}
@@ -5,6 +5,14 @@
 */
import type { Database } from "bun:sqlite";
import type { App } from "../app.ts";
import { addComment, setIssueState } from "../actions/issues.ts";
import { getIssueById } from "../core/issues.ts";
import type { Repo } from "../core/repos.ts";
import type { User } from "../core/users.ts";
import { createLog } from "../kernel/log.ts";
const log = createLog("sentinel");
/** Whether this issue is the open finding of a `setting:<key>` the sentinel reported. */
export function isSettingFinding(db: Database, issueId: string): boolean {
@@ -16,3 +24,33 @@
export function isSettingFinding(db: Database, issueId: string): boolean {
      .get(issueId) !== null
  );
}
/**
 * A setting away from its default is a line on the capacity page, not an
 * issue (#3269). Findings filed before that are closed, with a pointer.
 */
export function closeSettingFindings(app: App, repo: Repo, actor: User): void {
  const open = app.db
    .query<{ issue_id: string }, []>(
      `SELECT DISTINCT t.issue_id FROM sentinel_finding t JOIN issue i ON i.id = t.issue_id
        WHERE i.state = 'open' AND t.key LIKE 'setting:%'`,
    )
    .all();
  for (const t of open) {
    try {
      const issue = getIssueById(app.db, t.issue_id);
      if (issue === undefined || issue.repoId !== repo.id) continue;
      const said = addComment(
        app,
        actor,
        repo,
        issue,
        "A setting away from its default is now one line beside it on the capacity page (/settings/capacity), with Put back; the sentinel files no issue for it. Closing.",
      );
      if (!said.ok) continue;
      setIssueState(app, actor, repo, issue, "closed", { reason: "done" });
    } catch (error) {
      log.error("sentinel could not close a setting finding", { issue: t.issue_id, error: String(error) });
    }
  }
}
@@ -6,7 +6,7 @@
 * and each one is listed below the settings: who, when, from what to what.
 */
import { changeCapacity, listCapacity } from "../../actions/capacity.ts";
import { changeCapacity, listCapacity, type CapacityRow } from "../../actions/capacity.ts";
import { CAPACITY, type CapacityKey } from "../../core/capacity.ts";
import type { Ctx } from "../../http/context.ts";
import { notFoundPage, signInFirst } from "../chrome.ts";
@@ -22,6 +22,22 @@
function requireAdmin(ctx: Ctx): Response | undefined {
  return undefined;
}
const DAY_MS = 86_400_000;
/** "today", "1 day ago", "9 days ago". */
export function daysAgo(at: string, now: number): string {
  const days = Math.max(0, Math.floor((now - Date.parse(at)) / DAY_MS));
  return days === 0 ? "today" : `${days} day${days === 1 ? "" : "s"} ago`;
}
/** One muted line beside a setting away from its default; Put back is in the row's trail form. */
function awayLine(s: CapacityRow, now: number) {
  if (s.changed === undefined) return null;
  return html`<span class="muted small">changed from
    <span class="mono">${s.default === "" ? "auto" : s.default}</span>
    ${daysAgo(s.changed.at, now)} by ${s.changed.by}</span>`;
      
        junipercrew
        24d ago
        outdated
      
      row-fields is the trail form class in DESIGN.md; here it wraps a line inside the row meta. Move Put back into the trail form, or document this as a component.
    
}
function input(key: CapacityKey, value: string) {
  const kind = CAPACITY[key].kind;
  if (kind.type === "choice")
@@ -55,6 +71,7 @@
export function capacityPage(ctx: Ctx): Response {
  const listed = listCapacity(ctx.app, ctx.auth.user);
  if (!listed.ok) return notFoundPage(ctx);
  const { settings, changes } = listed.value;
  const now = Date.now();
  const rows = settings.map((s) =>
    row({
      lead: icon("activity"),
@@ -63,7 +80,8 @@
export function capacityPage(ctx: Ctx): Response {
          ? html`Set here; the boot default is
              <span class="mono">${s.default === "" ? "auto" : s.default}</span>
              (<span class="mono">${s.env}</span>).`
          : html`The boot default, from <span class="mono">${s.env}</span>.`}`,
          : html`The boot default, from <span class="mono">${s.env}</span>.`}
        ${awayLine(s, now)}`,
      trail: html`<form
        class="row-fields"
        method="post"
@@ -79,11 +97,11 @@
export function capacityPage(ctx: Ctx): Response {
        ${when(
          s.source === "setting",
          () =>
            html`${btn("Default", {
            html`${btn(s.changed === undefined ? "Default" : "Put back", {
              size: "sm",
              name: "reset",
              value: "1",
              label: `Back to the default for ${s.label}`,
              label: s.changed === undefined ? `Back to the default for ${s.label}` : `Put ${s.label} back to the default`,
            })}`,
        )}
      </form>`,
@@ -14,7 +14,8 @@
import { openPull } from "../src/actions/pulls.ts";
import { commitToBranch, createRepo } from "../src/actions/repos.ts";
import { TESTS_PASSED_TAIL } from "../src/apps/runtime.ts";
import { recordDecision, resolveOpen } from "../src/core/decisions.ts";
import type { Repo } from "../src/core/repos.ts";
import { createIssue } from "../src/core/issues.ts";
import { getRepo, type Repo } from "../src/core/repos.ts";
import { claimRun, createRun, finishRun } from "../src/core/runs.ts";
import { getUserByName } from "../src/core/users.ts";
import { markVerified } from "../src/core/verify.ts";
@@ -283,6 +284,26 @@
describe("the asks page", () => {
        .query("UPDATE forge_setting SET updated_at = ?2 WHERE key = ?1")
        .run(key, new Date(Date.now() - 10 * 86_400_000).toISOString());
    };
    // The sentinel no longer files a setting (#3269); an issue it filed
    // before still waits for an answer, so these tests plant one.
    const fileLegacy = (path: string): void => {
      const [owner = "", name = ""] = path.split("/");
      const target = getRepo(app.db, owner, name)!;
      const author = getUserByName(app.db, "alice")!;
      const rows = app.db.query<{ key: string }, []>("SELECT key FROM forge_setting").all();
      for (const { key } of rows) {
        const made = createIssue(app.db, {
          repoId: target.id,
          authorId: author.id,
          title: `Sentinel: Setting ${key} has been away from its default for 10 days`,
          body: `${key} is away`,
        });
        if (!made.ok) throw new Error(made.error);
        app.db
          .query("INSERT INTO sentinel_finding (key, issue_id, evidence, first_seen, last_seen) VALUES (?1, ?2, '', 'a', 'a')")
          .run(`setting:${key}`, made.value.id);
      }
    };
    const post = (cookie: string, fields: Record<string, string>, origin = servers.url) =>
      fetch(`${servers.url}/asks/setting`, {
        method: "POST",
@@ -305,8 +326,8 @@
describe("the asks page", () => {
      reset();
      away("checks.concurrency", "4");
      away("cells.egress", "report");
      hunt(app, "alice/garden");
      hunt(app, "alice/garden", Date.now() + 3 * 3_600_000);
      fileLegacy("alice/garden");
      fileLegacy("alice/garden");
      expect(settingCards().sort()).toEqual(["setting:cells.egress", "setting:checks.concurrency"]);
      const card = decisionsFor({ app, auth: { user: getUserByName(app.db, "alice")! } } as unknown as Ctx, [repo]).find(
        (d) => d.id === "setting:checks.concurrency",
@@ -323,12 +344,11 @@
describe("the asks page", () => {
      expect(listEvents(app.db, { kind: "issue.assigned" }).length).toBe(0);
    });
    test("Keep closes the finding as kept, and it stays quiet", async () => {
    test("Keep closes the finding as kept", async () => {
      const res = await post(alice, { key: "checks.concurrency", answer: "keep" });
      expect(res.status).toBe(303);
      expect(settingCards()).toEqual(["setting:cells.egress"]);
      expect(app.db.query("SELECT value FROM sentinel_kept WHERE key = 'setting:checks.concurrency'").get()).toEqual({ value: "4" });
      expect(hunt(app, "alice/garden", Date.now() + 5 * 3_600_000).filed).toBe(0);
      expect(storedCapacity(app.db, "checks.concurrency")).toBe("4");
      const closed = app.db
        .query<{ close_reason: string }, []>("SELECT close_reason FROM issue WHERE title LIKE 'Sentinel: Setting checks.concurrency %'")
@@ -348,7 +368,7 @@
describe("the asks page", () => {
    test("a stranger, a bad answer and a foreign origin change nothing", async () => {
      reset();
      away("cells.egress", "report");
      hunt(app, "alice/garden");
      fileLegacy("alice/garden");
      await post(bob, { key: "cells.egress", answer: "put" });
      await post(alice, { key: "cells.egress", answer: "burn" });
      await post(alice, { key: "nope", answer: "put" });
@@ -361,7 +381,7 @@
describe("the asks page", () => {
    test("Put back changes nothing when no finding can be closed, and says why", async () => {
      reset();
      away("cells.egress", "report");
      hunt(app, "alice/garden");
      fileLegacy("alice/garden");
      const alice0 = getUserByName(app.db, "alice")!;
      const made = createOrigin(app.db, dataDir, {
        userId: alice0.id,
@@ -392,7 +412,7 @@
describe("the asks page", () => {
    test("Put back changes nothing when the close is refused for a reason other than a mirror", () => {
      reset();
      away("cells.egress", "report");
      hunt(app, "alice/garden");
      fileLegacy("alice/garden");
      // a pull cannot be closed with an issue's reason, so setIssueState refuses
      app.db.query("UPDATE issue SET kind = 'pull' WHERE title LIKE 'Sentinel: Setting cells.egress %'").run();
      const alice0 = getUserByName(app.db, "alice")!;
@@ -413,7 +433,7 @@
describe("the asks page", () => {
      const plot = await createRepo(app, carolUser, { name: "plot", visibility: "public" });
      if (!plot.ok) throw new Error(String(plot.error));
      away("cells.egress", "report");
      hunt(app, "carol/plot");
      fileLegacy("carol/plot");
      expect(app.db.query("SELECT COUNT(*) AS n FROM sentinel_finding WHERE key = 'setting:cells.egress'").get()).toEqual({ n: 1 });
      await post(carol, { key: "cells.egress", answer: "keep" });
      await post(carol, { key: "cells.egress", answer: "put" });
@@ -20,11 +20,10 @@
import { mergePull, openPull } from "../src/actions/pulls.ts";
import { commitToBranch, createRepo } from "../src/actions/repos.ts";
import { getAppByRepo } from "../src/core/apps.ts";
import { hunt } from "../src/crew/sentinel.ts";
import { setCapacity } from "../src/core/capacity.ts";
import { recordDecision } from "../src/core/decisions.ts";
import { recordLook } from "../src/core/looks.ts";
import { getPull } from "../src/core/pulls.ts";
import { updateRepoSettings, type Repo } from "../src/core/repos.ts";
import { getRepo, updateRepoSettings, type Repo } from "../src/core/repos.ts";
import { claimRun, createRun, finishRun } from "../src/core/runs.ts";
import { submitReview } from "../src/core/reviews.ts";
import { createUser, getUserByName, type User } from "../src/core/users.ts";
@@ -570,10 +569,12 @@
describe("the board's forge strip", () => {
  test("the sentinel's line is the admin's alone, and absent before the first hunt", async () => {
    expect(await board("root")).not.toContain("sentinel");
    setCapacity(app.db, "checks.concurrency", "4", admin.id);
    // A check queued two hours with nothing running: a held slot, a finding.
    const stuck = getRepo(app.db, "root", "stuck")!;
    const queued = createRun(app.db, { repoId: stuck.id, kind: "check" });
    app.db
      .query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'")
      .run("2020-01-01T00:00:00.000Z");
      .query("UPDATE run SET created_at = ?2 WHERE id = ?1")
      .run(queued.id, new Date(Date.now() - 2 * 3_600_000).toISOString());
    process.env["FORGE_SENTINEL_REPO"] = "root/stuck";
    try {
      expect(hunt(app, "root/stuck").filed).toBeGreaterThan(0);
@@ -586,7 +587,7 @@
describe("the board's forge strip", () => {
      expect(guest).not.toMatch(/sentinel/i);
    } finally {
      delete process.env["FORGE_SENTINEL_REPO"];
      setCapacity(app.db, "checks.concurrency", "1", admin.id);
      app.db.query("DELETE FROM run WHERE id = ?1").run(queued.id);
    }
  });
});
@@ -13,6 +13,8 @@
import { join } from "node:path";
import { createApp } from "../src/app.ts";
import {
  CAPACITY,
  CAPACITY_KEYS,
  bootDefault,
  capacityDefault,
  capacityEnv,
  capacityValue,
@@ -22,7 +24,7 @@
import {
  setCapacity,
  shardsForCpus,
} from "../src/core/capacity.ts";
import { settingsAway } from "../src/crew/sentinel.ts";
import { changedOf } from "../src/actions/capacity.ts";
import { createUser } from "../src/core/users.ts";
const ENV = ["FORGE_CHECK_SHARDS", "FORGE_CHECKS_CONCURRENCY"] as const;
@@ -70,7 +72,7 @@
describe("the defaults follow the CPUs", () => {
  });
});
describe("a stored value overrides the default; the sentinel compares with it", () => {
describe("a stored value overrides the default; the capacity page compares with it", () => {
  test("80 CPUs: 4 shards and 6 checks are the defaults, 8 shards is away", async () => {
    delete process.env.FORGE_CHECK_SHARDS;
    delete process.env.FORGE_CHECKS_CONCURRENCY;
@@ -98,15 +100,15 @@
describe("a stored value overrides the default; the sentinel compares with it",
      setCapacity(app.db, "crew.concurrency", "5", a.value.id);
      const old = new Date(Date.now() - 3 * 24 * 3600_000).toISOString();
      app.db.query("UPDATE forge_setting SET updated_at = ?1").run(old);
      expect(settingsAway(app.db, Date.now()).map((f) => f.key)).toEqual(["setting:crew.concurrency"]);
      expect(CAPACITY_KEYS.filter((k) => changedOf(app.db, k).changed !== undefined)).toEqual(["crew.concurrency"]);
      // A stored value overrides the default, even a cell's smaller one.
      setCapacity(app.db, "checks.shards", "8", a.value.id);
      app.db.query("UPDATE forge_setting SET updated_at = ?1").run(old);
      expect(capacityValue(app.db, "checks.shards")).toBe("8");
      expect(capacityEnv(app.db, 2)).toEqual({ FORGE_CHECK_SHARDS: "8" });
      const away = settingsAway(app.db, Date.now()).find((f) => f.key === "setting:checks.shards");
      expect(away?.evidence[0]).toBe('checks.shards = "8", default "4"');
      expect(changedOf(app.db, "checks.shards").changed?.by).toBe("alice");
      expect(bootDefault(app.db, "checks.shards")).toBe("4");
    } finally {
      app.close();
      await rm(dataDir, { recursive: true, force: true });
@@ -19,7 +19,7 @@
import {
  memoryLimitBytes,
  setCapacity,
} from "../src/core/capacity.ts";
import { settingsAway } from "../src/crew/sentinel.ts";
import { changedOf } from "../src/actions/capacity.ts";
const GiB = 1024 ** 3;
@@ -77,13 +77,13 @@
describe("the crew's default, and what overrides it", () => {
    expect(capacityValue(app.db, "crew.concurrency")).toBe(capacityDefault("crew.concurrency"));
  });
  test("the sentinel flags a value away from the memory default, not one at it", () => {
  test("the capacity page marks a value away from the memory default, not one at it", () => {
    const now = Date.now();
    const age = () =>
      app.db
        .query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'crew.concurrency'")
        .run(new Date(now - 3 * 86_400_000).toISOString());
    const crew = () => settingsAway(app.db, now).filter((f) => f.key === "setting:crew.concurrency");
    const crew = () => (changedOf(app.db, "crew.concurrency").changed === undefined ? [] : ["changed"]);
    const fallback = capacityDefault("crew.concurrency");
    setCapacity(app.db, "crew.concurrency", fallback, "system");
    age();
@@ -91,8 +91,6 @@
describe("the crew's default, and what overrides it", () => {
    const other = fallback === "5" ? "6" : "5";
    setCapacity(app.db, "crew.concurrency", other, "system");
    age();
    const found = crew();
    expect(found).toHaveLength(1);
    expect(found[0]!.evidence[0]).toBe(`crew.concurrency = "${other}", default "${fallback}"`);
    expect(crew()).toEqual(["changed"]);
  });
});
@@ -21,6 +21,7 @@
import { enableApp } from "../src/actions/apps.ts";
import { AppsRuntime } from "../src/apps/runtime.ts";
import { startChecks, type ChecksHandle } from "../src/checks/service.ts";
import {
  CAPACITY,
  capacityChanges,
  capacityCount,
  capacityDefault,
@@ -393,6 +394,26 @@
describe("report mode is loud to admins, and to nobody else", () => {
    expect(api).not.toContain("gress");
  });
  test("away from its default: one muted line with Put back, escaped; none for a default (#3269)", async () => {
    for (const k of ["checks.shards", "cells.egress", "checks.concurrency"]) changeCapacity(app, alice, k, null);
    changeCapacity(app, alice, "crew.concurrency", null);
    const other = capacityDefault("crew.concurrency") === "5" ? "6" : "5";
    expect(changeCapacity(app, alice, "crew.concurrency", other).ok).toBe(true);
    app.db.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'crew.concurrency'").run(new Date(Date.now() - 3 * 86_400_000).toISOString());
    app.db.query("UPDATE forge_setting SET updated_by = ?1 WHERE key = 'crew.concurrency'").run(bob.id);
    app.db.query("UPDATE user SET username = ?2 WHERE id = ?1").run(bob.id, "bo<b>b");
    const shown = (await page()).replace(/\s+/g, " ");
    expect(shown).toContain(`changed from <span class="mono">${capacityDefault("crew.concurrency")}</span> 3 days ago by bo&lt;b&gt;b`);
    expect(shown).not.toContain("bo<b>b");
    expect(shown.match(/changed from/g)).toHaveLength(1);
    expect(shown).toContain('<span class="muted small">changed from');
    expect(shown).not.toContain('class="away"');
    expect(shown).toContain(`Put ${CAPACITY["crew.concurrency"].label} back to the default`);
    app.db.query("UPDATE user SET username = 'bob' WHERE id = ?1").run(bob.id);
    expect(changeCapacity(app, alice, "crew.concurrency", null).ok).toBe(true);
    expect(await page()).not.toContain("changed from");
  });
  test("enforced: no warning anywhere", async () => {
    expect(changeCapacity(app, alice, "cells.egress", "enforce").ok).toBe(true);
    expect(await page()).not.toContain('id="egress-reported"');
@@ -20,6 +20,7 @@
import { listEvents } from "../src/core/events.ts";
import { getRepoById, type Repo } from "../src/core/repos.ts";
import { createRun } from "../src/core/runs.ts";
import { createUser, type User } from "../src/core/users.ts";
import { isSettingFinding } from "../src/crew/setting_finding.ts";
import { recordCap } from "../src/crew/guard.ts";
import {
  budgetBreaches,
@@ -194,7 +195,7 @@
describe("the faults of the day", () => {
    // 3. pushes that can only succeed by leaving the sandbox.
    for (let n = 0; n < 3; n++)
      plantRun("agent", "failed", "git push: operation not permitted", ago((20 + n) * HOUR));
    // 4. a setting away from its default for a week.
    // 4. a setting away from its default for a week: a line on the capacity page, no finding (#3269).
    setCapacity(app.db, "checks.concurrency", "4", alice.id);
    app.db
      .query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'")
@@ -212,7 +213,6 @@
describe("the faults of the day", () => {
      [
        `gate:${repo.id}`,
        `push:${repo.id}`,
        "setting:checks.concurrency",
        "slot:check",
        "trigger:steward_guard_repo",
      ].sort(),
@@ -222,7 +222,7 @@
describe("the faults of the day", () => {
    expect(byKey.get(`gate:${repo.id}`)!.evidence.join("\n")).toContain(
      "nested cell: no network namespace",
    );
    expect(byKey.get("setting:checks.concurrency")!.evidence[0]).toContain('"4", default "1"');
    expect(byKey.has("setting:checks.concurrency")).toBe(false);
    expect(byKey.get("trigger:steward_guard_repo")!.evidence.join("\n")).toContain(
      "paused for new accounts",
    );
@@ -230,19 +230,19 @@
describe("the faults of the day", () => {
  test("a hunt files each once, and a second hunt files nothing", () => {
    const first = hunt(app, "alice/forge", NOW);
    expect(first.filed).toBe(5);
    expect(first.filed).toBe(4);
    const open = () => listIssues(app.db, repo.id, { state: "open", limit: 200 });
    expect(open().length).toBe(5);
    expect(open().length).toBe(4);
    const slot = open().find((i) => i.title.includes("wait with no slot"))!;
    expect(slot.body).toContain("**Evidence**");
    expect(slot.body).toContain("**What to look for when fixed:**");
    expect(slot.body).toContain("**Pillar:**");
    const second = hunt(app, "alice/forge", NOW + 30 * 60_000);
    expect(second).toMatchObject({ found: 5, filed: 0, commented: 0 });
    expect(open().length).toBe(5);
    expect(sentinelStanding(app.db).openFindings).toBe(5);
    expect(sentinelLine(app.db)).toBe("The sentinel has 5 open findings");
    expect(second).toMatchObject({ found: 4, filed: 0, commented: 0 });
    expect(open().length).toBe(4);
    expect(sentinelStanding(app.db).openFindings).toBe(4);
    expect(sentinelLine(app.db)).toBe("The sentinel has 4 open findings");
  });
  test("a finding whose evidence changed is a comment, not a new issue", () => {
@@ -477,137 +477,29 @@
describe("the signals inside the hour", () => {
    }
  };
  test("a new setting away hunts after ten minutes; tracked, it does not", () => {
  test("a setting away from its default brings no hunt and files nothing (#3269)", () => {
    setCapacity(app.db, "cells.egress", "report", alice.id);
    app.db
      .query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'cells.egress'")
      .run(ago(10 * 24 * HOUR));
    const l = lastHunt();
    expect(withRepo(() => sentinelTick(app, l + 5 * 60_000))).toBeNull();
    const r = withRepo(() => sentinelTick(app, l + 20 * 60_000));
    expect(r?.filed).toBe(1);
    expect(withRepo(() => sentinelTick(app, lastHunt() + 20 * 60_000))).toBeNull();
  });
  /** Close an issue the way the close path does: the state, and the event naming who. */
  const closeBy = (issueId: string, actorId: string, at: string): void => {
    app.db.query("UPDATE issue SET state = 'closed', closed_at = ?2 WHERE id = ?1").run(issueId, at);
    const i = app.db.query<{ number: number; repo_id: string }, [string]>("SELECT number, repo_id FROM issue WHERE id = ?1").get(issueId)!;
    app.bus.append({ kind: "issue.closed", repoId: i.repo_id, actorId, payload: { number: i.number } });
  };
  const planter = async (name: string): Promise<User> => {
    const made = await createUser(app.db, { origin: "seed", trusted: true, username: name, email: `${name}@forge.local`, password: "a-good-passphrase" });
    if (!made.ok) throw new Error(made.error);
    app.db.query("UPDATE user SET is_agent = 1 WHERE id = ?1").run(made.value.id);
    return made.value;
  };
  test("an agent's close does not keep a setting, and the migration takes back one it did (#2029)", async () => {
    const bot = await planter("plannerbot");
    app.db.query("DELETE FROM issue WHERE title LIKE 'Sentinel: Setting %'").run();
    app.db.query("DELETE FROM sentinel_kept").run();
    const base = Date.now();
    setCapacity(app.db, "checks.concurrency", "4", alice.id);
    const setAt = new Date(base - 10 * 24 * HOUR).toISOString();
    app.db.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'").run(setAt);
    hunt(app, "alice/forge", base);
    const issue = app.db.query<{ id: string }, []>("SELECT id FROM issue WHERE state = 'open' AND title LIKE 'Sentinel: Setting checks.concurrency %'").get()!;
    closeBy(issue.id, bot.id, new Date(base + 1000).toISOString());
    expect(hunt(app, "alice/forge", base + 2 * HOUR).filed).toBe(1);
    expect(app.db.query("SELECT COUNT(*) AS n FROM sentinel_kept").get()).toEqual({ n: 0 });
    // a row an agent's close made before this rule, beside one a person's close made
    const plant = (key: string): void => {
      app.db.query("INSERT INTO sentinel_kept (key, value, updated_at, kept_at) VALUES (?1, '4', ?2, ?2)").run(key, setAt);
    };
    plant("setting:checks.concurrency");
    plant("setting:cells.egress");
    const mine = app.db.query<{ id: string }, []>("SELECT id FROM issue WHERE state = 'open' AND title LIKE 'Sentinel: Setting checks.concurrency %'").get()!;
    closeBy(mine.id, bot.id, new Date(base + 3000).toISOString());
    setCapacity(app.db, "cells.egress", "report", alice.id);
    app.db.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'cells.egress'").run(setAt);
    hunt(app, "alice/forge", base + 4 * HOUR);
    const egress = app.db.query<{ id: string }, []>("SELECT id FROM issue WHERE state = 'open' AND title LIKE 'Sentinel: Setting cells.egress %'").get()!;
    closeBy(egress.id, alice.id, new Date(base + 5000).toISOString());
    app.db.exec(keptIsAPersons);
    const left = app.db.query<{ key: string }, []>("SELECT key FROM sentinel_kept").all().map((r) => r.key);
    expect(left).toEqual(["setting:cells.egress"]);
    setCapacity(app.db, "checks.concurrency", "1", alice.id);
    expect(withRepo(() => sentinelTick(app, l + 20 * 60_000))).toBeNull();
    hunt(app, "alice/forge", l + 2 * HOUR);
    expect(app.db.query("SELECT COUNT(*) AS n FROM sentinel_finding WHERE key LIKE 'setting:%'").get()).toEqual({ n: 0 });
    expect(app.db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM issue WHERE title LIKE 'Sentinel: Setting %'").get()!.n).toBe(0);
    setCapacity(app.db, "cells.egress", "enforce", alice.id);
  });
  test("a setting kept on purpose is not refiled until its value changes (#2001)", () => {
    const setAt = (value: string, at: string): void => {
      setCapacity(app.db, "checks.concurrency", value, alice.id);
      app.db.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'").run(at);
    };
    const issuesFor = (): number =>
      app.db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM issue WHERE title LIKE 'Sentinel: Setting checks.concurrency %'").get()!.n;
    app.db.query("DELETE FROM issue WHERE title LIKE 'Sentinel: Setting %'").run();
    app.db.query("DELETE FROM sentinel_kept").run();
    app.db.query("DELETE FROM forge_setting WHERE key IN ('cells.egress', 'checks.concurrency')").run();
    const base = Date.now();
    setAt("4", new Date(base - 10 * 24 * HOUR).toISOString());
    expect(hunt(app, "alice/forge", base).filed).toBe(1);
    const first = app.db.query<{ id: string; body: string }, []>("SELECT id, body FROM issue WHERE state = 'open' AND title LIKE 'Sentinel: Setting checks.concurrency %'").get()!;
    expect(first.body).toContain("Keep or Put back");
    // closed with a reason: kept
    closeBy(first.id, alice.id, new Date(base).toISOString());
    const comments = listComments(app.db, first.id).length;
    expect(hunt(app, "alice/forge", base + 2 * HOUR).filed).toBe(0);
    expect(hunt(app, "alice/forge", base + 4 * HOUR).filed).toBe(0);
    expect(issuesFor()).toBe(1);
    expect(listComments(app.db, first.id).length).toBe(comments);
    // the value changes, and later is away a day: reported again
    setAt("6", new Date(base + 3 * HOUR).toISOString());
    expect(hunt(app, "alice/forge", base + 6 * HOUR).filed).toBe(0); // not yet a day away
    expect(hunt(app, "alice/forge", base + 30 * HOUR).filed).toBe(1);
    expect(issuesFor()).toBe(2);
    expect(app.db.query("SELECT value FROM forge_setting WHERE key = 'checks.concurrency'").get()).toEqual({ value: "6" });
    app.db.query("DELETE FROM forge_setting WHERE key = 'checks.concurrency'").run();
  });
  test("a comment saying why a setting stays keeps it too (#2001)", async () => {
    const owner = await createUser(app.db, {
      origin: "seed",
      trusted: true,
      username: "carol",
      email: "[email protected]",
      password: "a-good-passphrase",
    });
    if (!owner.ok) throw new Error(owner.error);
    app.db.query("DELETE FROM issue WHERE title LIKE 'Sentinel: Setting %'").run();
    app.db.query("DELETE FROM sentinel_kept").run();
    const base = Date.now();
    setCapacity(app.db, "checks.concurrency", "4", alice.id);
    app.db.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'").run(new Date(base - 10 * 24 * HOUR).toISOString());
    hunt(app, "alice/forge", base);
    const issue = app.db.query<{ id: string; body: string }, []>("SELECT id, body FROM issue WHERE state = 'open' AND title LIKE 'Sentinel: Setting checks.concurrency %'").get()!;
    const said = addComment(app.db, { issueId: issue.id, authorId: owner.value.id, body: "Kept on purpose: P0 wants it." });
    expect(said.ok).toBe(true);
    app.db.query("UPDATE comment SET created_at = ?2 WHERE issue_id = ?1").run(issue.id, new Date(base + 1000).toISOString());
    const before = listComments(app.db, issue.id).length;
    expect(hunt(app, "alice/forge", base + 2 * HOUR).filed).toBe(0);
    expect(listComments(app.db, issue.id).length).toBe(before);
    expect(app.db.query("SELECT value FROM sentinel_kept WHERE key = 'setting:checks.concurrency'").get()).toEqual({ value: "4" });
    setCapacity(app.db, "checks.concurrency", "1", alice.id);
  });
  test("a comment from an agent does not keep a setting (#2001)", async () => {
    app.db.query("DELETE FROM issue WHERE title LIKE 'Sentinel: Setting %'").run();
    app.db.query("DELETE FROM sentinel_kept").run();
    const base = Date.now();
    setCapacity(app.db, "checks.concurrency", "4", alice.id);
    app.db.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'").run(new Date(base - 10 * 24 * HOUR).toISOString());
    hunt(app, "alice/forge", base);
    const issue = app.db.query<{ id: string }, []>("SELECT id FROM issue WHERE state = 'open' AND title LIKE 'Sentinel: Setting checks.concurrency %'").get()!;
    const bot = await createUser(app.db, { origin: "seed", trusted: true, username: "reviewbot", email: "[email protected]", password: "a-good-passphrase" });
    if (!bot.ok) throw new Error(bot.error);
    app.db.query("UPDATE user SET is_agent = 1 WHERE id = ?1").run(bot.value.id);
    addComment(app.db, { issueId: issue.id, authorId: bot.value.id, body: "Looks fine to me." });
    app.db.query("UPDATE comment SET created_at = ?2 WHERE issue_id = ?1").run(issue.id, new Date(base + 1000).toISOString());
    hunt(app, "alice/forge", base + 2 * HOUR);
    expect(app.db.query("SELECT COUNT(*) AS n FROM sentinel_kept").get()).toEqual({ n: 0 });
    setCapacity(app.db, "checks.concurrency", "1", alice.id);
  test("an open setting finding from before closes on the next pass, with a pointer to the capacity page (#3269)", () => {
    const old = createIssue(app.db, { repoId: repo.id, authorId: alice.id, title: "Sentinel: Setting checks.concurrency has been away from its default for 9 days" });
    if (!old.ok) throw new Error(String(old.error));
    app.db.query("INSERT INTO sentinel_finding (key, issue_id, evidence, first_seen, last_seen) VALUES ('setting:checks.concurrency', ?1, '', 'a', 'a')").run(old.value.id);
    hunt(app, "alice/forge", Date.now());
    const row = app.db.query<{ state: string; close_reason: string }, [string]>("SELECT state, close_reason FROM issue WHERE id = ?1").get(old.value.id)!;
    expect(row).toEqual({ state: "closed", close_reason: "done" });
    expect(listComments(app.db, old.value.id).some((c) => c.body.includes("/settings/capacity"))).toBe(true);
    // closed ones are still known as setting findings
    expect(isSettingFinding(app.db, old.value.id)).toBe(true);
  });
  test("a newly held slot hunts too", () => {
@@ -890,7 +782,7 @@
describe("a degraded finding whose cause has stayed closed", () => {
    await rm(hdir, { recursive: true, force: true });
  });
  test("23 hours closed stays open; a person's comment and a setting keep theirs at 25", () => {
  test("23 hours closed stays open; a person's comment keeps theirs at 25, and a setting finding closes", () => {
    // A setting finding, tracked as the sentinel would.
    const s = createIssue(happ.db, { repoId: hrepo.id, authorId: halice.id, title: "Sentinel: Setting x" });
    if (!s.ok) throw new Error(String(s.error));
@@ -909,7 +801,7 @@
describe("a degraded finding whose cause has stayed closed", () => {
    expect(said.length).toBe(1);
    expect(said[0]!.body).toContain(new Date(T0).toISOString());
    expect(state("disk")).toBe("open");
    expect(state("Setting x")).toBe("open");
    expect(state("Setting x")).toBe("closed");
  });
  test("a finding that comes back is a new issue", () => {
@@ -14,7 +14,7 @@
import { createRepo } from "../src/actions/repos.ts";
import { signUp } from "../src/actions/users.ts";
import { createApp, type App as Forge } from "../src/app.ts";
import { chipsOf, setChip } from "../src/core/feed_chips.ts";
import { getIssue } from "../src/core/issues.ts";
import { createIssue, getIssue } from "../src/core/issues.ts";
import { mutedRepos, recordInteraction } from "../src/core/interactions.ts";
import { setCapacity, storedCapacity } from "../src/core/capacity.ts";
import { UNCLOSABLE_WORDS } from "../src/actions/setting_ask.ts";
@@ -231,12 +231,26 @@
describe("terminal On you: mutes and many in one move", () => {
    h.stop();
  });
  // The sentinel files no setting now (#3269); one it filed before still waits for an answer.
  const legacySetting = (key: string): void => {
    const made = createIssue(forge.db, {
      repoId: lamp.id,
      authorId: alice.id,
      title: `Sentinel: Setting ${key} has been away from its default for 10 days`,
      body: `${key} is away`,
    });
    if (!made.ok) throw new Error(made.error);
    forge.db
      .query("INSERT INTO sentinel_finding (key, issue_id, evidence, first_seen, last_seen) VALUES (?1, ?2, '', 'a', 'a')")
      .run(`setting:${key}`, made.value.id);
  };
  test("a setting away from its default is Kept or Put back from the terminal", async () => {
    setCapacity(forge.db, "cells.egress", "report", alice.id);
    forge.db
      .query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'cells.egress'")
      .run(new Date(Date.now() - 10 * 86_400_000).toISOString());
    hunt(forge, "alice/lamp");
    legacySetting("cells.egress");
    const h = await boot();
    for (let i = 0; i < 20; i++) await h.press("j"); // the setting row is last, and the cursor stops there
    expect(h.text()).toContain("setting cells.egress");
@@ -247,7 +261,7 @@
describe("terminal On you: mutes and many in one move", () => {
    forge.db
      .query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'cells.egress'")
      .run(new Date(Date.now() - 10 * 86_400_000).toISOString());
    hunt(forge, "alice/lamp", Date.now() + 3 * 3_600_000);
    legacySetting("cells.egress");
    await h.press("r");
    for (let i = 0; i < 20; i++) await h.press("j"); // the setting row is last, and the cursor stops there
    await h.press("K");
@@ -263,7 +277,7 @@
describe("terminal On you: mutes and many in one move", () => {
      .run(new Date(Date.now() - 9 * 86_400_000).toISOString());
    forge.db.query("DELETE FROM sentinel_kept").run();
    forge.db.query("DELETE FROM sentinel_finding").run();
    hunt(forge, "alice/lamp", Date.now() + 6 * 3_600_000);
    legacySetting("cells.egress");
    const made = createOrigin(forge.db, dir, {
      userId: alice.id,
      kind: "github",
		@@ -1248,35 +1248,31 @@ given up.
1248	1248	### The sentinel
1249	1249
1250	1250	The forge hunts its own faults (`docs/CREW.md`): a hunt an hour on the
1251		autonomy clock, sooner when a held slot or a setting left away from its
1252		default is new. Each finding is one issue with its evidence, filed in
1253		`FORGE_SENTINEL_REPO` (default `trevato/forge`) as the `sentinel` agent, else
1254		the first admin. Evidence is scrubbed of anything credential-shaped before it
1255		is filed.
	1251	autonomy clock, sooner when a held slot is new. Each finding is one issue with
	1252	its evidence, filed in `FORGE_SENTINEL_REPO` (default `trevato/forge`) as the
	1253	`sentinel` agent, else the first admin. Evidence is scrubbed of anything
	1254	credential-shaped before it is filed. Other findings (`held`, `ledger`, `red`)
	1255	are filed anew when they return.
1256	1256
1257		A setting kept on purpose is a decision, not drift. Closing a `setting:<key>`
1258		finding's issue, or commenting on it (a person, not the filer and not an agent), records the value
1259		and its `updated_at` in `sentinel_kept`; while both match, the sentinel files
1260		nothing and comments nothing for that key. Changing the value on the capacity
1261		page ends the keeping, and the setting is reported again under the usual
1262		one-day rule. The rule is mechanical: any close counts, with or without a
1263		reason, so a bare close silences the setting until its value changes. Other
1264		findings (`held`, `ledger`, `red`) are filed anew when they return.
	1257	A capacity setting away from its default is not a finding: the sentinel files
	1258	nothing for it (#3269). On `/settings/capacity` the row of each such setting
	1259	carries one muted line, "changed from X N days ago by who", and its Put back
	1260	button (in the row's own form, beside Save) restores the default through
	1261	`changeCapacity` (same validation, record, event and reversal). The line stays
	1262	until the value is back at the default; nothing else reminds anyone. The
	1263	JSON API and MCP carry the same fact: a setting away from its default has a
	1264	`changed` field (`at`, `by`) in the capacity listing (`listCapacity`).
1265	1265
1266		A setting finding is the site admin's ask, never a crew run: it is not
1267		assigned, and the autonomy clock leaves it out of the crew's ready list (`crew/setting_finding.ts`). It shows on the home (kind `setting`,
1268		one card per key: the value, its default, who set it and when) and in the
1269		terminal's On you card, with two answers — Keep and Put back. On the
1270		web they are `POST /asks/setting` (`key`, `answer=keep\|put`, same-origin only);
1271		in the terminal, `K` keeps and `B` puts back the row under the cursor. Only a
1272		signed-in site admin can answer, whatever they can write: anyone else, and any
1273		agent, gets `not_found`/`unauthorized` and nothing changes. Keep closes the
1274		finding and records the value in `sentinel_kept`; Put back goes through the
1275		capacity page's own door (`changeCapacity`: same validation, record, event and
1276		reversal). The close is the gate: Put back changes the setting, and Keep records
1277		the value, only after a finding has closed. If none can (every one is on a
1278		mirrored repository, or the close is refused), nothing changes and the page and
1279		the terminal say so (`unclosable`). Only a person's close keeps a setting.
	1266	A `setting:<key>` finding filed before that is closed on the next pass with a
	1267	comment pointing at the capacity page (`closeSettingFindings` in
	1268	`crew/setting_finding.ts`). Until it closes, it is the site admin's ask, never a
	1269	crew run: not assigned, left out of the crew's ready list, and shown on the home
	1270	(kind `setting`) and in the terminal's On you card with Keep and **Put
	1271	back** (`POST /asks/setting`, `key`, `answer=keep\|put`, same-origin; `K` and `B`
	1272	in the terminal). Only a signed-in site admin can answer. Keep and Put back act
	1273	only after the finding has closed; if none can (a mirrored repository, or the
	1274	close is refused) nothing changes and the page says so (`unclosable`). This is
	1275	the legacy path, kept until no open setting finding is left.
1280	1276
1281	1277	- Board: a site admin's forge strip shows the open findings and the last
1282	1278	hunt; a guest or member sees nothing.
		@@ -39,10 +39,9 @@ the `denyRead` list) and not the boundary itself.
39	39	The forge hunts its own faults. `src/crew/sentinel.ts` runs on the autonomy
40	40	clock (a hunt an hour; a signal may bring one sooner, never within ten
41	41	minutes). It reads what the forge already holds — the run table, the decision
42		ledger (`retry stop` streaks, `guard` parks, a direction at its cap), capacity
43		settings away from their defaults for more than a day, `sqlite_master` for
44		triggers the schema does not create, and the error ring — and names each fault
45		with the evidence quoted (secret-shaped text scrubbed).
	42	ledger (`retry stop` streaks, `guard` parks, a direction at its cap),
	43	`sqlite_master` for triggers the schema does not create, and the error ring —
	44	and names each fault with the evidence quoted (secret-shaped text scrubbed).
46	45
47	46	Each finding is exactly one open issue, keyed so it is never filed twice; a
48	47	finding still there is a comment, and only when the evidence changed. The board
		@@ -5,7 +5,8 @@ the forge already holds and files what is wrong, with the evidence quoted.
5	5
6	6	When. On the autonomy clock: a hunt an hour, and one sooner (never within
7	7	ten minutes of the last) when a signal is new — queued checks waiting while no
8		run uses a slot, or a capacity setting away from its default for over a day.
	8	run uses a slot. A capacity setting away from its default is no signal and
	9	no finding: it is a line on `/settings/capacity` (#3269).
9	10
10	11	What it reads.
11	12
		@@ -14,7 +15,6 @@ run uses a slot, or a capacity setting away from its default for over a day.
14	15	\| a slot held by a finished run \| `run`: queued work older than 30 min, nothing running or every lease lapsed \|
15	16	\| a check never green; pushes that cannot land \| the last 20 `check` / `agent` run summaries per repository \|
16	17	\| `retry` stops, guard parks \| the decision ledger (`decisionsSince`) \|
17		\| a setting left away from default \| `forge_setting` against `capacityDefault` \|
18	18	\| a hand-installed trigger \| `sqlite_master` against a scratch database migrated from the schema \|
19	19	\| the same ERROR line again and again \| the error ring (`kernel/log.ts`) \|
20	20
		@@ -7,6 +7,7 @@
7	7	* an event: the key, from what to what, and who.
8	8	*/
9	9
	10	import type { Database } from "bun:sqlite";
10	11	import type { App } from "../app.ts";
11	12	import {
12	13	CAPACITY,
		@@ -34,6 +35,8 @@ export interface CapacityRow {
34	35	readonly source: "setting" \| "default";
35	36	readonly default: string;
36	37	readonly env: string;
	38	/** For a setting away from its default: when and by whom it was set. */
	39	readonly changed?: { readonly at: string; readonly by: string };
37	40	}
38	41
39	42	function admin(actor: User \| undefined): Result<User, "not_found"> {
		@@ -42,6 +45,18 @@ function admin(actor: User \| undefined): Result<User, "not_found"> {
42	45	: err("not_found");
43	46	}
44	47
	48	/** Who set `key` and when, only while its stored value is not the default. */
	49	export function changedOf(db: Database, key: CapacityKey): { changed?: { at: string; by: string } } {
	50	const value = storedCapacity(db, key);
	51	if (value === undefined \|\| value === bootDefault(db, key)) return {};
	52	const row = db
	53	.query<{ updated_at: string; username: string \| null }, [string]>(
	54	"SELECT s.updated_at, u.username FROM forge_setting s LEFT JOIN user u ON u.id = s.updated_by WHERE s.key = ?1",
	55	)
	56	.get(key);
	57	return row === null ? {} : { changed: { at: row.updated_at, by: row.username ?? "someone" } };
	58	}
	59
45	60	export function listCapacity(
46	61	app: App,
47	62	actor: User \| undefined,
		@@ -57,6 +72,7 @@ export function listCapacity(
57	72	source: storedCapacity(app.db, key) === undefined ? "default" : "setting",
58	73	default: bootDefault(app.db, key),
59	74	env: CAPACITY[key].env,
	75	...changedOf(app.db, key),
60	76	})),
61	77	changes: capacityChanges(app.db),
62	78	});
		@@ -16,11 +16,6 @@
16	16	import { Database } from "bun:sqlite";
17	17	import type { App } from "../app.ts";
18	18	import { addComment, openIssue, setIssueState } from "../actions/issues.ts";
19		import {
20		CAPACITY_KEYS,
21		capacityDefault,
22		storedCapacity,
23		} from "../core/capacity.ts";
24	19	import { decisionsSince } from "../core/decisions.ts";
25	20	import { getAgentByName } from "../core/agents.ts";
26	21	import { getIssueById, getLabel, setIssueLabels as writeLabels, type Issue } from "../core/issues.ts";
		@@ -34,6 +29,7 @@ import { spans } from "../kernel/degraded_history.ts";
34	29	import { QUEUE_WAIT_MAX_MS } from "../kernel/vitals.ts";
35	30	import { STALE_AFTER_MS } from "../runs/lease.ts";
36	31	import { isOutageSummary } from "./network.ts";
	32	import { closeSettingFindings } from "./setting_finding.ts";
37	33
38	34	const log = createLog("sentinel");
39	35
		@@ -42,8 +38,6 @@ const DAY_MS = 24 * HOUR_MS;
42	38	/** A hunt an hour; a signal may bring one sooner, but never within this. */
43	39	export const HUNT_EVERY_MS = HOUR_MS;
44	40	export const SIGNAL_GAP_MS = 10 * 60_000;
45		/** A setting away from its default this long is a finding. */
46		export const SETTING_AWAY_MS = DAY_MS;
47	41	/** A cause degraded longer than this in its day is a finding. */
48	42	export const DEGRADED_DAY_MS = 10 * 60_000;
49	43	/** How many alike make a streak. */
		@@ -63,8 +57,6 @@ export interface Finding {
63	57	readonly urgent?: true;
64	58	/** What a person or a run would do, when it is mechanical and reversible. */
65	59	readonly fix?: string;
66		/** A setting finding: the value and `updated_at` a person may have kept (#2001). */
67		readonly kept?: { readonly value: string; readonly updatedAt: string };
68	60	/** A degraded finding: when its cause first opened. Same cause and time, same incident (#2553). */
69	61	readonly incident?: string;
70	62	}
		@@ -168,37 +160,6 @@ export function heldSlots(db: Database, now: number): Finding[] {
168	160	return out;
169	161	}
170	162
171		/** A capacity setting that is not its default, for more than a day. */
172		export function settingsAway(db: Database, now: number): Finding[] {
173		const out: Finding[] = [];
174		for (const key of CAPACITY_KEYS) {
175		const value = storedCapacity(db, key);
176		if (value === undefined \|\| value === capacityDefault(key)) continue;
177		const row = db
178		.query<
179		{ updated_at: string; username: string \| null },
180		[string]
181		>("SELECT s.updated_at, u.username FROM forge_setting s LEFT JOIN user u ON u.id = s.updated_by WHERE s.key = ?1")
182		.get(key);
183		if (row === null \|\| now - new Date(row.updated_at).getTime() < SETTING_AWAY_MS)
184		continue;
185		const days = Math.floor((now - new Date(row.updated_at).getTime()) / DAY_MS);
186		out.push({
187		key: `setting:${key}`,
188		title: `Setting ${key} has been away from its default for ${days} days`,
189		evidence: [
190		`${key} = "${value}", default "${capacityDefault(key)}"`,
191		`set by ${row.username ?? "unknown"} at ${row.updated_at}`,
192		],
193		pillar: "Trust: a setting is a decision, not a leftover",
194		lookFor: `${key} is back at "${capacityDefault(key)}", or a person says why it stays: the owner is asked in the inbox, Keep or Put back, and a person's close or comment keeps it, reported again only if its value changes. No agent is assigned; a run cannot change a setting.`,
195		kept: { value, updatedAt: row.updated_at },
196		fix: `Put it back: clear ${key} on the capacity page (it is recorded and reversible).`,
197		});
198		}
199		return out;
200		}
201
202	163	/** The crew's stops and guard parks, read from the decision ledger. */
203	164	export function ledgerStreaks(db: Database, now: number): Finding[] {
204	165	const out: Finding[] = [];
		@@ -486,7 +447,6 @@ export function findFaults(db: Database, now = Date.now()): Finding[] {
486	447	const found = [
487	448	...budgetBreaches(db, now),
488	449	...heldSlots(db, now),
489		...settingsAway(db, now),
490	450	...strayTriggers(db),
491	451	...ledgerStreaks(db, now),
492	452	...runStreaks(db),
		@@ -583,43 +543,6 @@ function labelBreach(app: App, repo: Repo, actor: User, issue: Issue): void {
583	543	}
584	544	}
585	545
586		/** The finding is a setting whose kept value and `updated_at` are on record. */
587		function keptRecorded(db: Database, f: Finding): boolean {
588		if (f.kept === undefined) return false;
589		const kept = db
590		.query<{ value: string; updated_at: string }, [string]>("SELECT value, updated_at FROM sentinel_kept WHERE key = ?1")
591		.get(f.key);
592		return kept !== null && kept.value === f.kept.value && kept.updated_at === f.kept.updatedAt;
593		}
594
595		/**
596		* Whether a person decided to keep this setting as it now stands: its
597		* finding was closed, or explained in a comment by a person (not the sentinel, not an agent),
598		* after the value was last set. A decision is recorded with the value and
599		* `updated_at` it was made at; the same pair is kept, any other is drift.
600		*/
601		function settingKept(db: Database, f: Finding, actorId: string, at: string): boolean {
602		if (f.kept === undefined) return false;
603		if (keptRecorded(db, f)) return true;
604		const { value, updatedAt } = f.kept;
605		const decided = db
606		.query<{ n: number }, [string, string, string]>(
607		`SELECT COUNT(*) AS n FROM sentinel_finding t JOIN issue i ON i.id = t.issue_id
608		WHERE t.key = ?1 AND ((i.state = 'closed' AND i.closed_at >= ?2
609		AND (SELECT e.actor_id FROM event e
610		WHERE e.kind = 'issue.closed' AND e.repo_id = i.repo_id
611		AND json_extract(e.payload, '$.number') = i.number
612		ORDER BY e.id DESC LIMIT 1)
613		IN (SELECT id FROM user WHERE is_agent = 0 AND kind = 'user'))
614		OR EXISTS (SELECT 1 FROM comment c WHERE c.issue_id = i.id AND c.author_id <> ?3 AND c.created_at >= ?2
615		AND EXISTS (SELECT 1 FROM user u WHERE u.id = c.author_id AND u.is_agent = 0 AND u.kind = 'user')))`,
616		)
617		.get(f.key, updatedAt, actorId);
618		if (decided === null \|\| decided.n === 0) return false;
619		recordKept(db, f.key, value, updatedAt, at);
620		return true;
621		}
622
623	546	/** Remember that a person kept `key` as it stands: this value, set at `updatedAt`. */
624	547	export function recordKept(db: Database, key: string, value: string, updatedAt: string, at: string): void {
625	548	db.query(
		@@ -720,8 +643,8 @@ function closeHeld(app: App, repo: Repo, actor: User, active: ReadonlySet<string
720	643	/**
721	644	* Read, file, record. One open issue per finding: a tracked finding whose
722	645	* issue is open is commented on only when its evidence changed; one whose
723		* issue is closed and returns is a new issue — except a setting a person
724		* kept (`settingKept`), which stays quiet until its value changes.
	646	* issue is closed and returns is a new issue. A setting away from its default
	647	* is no finding: the capacity page says so beside it (#3269).
725	648	*/
726	649	export function hunt(
727	650	app: App,
		@@ -738,7 +661,6 @@ export function hunt(
738	661	if (repo !== undefined && actor !== undefined) {
739	662	for (const f of findings) {
740	663	try {
741		if (settingKept(app.db, f, actor.id, at)) continue;
742	664	const evidence = f.evidence.join("\n");
743	665	const tracked = app.db
744	666	.query<
		@@ -787,7 +709,7 @@ export function hunt(
787	709	}
788	710	issueId = made.value.id;
789	711	filed++;
790		if (f.urgent && f.kept === undefined) labelBreach(app, repo, actor, made.value);
	712	if (f.urgent) labelBreach(app, repo, actor, made.value);
791	713	}
792	714	app.db
793	715	.query("INSERT INTO sentinel_finding (key, issue_id, evidence, first_seen, last_seen, incident) VALUES (?1, ?2, ?3, ?4, ?4, ?5)")
		@@ -797,8 +719,10 @@ export function hunt(
797	719	}
798	720	}
799	721	}
800		if (repo !== undefined && actor !== undefined)
	722	if (repo !== undefined && actor !== undefined) {
801	723	closeHeld(app, repo, actor, new Set(findings.map((f) => f.key)), now);
	724	closeSettingFindings(app, repo, actor);
	725	}
802	726	// A hunt with nowhere to file is recorded as such, never as a quiet one.
803	727	const unfiled = repo === undefined \|\| actor === undefined;
804	728	app.db
		@@ -856,8 +780,7 @@ export function sentinelSummary(db: Database): string \| null {
856	780	}
857	781
858	782	/**
859		* Hunt when it is time: an hour since the last, or a signal (a slot held, a
860		* setting away) not yet tracked, and not within ten minutes of the last.
	783	* Hunt when it is time: an hour since the last, or a signal (a slot held) not yet tracked, and not within ten minutes of the last.
		junipercrew 24d ago Docs (README sentinel section, docs/SENTINEL.md, docs/CREW.md) still describe the removed setting finding and its sooner hunt; update them.
861	784	*/
862	785	export function sentinelTick(app: App, now = Date.now()): HuntResult \| null {
863	786	const repoPath = sentinelRepoPath();
		@@ -869,7 +792,7 @@ export function sentinelTick(app: App, now = Date.now()): HuntResult \| null {
869	792	// A breach is not held back by the gap: the crew hears within a tick.
870	793	const signals = [
871	794	...breaches,
872		...(since < SIGNAL_GAP_MS ? [] : [...heldSlots(app.db, now), ...settingsAway(app.db, now)]),
	795	...(since < SIGNAL_GAP_MS ? [] : heldSlots(app.db, now)),
873	796	];
874	797	const tracked = new Set(
875	798	app.db
		@@ -879,7 +802,7 @@ export function sentinelTick(app: App, now = Date.now()): HuntResult \| null {
879	802	.all()
880	803	.map((r) => r.key),
881	804	);
882		if (!signals.some((s) => !tracked.has(s.key) && !keptRecorded(app.db, s))) return null;
	805	if (!signals.some((s) => !tracked.has(s.key))) return null;
883	806	}
884	807	return hunt(app, repoPath, now);
885	808	}
		@@ -5,6 +5,14 @@
5	5	*/
6	6
7	7	import type { Database } from "bun:sqlite";
	8	import type { App } from "../app.ts";
	9	import { addComment, setIssueState } from "../actions/issues.ts";
	10	import { getIssueById } from "../core/issues.ts";
	11	import type { Repo } from "../core/repos.ts";
	12	import type { User } from "../core/users.ts";
	13	import { createLog } from "../kernel/log.ts";
	14
	15	const log = createLog("sentinel");
8	16
9	17	/** Whether this issue is the open finding of a `setting:<key>` the sentinel reported. */
10	18	export function isSettingFinding(db: Database, issueId: string): boolean {
		@@ -16,3 +24,33 @@ export function isSettingFinding(db: Database, issueId: string): boolean {
16	24	.get(issueId) !== null
17	25	);
18	26	}
	27
	28	/**
	29	* A setting away from its default is a line on the capacity page, not an
	30	* issue (#3269). Findings filed before that are closed, with a pointer.
	31	*/
	32	export function closeSettingFindings(app: App, repo: Repo, actor: User): void {
	33	const open = app.db
	34	.query<{ issue_id: string }, []>(
	35	`SELECT DISTINCT t.issue_id FROM sentinel_finding t JOIN issue i ON i.id = t.issue_id
	36	WHERE i.state = 'open' AND t.key LIKE 'setting:%'`,
	37	)
	38	.all();
	39	for (const t of open) {
	40	try {
	41	const issue = getIssueById(app.db, t.issue_id);
	42	if (issue === undefined \|\| issue.repoId !== repo.id) continue;
	43	const said = addComment(
	44	app,
	45	actor,
	46	repo,
	47	issue,
	48	"A setting away from its default is now one line beside it on the capacity page (/settings/capacity), with Put back; the sentinel files no issue for it. Closing.",
	49	);
	50	if (!said.ok) continue;
	51	setIssueState(app, actor, repo, issue, "closed", { reason: "done" });
	52	} catch (error) {
	53	log.error("sentinel could not close a setting finding", { issue: t.issue_id, error: String(error) });
	54	}
	55	}
	56	}
		@@ -6,7 +6,7 @@
6	6	* and each one is listed below the settings: who, when, from what to what.
7	7	*/
8	8
9		import { changeCapacity, listCapacity } from "../../actions/capacity.ts";
	9	import { changeCapacity, listCapacity, type CapacityRow } from "../../actions/capacity.ts";
10	10	import { CAPACITY, type CapacityKey } from "../../core/capacity.ts";
11	11	import type { Ctx } from "../../http/context.ts";
12	12	import { notFoundPage, signInFirst } from "../chrome.ts";
		@@ -22,6 +22,22 @@ function requireAdmin(ctx: Ctx): Response \| undefined {
22	22	return undefined;
23	23	}
24	24
	25	const DAY_MS = 86_400_000;
	26
	27	/** "today", "1 day ago", "9 days ago". */
	28	export function daysAgo(at: string, now: number): string {
	29	const days = Math.max(0, Math.floor((now - Date.parse(at)) / DAY_MS));
	30	return days === 0 ? "today" : `${days} day${days === 1 ? "" : "s"} ago`;
	31	}
	32
	33	/** One muted line beside a setting away from its default; Put back is in the row's trail form. */
	34	function awayLine(s: CapacityRow, now: number) {
	35	if (s.changed === undefined) return null;
	36	return html`<span class="muted small">changed from
	37	<span class="mono">${s.default === "" ? "auto" : s.default}</span>
	38	${daysAgo(s.changed.at, now)} by ${s.changed.by}</span>`;
		junipercrew 24d ago outdated `row-fields` is the trail form class in DESIGN.md; here it wraps a line inside the row meta. Move Put back into the trail form, or document this as a component.
	39	}
	40
25	41	function input(key: CapacityKey, value: string) {
26	42	const kind = CAPACITY[key].kind;
27	43	if (kind.type === "choice")
		@@ -55,6 +71,7 @@ export function capacityPage(ctx: Ctx): Response {
55	71	const listed = listCapacity(ctx.app, ctx.auth.user);
56	72	if (!listed.ok) return notFoundPage(ctx);
57	73	const { settings, changes } = listed.value;
	74	const now = Date.now();
58	75	const rows = settings.map((s) =>
59	76	row({
60	77	lead: icon("activity"),
		@@ -63,7 +80,8 @@ export function capacityPage(ctx: Ctx): Response {
63	80	? html`Set here; the boot default is
64	81	<span class="mono">${s.default === "" ? "auto" : s.default}</span>
65	82	(<span class="mono">${s.env}</span>).`
66		: html`The boot default, from <span class="mono">${s.env}</span>.`}`,
	83	: html`The boot default, from <span class="mono">${s.env}</span>.`}
	84	${awayLine(s, now)}`,
67	85	trail: html`<form
68	86	class="row-fields"
69	87	method="post"
		@@ -79,11 +97,11 @@ export function capacityPage(ctx: Ctx): Response {
79	97	${when(
80	98	s.source === "setting",
81	99	() =>
82		html`${btn("Default", {
	100	html`${btn(s.changed === undefined ? "Default" : "Put back", {
83	101	size: "sm",
84	102	name: "reset",
85	103	value: "1",
86		label: `Back to the default for ${s.label}`,
	104	label: s.changed === undefined ? `Back to the default for ${s.label}` : `Put ${s.label} back to the default`,
87	105	})}`,
88	106	)}
89	107	</form>`,
		@@ -14,7 +14,8 @@ import { openPull } from "../src/actions/pulls.ts";
14	14	import { commitToBranch, createRepo } from "../src/actions/repos.ts";
15	15	import { TESTS_PASSED_TAIL } from "../src/apps/runtime.ts";
16	16	import { recordDecision, resolveOpen } from "../src/core/decisions.ts";
17		import type { Repo } from "../src/core/repos.ts";
	17	import { createIssue } from "../src/core/issues.ts";
	18	import { getRepo, type Repo } from "../src/core/repos.ts";
18	19	import { claimRun, createRun, finishRun } from "../src/core/runs.ts";
19	20	import { getUserByName } from "../src/core/users.ts";
20	21	import { markVerified } from "../src/core/verify.ts";
		@@ -283,6 +284,26 @@ describe("the asks page", () => {
283	284	.query("UPDATE forge_setting SET updated_at = ?2 WHERE key = ?1")
284	285	.run(key, new Date(Date.now() - 10 * 86_400_000).toISOString());
285	286	};
	287	// The sentinel no longer files a setting (#3269); an issue it filed
	288	// before still waits for an answer, so these tests plant one.
	289	const fileLegacy = (path: string): void => {
	290	const [owner = "", name = ""] = path.split("/");
	291	const target = getRepo(app.db, owner, name)!;
	292	const author = getUserByName(app.db, "alice")!;
	293	const rows = app.db.query<{ key: string }, []>("SELECT key FROM forge_setting").all();
	294	for (const { key } of rows) {
	295	const made = createIssue(app.db, {
	296	repoId: target.id,
	297	authorId: author.id,
	298	title: `Sentinel: Setting ${key} has been away from its default for 10 days`,
	299	body: `${key} is away`,
	300	});
	301	if (!made.ok) throw new Error(made.error);
	302	app.db
	303	.query("INSERT INTO sentinel_finding (key, issue_id, evidence, first_seen, last_seen) VALUES (?1, ?2, '', 'a', 'a')")
	304	.run(`setting:${key}`, made.value.id);
	305	}
	306	};
286	307	const post = (cookie: string, fields: Record<string, string>, origin = servers.url) =>
287	308	fetch(`${servers.url}/asks/setting`, {
288	309	method: "POST",
		@@ -305,8 +326,8 @@ describe("the asks page", () => {
305	326	reset();
306	327	away("checks.concurrency", "4");
307	328	away("cells.egress", "report");
308		hunt(app, "alice/garden");
309		hunt(app, "alice/garden", Date.now() + 3 * 3_600_000);
	329	fileLegacy("alice/garden");
	330	fileLegacy("alice/garden");
310	331	expect(settingCards().sort()).toEqual(["setting:cells.egress", "setting:checks.concurrency"]);
311	332	const card = decisionsFor({ app, auth: { user: getUserByName(app.db, "alice")! } } as unknown as Ctx, [repo]).find(
312	333	(d) => d.id === "setting:checks.concurrency",
		@@ -323,12 +344,11 @@ describe("the asks page", () => {
323	344	expect(listEvents(app.db, { kind: "issue.assigned" }).length).toBe(0);
324	345	});
325	346
326		test("Keep closes the finding as kept, and it stays quiet", async () => {
	347	test("Keep closes the finding as kept", async () => {
327	348	const res = await post(alice, { key: "checks.concurrency", answer: "keep" });
328	349	expect(res.status).toBe(303);
329	350	expect(settingCards()).toEqual(["setting:cells.egress"]);
330	351	expect(app.db.query("SELECT value FROM sentinel_kept WHERE key = 'setting:checks.concurrency'").get()).toEqual({ value: "4" });
331		expect(hunt(app, "alice/garden", Date.now() + 5 * 3_600_000).filed).toBe(0);
332	352	expect(storedCapacity(app.db, "checks.concurrency")).toBe("4");
333	353	const closed = app.db
334	354	.query<{ close_reason: string }, []>("SELECT close_reason FROM issue WHERE title LIKE 'Sentinel: Setting checks.concurrency %'")
		@@ -348,7 +368,7 @@ describe("the asks page", () => {
348	368	test("a stranger, a bad answer and a foreign origin change nothing", async () => {
349	369	reset();
350	370	away("cells.egress", "report");
351		hunt(app, "alice/garden");
	371	fileLegacy("alice/garden");
352	372	await post(bob, { key: "cells.egress", answer: "put" });
353	373	await post(alice, { key: "cells.egress", answer: "burn" });
354	374	await post(alice, { key: "nope", answer: "put" });
		@@ -361,7 +381,7 @@ describe("the asks page", () => {
361	381	test("Put back changes nothing when no finding can be closed, and says why", async () => {
362	382	reset();
363	383	away("cells.egress", "report");
364		hunt(app, "alice/garden");
	384	fileLegacy("alice/garden");
365	385	const alice0 = getUserByName(app.db, "alice")!;
366	386	const made = createOrigin(app.db, dataDir, {
367	387	userId: alice0.id,
		@@ -392,7 +412,7 @@ describe("the asks page", () => {
392	412	test("Put back changes nothing when the close is refused for a reason other than a mirror", () => {
393	413	reset();
394	414	away("cells.egress", "report");
395		hunt(app, "alice/garden");
	415	fileLegacy("alice/garden");
396	416	// a pull cannot be closed with an issue's reason, so setIssueState refuses
397	417	app.db.query("UPDATE issue SET kind = 'pull' WHERE title LIKE 'Sentinel: Setting cells.egress %'").run();
398	418	const alice0 = getUserByName(app.db, "alice")!;
		@@ -413,7 +433,7 @@ describe("the asks page", () => {
413	433	const plot = await createRepo(app, carolUser, { name: "plot", visibility: "public" });
414	434	if (!plot.ok) throw new Error(String(plot.error));
415	435	away("cells.egress", "report");
416		hunt(app, "carol/plot");
	436	fileLegacy("carol/plot");
417	437	expect(app.db.query("SELECT COUNT(*) AS n FROM sentinel_finding WHERE key = 'setting:cells.egress'").get()).toEqual({ n: 1 });
418	438	await post(carol, { key: "cells.egress", answer: "keep" });
419	439	await post(carol, { key: "cells.egress", answer: "put" });
		@@ -20,11 +20,10 @@ import { mergePull, openPull } from "../src/actions/pulls.ts";
20	20	import { commitToBranch, createRepo } from "../src/actions/repos.ts";
21	21	import { getAppByRepo } from "../src/core/apps.ts";
22	22	import { hunt } from "../src/crew/sentinel.ts";
23		import { setCapacity } from "../src/core/capacity.ts";
24	23	import { recordDecision } from "../src/core/decisions.ts";
25	24	import { recordLook } from "../src/core/looks.ts";
26	25	import { getPull } from "../src/core/pulls.ts";
27		import { updateRepoSettings, type Repo } from "../src/core/repos.ts";
	26	import { getRepo, updateRepoSettings, type Repo } from "../src/core/repos.ts";
28	27	import { claimRun, createRun, finishRun } from "../src/core/runs.ts";
29	28	import { submitReview } from "../src/core/reviews.ts";
30	29	import { createUser, getUserByName, type User } from "../src/core/users.ts";
		@@ -570,10 +569,12 @@ describe("the board's forge strip", () => {
570	569	test("the sentinel's line is the admin's alone, and absent before the first hunt", async () => {
571	570	expect(await board("root")).not.toContain("sentinel");
572	571
573		setCapacity(app.db, "checks.concurrency", "4", admin.id);
	572	// A check queued two hours with nothing running: a held slot, a finding.
	573	const stuck = getRepo(app.db, "root", "stuck")!;
	574	const queued = createRun(app.db, { repoId: stuck.id, kind: "check" });
574	575	app.db
575		.query("UPDATE forge_setting SET updated_at = ?1 WHERE key = 'checks.concurrency'")
576		.run("2020-01-01T00:00:00.000Z");
	576	.query("UPDATE run SET created_at = ?2 WHERE id = ?1")
	577	.run(queued.id, new Date(Date.now() - 2 * 3_600_000).toISOString());
577	578	process.env["FORGE_SENTINEL_REPO"] = "root/stuck";
578	579	try {
579	580	expect(hunt(app, "root/stuck").filed).toBeGreaterThan(0);
		@@ -586,7 +587,7 @@ describe("the board's forge strip", () => {
586	587	expect(guest).not.toMatch(/sentinel/i);
587	588	} finally {
588	589	delete process.env["FORGE_SENTINEL_REPO"];
589		setCapacity(app.db, "checks.concurrency", "1", admin.id);
	590	app.db.query("DELETE FROM run WHERE id = ?1").run(queued.id);
590	591	}
591	592	});
592	593	});
		@@ -13,6 +13,8 @@ import { join } from "node:path";
13	13	import { createApp } from "../src/app.ts";
14	14	import {
15	15	CAPACITY,
	16	CAPACITY_KEYS,
	17	bootDefault,
16	18	capacityDefault,
17	19	capacityEnv,
18	20	capacityValue,
		@@ -22,7 +24,7 @@ import {
22	24	setCapacity,
23	25	shardsForCpus,
24	26	} from "../src/core/capacity.ts";
25		import { settingsAway } from "../src/crew/sentinel.ts";
	27	import { changedOf } from "../src/actions/capacity.ts";
26	28	import { createUser } from "../src/core/users.ts";
27	29
28	30	const ENV = ["FORGE_CHECK_SHARDS", "FORGE_CHECKS_CONCURRENCY"] as const;
		@@ -70,7 +72,7 @@ describe("the defaults follow the CPUs", () => {
70	72	});
71	73	});
72	74
73		describe("a stored value overrides the default; the sentinel compares with it", () => {
	75	describe("a stored value overrides the default; the capacity page compares with it", () => {
74	76	test("80 CPUs: 4 shards and 6 checks are the defaults, 8 shards is away", async () => {
75	77	delete process.env.FORGE_CHECK_SHARDS;
76	78	delete process.env.FORGE_CHECKS_CONCURRENCY;
		@@ -98,15 +100,15 @@ describe("a stored value overrides the default; the sentinel compares with it",
98	100	setCapacity(app.db, "crew.concurrency", "5", a.value.id);
99	101	const old = new Date(Date.now() - 3 * 24 * 3600_000).toISOString();
100	102	app.db.query("UPDATE forge_setting SET updated_at = ?1").run(old);
101		expect(settingsAway(app.db, Date.now()).map((f) => f.key)).toEqual(["setting:crew.concurrency"]);
	103	expect(CAPACITY_KEYS.filter((k) => changedOf(app.db, k).changed !== undefined)).toEqual(["crew.concurrency"]);
102	104
103	105	// A stored value overrides the default, even a cell's smaller one.
104	106	setCapacity(app.db, "checks.shards", "8", a.value.id);
105	107	app.db.query("UPDATE forge_setting SET updated_at = ?1").run(old);
106	108	expect(capacityValue(app.db, "checks.shards")).toBe("8");
107	109	expect(capacityEnv(app.db, 2)).toEqual({ FORGE_CHECK_SHARDS: "8" });
108		const away = settingsAway(app.db, Date.now()).find((f) => f.key === "setting:checks.shards");
109		expect(away?.evidence[0]).toBe('checks.shards = "8", default "4"');
	110	expect(changedOf(app.db, "checks.shards").changed?.by).toBe("alice");
	111	expect(bootDefault(app.db, "checks.shards")).toBe("4");
110	112	} finally {
111	113	app.close();
112	114	await rm(dataDir, { recursive: true, force: true });