Data explorer: a model fallback for sentences the rules cannot read #9627

A sentence parseSentence cannot read now goes to the model, on both /boards/data?ask= and the terminal explorer's / box. The new tests, the existing boards-guide and tui-ask-data suites, tsc, scripts/gate.ts and design-lint pass. I did not run the full suite, and the real model call (runClaude) is untested, since the tests stub the model.

What changed

• src/boards/model_fill.ts (new) — fillFromSentence tries parseSentence first and costs nothing on a hit. Only a miss asks the model. The model must reply with one JSON object using only source, aggregate, column, by, series, from, to, filter and chart. validFill checks that reply:

- Every field must be known, and every source and column must exist in SOURCES. - The aggregate, filter operator, chart, bucket and times must be valid. - Each value must survive readParams unchanged. - An unknown source or column, an extra field or a bad value is refused. runQuery then judges the accepted query as it judges any link.

• src/crew/one_shot.ts (new) — one model turn with no tools, no shell and no MCP. It picks the asker's own subscription the way a conversation turn does, so the per-turn cap and daily ceiling apply. It spends at most $0.05 and books the cost to that account. I had to add this because the repo had no lightweight model call. It reuses runClaude with an empty MCP config.
• Web — src/web/pages/boards.ts redirects with &model=1 when the model filled the form, and the result card's foot says "The model filled this form from your sentence. Check it before you trust it."
• Terminal — src/tui/screens/ask_data.ts shows the same note on the result screen. A refused or unreadable sentence now opens a short message screen instead of keeping the prompt open with an error. This is a small change in behaviour, because the model call is asynchronous.
• Gate — unchanged: a site admin who is a person. Nobody else reaches the model.
• Limits — six model asks per minute per person. A request marked Sec-Fetch-Site: cross-site never reaches the model, so a link on another site cannot spend an admin's budget.
• Tests — test/boards-model-fill.test.ts and one added test in test/tui-ask-data.test.ts cover the cases below. The model is stubbed in all of them.

- A rules hit never calls the model. - A miss reaches it. - An answer naming an unknown source or column is refused. - The rate limit holds. - A cross-site request and a non-admin never reach the model. - The note appears on both web and terminal.

Not done

• The model's replies are not cached. Each miss costs a turn, bounded by the rate limit and the daily ceiling.
• Without a configured crew or granted account the model is unavailable, and a miss says "I could not read that" as before.
• No documentation page was changed. No existing doc covers the sentence box.

To try Sign in as a site admin and open /boards/data?ask=how+pricey+were+the+robots on a forge with the crew on. In the terminal, open Data explorer, press / and type the same sentence.

Closes #9569

A sentence parseSentence cannot read now goes to the model, on both /boards/data?ask= and the terminal explorer's / box. The new tests, the existing boards-guide and tui-ask-data suites, tsc, scripts/gate.ts and design-lint pass. I did not run the full suite, and the real model call (runClaude) is untested, since the tests stub the model.

What changed

• src/boards/model_fill.ts (new) — fillFromSentence tries parseSentence first and costs nothing on a hit. Only a miss asks the model. The model must reply with one JSON object using only source, aggregate, column, by, series, from, to, filter and chart. validFill checks that reply:

- Every field must be known, and every source and column must exist in SOURCES. - The aggregate, filter operator, chart, bucket and times must be valid. - Each value must survive readParams unchanged. - An unknown source or column, an extra field or a bad value is refused. runQuery then judges the accepted query as it judges any link.

• src/crew/one_shot.ts (new) — one model turn with no tools, no shell and no MCP. It picks the asker's own subscription the way a conversation turn does, so the per-turn cap and daily ceiling apply. It spends at most $0.05 and books the cost to that account. I had to add this because the repo had no lightweight model call. It reuses runClaude with an empty MCP config.
• Web — src/web/pages/boards.ts redirects with &model=1 when the model filled the form, and the result card's foot says "The model filled this form from your sentence. Check it before you trust it."
• Terminal — src/tui/screens/ask_data.ts shows the same note on the result screen. A refused or unreadable sentence now opens a short message screen instead of keeping the prompt open with an error. This is a small change in behaviour, because the model call is asynchronous.
• Gate — unchanged: a site admin who is a person. Nobody else reaches the model.
• Limits — six model asks per minute per person. A request marked Sec-Fetch-Site: cross-site never reaches the model, so a link on another site cannot spend an admin's budget.
• Tests — test/boards-model-fill.test.ts and one added test in test/tui-ask-data.test.ts cover the cases below. The model is stubbed in all of them.

- A rules hit never calls the model. - A miss reaches it. - An answer naming an unknown source or column is refused. - The rate limit holds. - A cross-site request and a non-admin never reach the model. - The note appears on both web and terminal.

Not done

• The model's replies are not cached. Each miss costs a turn, bounded by the rate limit and the daily ceiling.
• Without a configured crew or granted account the model is unavailable, and a miss says "I could not read that" as before.
• No documentation page was changed. No existing doc covers the sentence box.

To try Sign in as a site admin and open /boards/data?ask=how+pricey+were+the+robots on a forge with the crew on. In the terminal, open Data explorer, press / and type the same sentence.

The check failed because no commit message had a "Try:" line. I added an empty commit with one and pushed it. bun scripts/gate.ts passed locally before the push. I did not re-run the tests, since no code changed.

What changed

• A sentence parseSentence cannot read goes to the model on both /boards/data?ask= and the terminal explorer's / box.
• The model may answer only with the explorer's own fields. validFill refuses any unknown source, column or field, and runQuery judges the rest as it judges any link.
• The rules answer first and cost nothing. The model is asked only on a miss, on the asker's own account, within its budget, at most six times a minute. A request from another site never reaches it.
• The chart's foot (web) or the result screen (terminal) says when the model filled the form.
• The gate is unchanged: a site admin who is a person.

What I chose not to do

• I do not cache the model's replies. Each miss costs a turn, bounded by the rate limit and the daily ceiling.
• Without a configured crew or a granted account, a miss still says "I could not read that".

To try Sign in as a site admin and open /boards/data?ask=how+pricey+were+the+robots on the preview; the chart's foot says the model filled the form. In the terminal, open Data explorer, press / and type the same sentence. This needs the crew on and a granted account.

The data explorer's sentence box now asks the model only when the request came from the page itself, and every item from the review is addressed except the terminal's error placement, which I changed differently (below). bun scripts/gate.ts, tsc --noEmit and design-lint are clean. The four affected test files pass (27 tests). I did not run bun scripts/fast.ts's full set, because it named 1450 files, which is too broad. I also could not exercise the real model, so the only coverage of the model path is the stub and fake-binary tests.

What changed

• Budget guard. The web asks the model only when Sec-Fetch-Site is same-origin or none. A missing header, same-site and cross-site get the rules only, so a link cannot spend an admin's budget. I kept the GET and did not move the model call behind a POST.
• The model=1 note. The &model=1 redirect is gone. A model fill is now drawn in place, so only that response can show the "model filled this form" note, and a hand-made link can't claim it. A rules hit still redirects to its link. A reload of a model-filled page asks the model again, within the six-a-minute limit.
• oneShot tests. New test/crew-one-shot.test.ts runs oneShot against a fake binary and covers:

- the crew being off; - a member with no granted account; - the --max-budget-usd clamp, both the configured budget and the $0.05 cap; - the cost being booked to the subscription; - a spent daily ceiling resting it without starting the binary.

• How-to. New docs/features/data-explorer-sentence.md covers rules first, the model on a miss, the checks, six asks a minute, the note, the header rule, what it needs and how to try it. The README is untouched.
• Shared wording. The unread, limit and refused text is now unreadWords in src/boards/model_fill.ts, used by both web and terminal.
• Comment. The header comment in ask_data.ts is wrapped.

Not done as asked

• Terminal error. The shared prompt in common.ts closes before an async result arrives, so an inline error isn't possible without changing every prompt. The model's refusal now shows as a toast instead of replacing the prompt with a message screen. I chose not to touch the shared prompt. Say if you want that change.

To try it Sign in as a site admin with a granted account and the crew on. Type "how pricey were the robots" into the sentence box on /boards/data on the preview. The chart's foot shows the note.

🤖 Generated with Claude Code

A question to the data explorer's model now counts against the asker's daily ceiling, and only a POST from the page's own sentence box can ask the model. A ?ask= link is now read by the rules only.

Daily ceiling (high finding). A new table, one_shot_spend (migration 20261108170000-one-shot-spend.ts, pinned), records each question to the model. daySpent in src/core/account_ceiling.ts now adds it in, so dailyLeft shrinks after each question. oneShot in src/crew/one_shot.ts writes the question's cap before the model runs, so questions still running are counted too. When the run ends it sets the real cost. If the run throws, the cap stays as the cost, since the real cost isn't known. The subscription's spent_usd is still added to as before.

Links cannot spend the budget (medium finding). A GET to /boards/data?ask= is now rules-only, whatever Sec-Fetch-Site says. The sentence box posts to the new POST /boards/data (boardsAsk). That handler applies the same admin gate, then readForm's cross-origin check, and only then may a miss ask the model. A link clicked from a mail client, a chat or another site cannot reach the model. A model's answer is still drawn in place with the note, so the note cannot be added to a link by hand.

Tests:

• test/crew-one-shot.test.ts checks that a question lowers dailyLeft by its cost.
• test/boards-model-fill.test.ts checks that a ?ask= GET never calls the model with any Sec-Fetch-Site value or none.
• It also checks that the page's form posts, that a POST from another origin gets a 403 before the model is asked, and that a non-admin's POST gets a 404.

docs/features/data-explorer-sentence.md is updated to match.

Checks run:

• bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts are clean.
• The changed test files, the ceiling, schema, route and boards suites pass.
• test/boards-guide.test.ts and test/boards-page.test.ts fail two tests ("a source-list link lands on that source's default chart", "a ready question opens a working chart from its link") when run in one process with the ceiling, schema, route and other boards suites. They fail the same way on the base commit with my changes stashed, and both files pass when run alone. That clash between test files predates this change and is not addressed here.

To try it: sign in as a site admin with a granted account and the crew on. On /boards/data, type "how pricey were the robots" in the box and press Show; the chart's foot says the model filled the form. Opening /boards/data?ask=how+pricey+were+the+robots directly says "I could not read that" and spends nothing.

🤖 Generated with Claude Code