Public: anyone can read this; the crew can change it.

A dataset as we go: the forge's events, decisions, runs, checks and usage exported as a documented, growing dataset the crew can learn from #1139

open deferred: behind the Later gate opened this
Held by the Later gate, set by trevato . The crew picks nothing here until it is lifted.
opened this

From the owner: "We have an intelligent crew and all of git history. Our iteration speed, our dev velocity, is there. We need to start generating a fantastic dataset as we go."

The forge already records more than most teams keep: every event on the bus, every decision in the ledger (who decided, on what, how sure), every run with its cost, turns and outcome, every check with its failures, and soon every request's use. None of it is written down as a dataset anyone can read back as one.

Done looks like:

  • [ ] A nightly export: JSONL files per table (events, decisions, runs, checks, pulls, usage), one schema document that names every field and its meaning, personal data removed or hashed by the same rule everywhere, written to a private data repository on the forge so the history is git's and the crew can read it.
  • [ ] A /data page: what the dataset holds, how big, when it last grew, and the one-line questions it already answers (time from tell to shipped by pillar; cost per landed pull by agent and model; which checks fail most; which pages nobody opens).
  • [ ] The planner reads the last week's export before it plans: what landed, what parked, what cost, and says so in the wave it proposes.
  • [ ] Tests: the export round-trips a sample of each table; no email, token or raw path with ids appears in any file; the schema document names every exported field.

Files I expect: src/core/dataset.ts (new), scripts/export-dataset.ts, src/web/pages/data.ts, docs/DATASET.md, src/crew/autonomy.ts (the planner's reading).

commented

Parked by the crew: the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1365 closed). Planner: split this smaller, or rewrite what done looks like; a person can also say what to try.

commented

Parked by the crew: the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1452 closed).

Parked twice since a person last spoke here, so it is the planner's now. Every reason:

  1. the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1365 closed)
  2. the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1452 closed)

The planner splits it, replaces it with a fresh issue, or defers it; a person can also say what to try. Nothing starts here until then, and the board lists it.

commented

Deferred behind the Later gate (#595): the dataset export should read from the Boards sources (#1559) and the usage table (#1564) once they land, rather than build its own. Both pulls went red on the gate's list.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs