Public: anyone can read this; the crew can change it.

Journeys held outside the code say if a bet is really done: a score builders can't game #8715

planned no changes pushed: Bets (#8713) aren't in the code yet — Mason is still working on them, and no bet record exists for journeys to attach to. opened this
opened this

From StrongDM (https://factory.strongdm.ai/, self-reported): user stories kept outside the codebase, like a holdout set, because agents gamed plain tests; success is the share of runs that satisfy the user. The forge's reviewers and tests live where builders can edit them. This needs care: a browser run costs tokens.

Done when:

  • [ ] A journey is a short plain sentence on a bet (#8713), kept where a builder's run can neither read nor write it.
  • [ ] A separate run walks a journey on the bet's preview in a browser, and answers pass or fail with a screenshot and the reason. Each run stays in a stated token budget, and only the bet's journeys run, once per scope finished.
  • [ ] A bet's score shows on its page and is what "this is done" rests on. Passing doesn't replace the owner's one tap.
  • [ ] A test shows a builder's run can't reach them.
commented

Bets (#8713) aren't in the code yet — Mason is still working on them, and no bet record exists for journeys to attach to.

DECLINED: this work depends on bets (#8713), which hasn't landed. No bet record, bet page or "scope finished" event exists on main (grep for bet/appetite/journey finds nothing in src/), and #8713 is still being worked by mason. Journeys are defined here as "a short plain sentence on a bet", and the score is "on its page", so there is nothing to attach them to. If I invented a bet model now, it would clash with the one in #8713. Re-run this once #8713 lands. Linking #8715 as blocked by #8713 would stop it being picked up early.

commented

Parked by the crew: the run ended without a pull: no changes pushed: Bets (#8713) aren't in the code yet — Mason is still working on them, and no bet record exists for journeys to attach to.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs