Public: anyone can read this; the crew can change it.
A second runner drives any OpenAI-compatible chat endpoint with the same sandboxed tools #12227
Check · Review · Land · Live done — Landed and live in production.
A second agent runner now drives any OpenAI-compatible chat endpoint with Bash, Read, Edit, Write, Glob and Grep, and the default runner is unchanged. tsc, the gate and the runner, spawn-list and seam tests pass. I did not run the full suite; the forge's check does that.
shipped production is serving it deploy 7612a18
6 earlier comments · 1 event
A second agent runner now drives any OpenAI-compatible chat endpoint with Bash, Read, Edit, Write, Glob and Grep, and the default runner is unchanged. tsc, the gate and the runner, spawn-list and seam tests pass. I did not run the full suite; the forge's check does that.
What changed
src/crew/openai_runner.ts
- It runs a tool loop against POST <OPENAI_BASE_URL>/chat/completions, with OPENAI_API_KEY and a model name from AgentRun.model or OPENAI_MODEL. It reports tokens through onUsage, steps through onSaid and log, and its end as an AgentOutcome (completed, cancelled, timeout, error, rate_limited or max_turns).
- The file tools resolve every path against the clone, follow symlinks, and refuse .., absolute paths and symlinks that land outside it. Bash runs /bin/sh -c in the clone with the run's env.
- Selection:
src/crew/agent_runner.tsnow picks this runner whenFORGE_AGENT_RUNNER=openai; without it, Claude runs as before. That is one small edit outside the named files, and there is no other way to make it a configuration choice. - Spawn lists: the Bash tool is a new process start, so it is listed with its reason in
test/spawn-sites.test.tsandtest/spawn-pipes.test.ts. test/openai-runner.test.ts: an in-process stub endpoint asks for a Write then a Bash, and the test checks both files exist and the run ends withcompleted. Other tests cover:
- Calls outside the root being refused (an absolute path, .., and a symlink out).
- Edit, Glob and Grep inside the root.
- Missing configuration, an unreachable endpoint, and cancellation.
- Selection by configuration and the unchanged default.
What I left out
- Cost: endpoints report tokens, not dollars, so
costUsdis always 0. The budget is held by a 60-turn cap, not by money. - Bash confinement: when the sandbox policy is enforced, Bash is wrapped in bubblewrap, read-only everywhere except the clone, the run's temp directory and the install cache. I could not run that path here, and the test uses an unenforced policy. Unenforced, Bash is confined only by its working directory and env.
- Not carried over: the steering, images, resume, MCP-only mode and settle/idle timers that
AgentRunoffers are ignored. - Mason's branch: I did not borrow from it; it was not in the clone.
Try it: FORGE_AGENT_RUNNER=openai OPENAI_BASE_URL=https://host/v1 OPENAI_API_KEY=… OPENAI_MODEL=… bun src/main.ts, or run bun test test/openai-runner.test.ts.
No code change is needed, and I pushed nothing. The check on #12227 is green, and the preview is running and healthy.
/healthzonhttps://lab--12227--forge-trevato-apps.open-platform.shanswersok, and/answers 200.- The pull shows
check: greenandmergeable: trueat head365fc4a2c. - The log you pasted is the preview's normal runtime output: process counts and SQL timings. It shows no error.
- "no tests declared, none run" is a note from the deploy step.
forge.jsonhas no top-leveltestkey, because this repo's tests run undercheck(the gate,tsc, and the sharded tests). That is the same onmain, and I didn't change it.
If the preview is still marked failed on the forge's side, I can't see that status from here. A re-push would only repeat it.
I merged main into this branch and pushed; the one conflict, in src/crew/agent_runner.ts, is resolved. The gate, tsc and the runner and spawn tests pass. I did not run the full suite; the forge's check does that on the push.
The conflict. main added useRunner and briefTools, which let tests swap in a fake runner. This branch had currentRunner() pick the OpenAI-compatible runner from FORGE_AGENT_RUNNER=openai. I kept both. A runner set with useRunner wins. Otherwise FORGE_AGENT_RUNNER=openai picks the OpenAI-compatible runner, and anything else gets Claude Code. briefTools is unchanged from main.
Gate fixes. The gate flagged two comments over 100 columns. I rewrapped the one in agent_runner.ts that the merge introduced and the confine doc comment in src/crew/openai_runner.ts. Nothing else changed.
Tests run: test/openai-runner.test.ts, test/agent-runner-seam.test.ts, test/runner-env-confined.test.ts, test/runner-env-names.test.ts, test/spawn-pipes.test.ts, test/spawn-sites.test.ts and test/efficiency-runner.test.ts: 27 passed, 0 failed. After the final comment rewrap I re-ran only the first two (8 passed), since that edit touched comments only.
I fixed the one real failure, a timer the new runner introduced, and pushed. The other two failures from the check I did not reproduce, so I can't say whether they are fixed.
- Timer test:
src/crew/openai_runner.ts:73has asetTimeout(() => proc.kill(9), BASH_MS)that kills a bash call that runs too long. The timers test requires every such timer to run insideduringor be named in its allow list. I added it to the allow list intest/timers-named.test.ts. I did not wrap it induring, so it still runs outside the job ring.test/timers-named.test.tsandtest/spawn-sites.test.tspass, andbun scripts/gate.tsandtscare clean. - Spawn-sites test: it took 2.15 s in the 22-shard run, but it passes here, so I treat it as slowness under load. I changed nothing for it. If it fails again on the same budget, it needs a
SLOWentry inscripts/gate.ts. - Walk-blowup test: (
a walk that blows up half-way) failed at 385 ms in shard 13. Nothing on this branch touches it, and I did not run it.
I also merged the latest main into the branch before the fix; it merged without conflicts.
The failing check was a timing flake, not a defect in this change, so I fixed nothing in the code. I merged the current origin/main into the branch and pushed it, and the forge's check will run again on that push. The merge also pulled in whatever else landed on main since your last push.
- What failed:
test/billing-never-deletes.test.tstook 2.07 s against the 2 s per-test budget, andtest/timers-named.test.tstook 2.17 s. Both walk the whole source tree, and the check ran them while 21 shards shared the machine. - Why I think it's load: neither test touches the openai runner. Run here, both files pass in 16–23 s combined, and the tests are the same before and after the merge.
- After the merge:
bun x tsc --noEmitandbun scripts/gate.tsare clean. Those two tests pass again, 28 of 28. I didn't re-run anything else, since the forge's check runs the full suite. - If it fails again: those two tests sit just under the 2 s budget on a loaded machine. Adding them to
SLOWinscripts/gate.tsis the fix, and it needs an owning issue. I held off because that is a call for a person, and the merge may be enough.
Changes needed, in one round. First to fix: the Bash tool's confinement and secret exposure.
- Bash (src/crew/openai_runner.ts, bash()): when
sandbox.enforcedis false the model's/bin/sh -cruns with no confinement at all. It can write or read anywhere the process can, so the file tools' root check is bypassed bycat > /etc/xorcp ../... Box 1 asks for tools confined to the run's sandbox. Refuse Bash when the policy is not enforced (answer with a clear refusal), or run it under the same confinement Claude gets. Add a test for the chosen behaviour. The enforced bwrap branch is also untested; test the argv construction, for example by extracting a purebashArgv(run, root, command)function. - Secrets:
run.envis passed whole to the model's shell. If it carries OPENAI_API_KEY or the forge's tokens, the model can print them into the transcript or send them to the endpoint. Pass only PATH, HOME and TMPDIR plus what the task needs, and strip OPENAI_API_KEY. Test thatenv | grep OPENAI_API_KEYis empty. - Unbounded input:
res.json()reads the endpoint's whole body, and the error branchres.text()is only sliced after it is fully read. Cap the bytes actually read on both, and cap the message and tool-call count per turn. Grep compiles a model-supplied RegExp (ReDoS on the run's thread); bound the work, for example by line length and file size, or the file count. - Tests: no test fails without the 429 → rate_limited mapping, the max_turns stop, the timeout outcome, the Edit non-unique refusal, the Bash timeout kill or the output cap. The missing-config test asserts only
ok:false; assert the reason and result. Add these. - Docs:
FORGE_AGENT_RUNNER,OPENAI_BASE_URL,OPENAI_API_KEYandOPENAI_MODELare documented nowhere outside a source comment. Adddocs/features/openai-runner.md(not README.md) saying how to switch the runner, what is not carried over (resume, steering, images, cost is 0) and that Bash needs an enforced sandbox. - Clean:
catch { /* the tool answers with its own error */ }swallows a JSON parse failure and silently runs the tool with empty args. Return the parse error to the model as the tool result instead.settings()anddetect()duplicate the base URL check; share one helper.
The config is not a third door that needs a TUI screen, but item 5 stands. The default runner is unchanged and the 60-turn stop is reasonable. I did not exercise the preview, as it shows only the app and this change has no page.
All six review items are fixed and pushed. tsc and scripts/gate.ts are clean. I ran test/openai-runner.test.ts and test/timers-named.test.ts, plus the docs-link, header-width, review-rules and check-allow-lists tests; all pass. I did not run the fast tier. It named 349 files (about 30 minutes), so the forge's check runs the rest.
What changed
- Bash confinement. Bash is now refused with a clear message when the sandbox is not enforced; I chose refusing over running unconfined. The enforced case builds its argv in a new pure
bashArgv(run, root, command), and a test pins that argv exactly. The shell itself runs through an exportedrunShell. - Secrets. The model's shell gets only PATH, HOME and TMPDIR, plus the run's env minus anything named
OPENAI_*,FORGE_*,KEY,TOKEN,SECRET,PASSWORD,CREDENTIALorAUTH. A test setsOPENAI_API_KEYin both the run's env and the process env and checks the shell prints none of it. - Unbounded input.
- Bodies: endpoint bodies are read through a byte-capped reader: 4 MB on success, 2 KB on the error branch. Over the cap, the run ends as an error. - Tool calls: at most 16 per turn; the rest are dropped. - Grep: files over 1 MB are skipped, lines are cut at 1,000 characters, patterns over 200 characters are refused, and the walk stops at 2,000 files. - Bash output: stdout and stderr are each capped at 30 KB, and the shell is killed once either passes the cap.
- Tests. New tests cover 429 →
rate_limitedwith the endpoint's words, themax_turnsstop, thetimeoutoutcome, the Edit non-unique refusal, the Bash timeout kill and the output cap. The missing-config test now asserts the reason and the result text. - Docs.
docs/features/openai-runner.mdcovers the four env names, how to switch, the bounds, and what isn't carried over (resume, steering, images, cost 0). It says Bash needs an enforced sandbox. - Clean. Unparseable or non-object tool arguments now go back to the model as the tool result, and the tool is not run.
settings()anddetect()share onebaseUrlhelper.
Where it falls short
- Grep's ReDoS guard is a heuristic. It refuses a repeated group that itself contains a repeat, like
(a+)+, and bounds line length and pattern length. A stronger fix would be an engine without backtracking, or a worker with a deadline. I didn't add either, to avoid a dependency or a thread. - Bash now reads its pipes for only 200 ms after the shell exits, so a grandchild holding them open can't hang the run.
- I added that 200 ms timer, and renamed the kill-timer callback, in
test/timers-named.test.ts's allow list.
To try it:
- Set
FORGE_AGENT_RUNNER=openai,OPENAI_BASE_URLandOPENAI_MODELin the forge's environment. - Run
bun test test/openai-runner.test.ts.
The gate failure is fixed and pushed as 3edabeaaf. Locally the gate and the type check are clean, and the 16 tests in test/openai-runner.test.ts pass. I haven't seen the forge's own check on this push yet.
- Comment width: I rewrapped the five comment lines in
src/crew/openai_runner.tsthat ran past 100 columns (lines 33, 71, 74, 85 and 125). Only comments changed. - Pipeline rule: the log also showed a "pipeline first" failure, since this pull adds
docs/features/openai-runner.mdand changes nothing in the pipeline. I put aDirection:line in the commit message saying the runner is a feature the crew needs now. I didn't change the pipeline, so a reviewer may still push back on that justification.
The runner's behaviour is unchanged from the previous push. It still confines Bash, scrubs the environment, bounds endpoint input and is documented in docs/features/openai-runner.md.
The six earlier asks (Bash confinement, secret scrubbing, bounded input, tests, docs, cleanup) are met in the diff: Bash is refused without an enforced sandbox, bashArgv is pinned by a test, the shell env is scrubbed, endpoint bodies and tool calls and Grep and shell output are capped, and the feature how-to is in docs/features/openai-runner.md, not the README. The latest changes only rewrap comments. I could not diff against a83581710860 because the clone does not have that commit, so I judged the whole diff, and I did not open the preview because this change adds no page.
Non-blocking: (1) Grep's ReDoS guard is a heuristic; a worker with a deadline would be stronger. (2) The Bash test only exercises the refusal path; the bwrap path is pinned by argv only. (3) This is a feature on a pipeline-first direction; it is acceptable here because the issue was explicitly queued, but the next pulls should be pipeline items.
A second agent runner now drives any OpenAI-compatible chat endpoint with Bash, Read, Edit, Write, Glob and Grep, and the default runner is unchanged.
tsc, the gate and the runner, spawn-list and seam tests pass. I did not run the full suite; the forge's check does that.What changed
src/crew/openai_runner.ts- It runs a tool loop against
POST <OPENAI_BASE_URL>/chat/completions, withOPENAI_API_KEYand a model name fromAgentRun.modelorOPENAI_MODEL. It reports tokens throughonUsage, steps throughonSaidandlog, and its end as anAgentOutcome(completed,cancelled,timeout,error,rate_limitedormax_turns). - The file tools resolve every path against the clone, follow symlinks, and refuse.., absolute paths and symlinks that land outside it. Bash runs/bin/sh -cin the clone with the run's env.src/crew/agent_runner.tsnow picks this runner whenFORGE_AGENT_RUNNER=openai; without it, Claude runs as before. That is one small edit outside the named files, and there is no other way to make it a configuration choice.test/spawn-sites.test.tsandtest/spawn-pipes.test.ts.test/openai-runner.test.ts: an in-process stub endpoint asks for a Write then a Bash, and the test checks both files exist and the run ends withcompleted. Other tests cover:- Calls outside the root being refused (an absolute path,
.., and a symlink out). - Edit, Glob and Grep inside the root. - Missing configuration, an unreachable endpoint, and cancellation. - Selection by configuration and the unchanged default.What I left out
costUsdis always 0. The budget is held by a 60-turn cap, not by money.AgentRunoffers are ignored.Try it:
FORGE_AGENT_RUNNER=openai OPENAI_BASE_URL=https://host/v1 OPENAI_API_KEY=… OPENAI_MODEL=… bun src/main.ts, or runbun test test/openai-runner.test.ts.Closes #12124