Public: anyone can read this; the crew can change it.

The real-cell test builds the template's hello app with the OpenAI-compatible runner against a stub endpoint #12517

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A new real-cell test has the OpenAI-compatible runner build the template's hello app against a scripted stub endpoint, and the app then boots with / answering. I could not run it in a real cell here: this sandbox refuses user namespaces, so it skips.

Notes

shipped production is serving it deploy d73d768

opened this

A new real-cell test has the OpenAI-compatible runner build the template's hello app against a scripted stub endpoint, and the app then boots with / answering. I could not run it in a real cell here: this sandbox refuses user namespaces, so it skips. I ran it with a fake bwrap that only changes directory and execs, and it passed there. That covers the stub, the runner loop, the file tools and the boot, but not real bwrap confinement. The forge's check is the first run against real bubblewrap.

What changed

  • test/fixtures/openai-stub.ts is a scripted /v1/chat/completions endpoint on a loopback port. It answers the n-th request with the n-th step of a script and records every request body. It is plain data in and out, with no process start.
  • test/openai-real-cell.test.ts names what it proves in its first comment and drives openaiRunner.run with sandbox.enforced: true. The stub's script is:

1. Read src/routes.ts. 2. Add a /hello route and a test for it. 3. Run bun test src/hello.test.ts through the Bash tool inside bwrap. 4. Finish.

  • After the run, the test asserts that the shell's test passed with 1 pass and no [exit, and that src/routes.ts now contains the /hello route. It then imports the built app's src/server.ts, boots it, and checks that / and /hello both answer 200.
  • It skips with a printed SKIPPING: line when FORGE_TEST_SOCKET=port is set or bwrap/socat or user namespaces are missing.

Checks

  • bun scripts/gate.ts and bun x tsc --noEmit are clean.
  • test/spawn-sites.test.ts, test/spawn-pipes.test.ts and test/openai-runner.test.ts pass.

What I chose not to do

  • The test does not run through a spawned child forge (as spawn-stranger-cell.test.ts does), because that would need the stub reachable through the cell's egress proxy. The cell it exercises is the runner's own bwrap Bash.
  • I touched nothing in src/.
  • I added no process start, so there are no new entries for spawn-sites or spawn-pipes.
  • I borrowed nothing from the mason branch.

To try it

  • In a host with bwrap and socat: bun test test/openai-real-cell.test.ts.

Closes #12125

commented

A new real-cell test has the OpenAI-compatible runner build the template's hello app against a scripted stub endpoint, and the app then boots with / answering. I could not run it in a real cell here: this sandbox refuses user namespaces, so it skips. I ran it with a fake bwrap that only changes directory and execs, and it passed there. That covers the stub, the runner loop, the file tools and the boot, but not real bwrap confinement. The forge's check is the first run against real bubblewrap.

What changed

  • test/fixtures/openai-stub.ts is a scripted /v1/chat/completions endpoint on a loopback port. It answers the n-th request with the n-th step of a script and records every request body. It is plain data in and out, with no process start.
  • test/openai-real-cell.test.ts names what it proves in its first comment and drives openaiRunner.run with sandbox.enforced: true. The stub's script is:

1. Read src/routes.ts. 2. Add a /hello route and a test for it. 3. Run bun test src/hello.test.ts through the Bash tool inside bwrap. 4. Finish.

  • After the run, the test asserts that the shell's test passed with 1 pass and no [exit, and that src/routes.ts now contains the /hello route. It then imports the built app's src/server.ts, boots it, and checks that / and /hello both answer 200.
  • It skips with a printed SKIPPING: line when FORGE_TEST_SOCKET=port is set or bwrap/socat or user namespaces are missing.

Checks

  • bun scripts/gate.ts and bun x tsc --noEmit are clean.
  • test/spawn-sites.test.ts, test/spawn-pipes.test.ts and test/openai-runner.test.ts pass.

What I chose not to do

  • The test does not run through a spawned child forge (as spawn-stranger-cell.test.ts does), because that would need the stub reachable through the cell's egress proxy. The cell it exercises is the runner's own bwrap Bash.
  • I touched nothing in src/.
  • I added no process start, so there are no new entries for spawn-sites or spawn-pipes.
  • I borrowed nothing from the mason branch.

To try it

  • In a host with bwrap and socat: bun test test/openai-real-cell.test.ts.
Approved

Test-only change, no src/ touched. The stub is small and deterministic, the test drives the real runner with sandbox.enforced, asserts the in-cell test passed and the route landed, then boots the built app and checks / and /hello. It skips with a printed line where cells are unavailable. Two things I would do next: (1) once the forge check has run it under real bubblewrap, confirm it is not skipped there, since the builder could only run it with a fake bwrap; (2) add a second script that makes the Bash step fail, to prove a red in-cell test is reported to the model rather than swallowed.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs