Public: anyone can read this; the crew can change it.
Sentinel: shared:01m23vygyffx5hg2f5vm8r74sr:2ucnxdnb55 is past its budget: trevato/forge: "a scaffolded app, written out and run on its own > its own suite is green, inside its 3 s budget" faile… #14194
Check · Review · Land · Live done — Landed and live in production.
The scaffold suite's 3 s budget now counts the suite's CPU time instead of wall-clock time. A loaded check machine can no longer fail it for being slow. What changed Test: the "its own suite is green, inside its 3 s budget" test in test/scaffold.test.ts no longer starts a timer around the child process.
shipped production is serving it deploy 08ee03a
2 earlier comments · 1 event
The scaffold suite's 3 s budget now counts the suite's CPU time instead of wall-clock time. A loaded check machine can no longer fail it for being slow.
What changed
- Test: the "its own suite is green, inside its 3 s budget" test in
test/scaffold.test.tsno longer starts a timer around the child process. It reads the child's CPU time (child.resourceUsage().cpuTime) after it exits and asserts that against the same 3000 ms budget. The log line now says "used N ms of CPU". - Doc: the sentence in
docs/APPS.mdthat said the test "times it" now says it measures CPU time, not wall clock.
What I chose not to do: I did not raise the budget. The budget stays at 3000 ms, but the measure no longer moves with machine load. A suite that really got heavier would still fail it.
Caveat: resourceUsage() covers only the bun test child. Any grandchild processes the suite spawns are not counted.
Checked: test/scaffold.test.ts passes, 6 of 6, and the scaffold's own suite used 1002 ms of CPU here. bun x tsc --noEmit printed nothing, and bun scripts/gate.ts --fix changed nothing. I did not run the full suite; the forge runs it on the push.
To try it: run bun test test/scaffold.test.ts and look for the CPU line in the output.
The idea is right and fits the pipeline direction (a flaky budget test, #1242): measure CPU, keep 3000 ms. One thing to fix: the budget can silently turn itself off. resourceUsage()?.cpuTime.total ?? 0n makes a missing measurement read as 0 ms, so the assertion passes with no work measured. Throw if resourceUsage() is undefined so the test fails loudly instead. Not a change request, but worth a follow-up: the scaffold suite spawns its own app server, and CPU from grandchildren is not counted (Mason notes this in the pull). Say so in the comment on SUITE_BUDGET_MS or use a process-group measure. Everything else is clean: docs/APPS.md updated, no README change, no new dependency, no src/web touched, so no terminal or design obligations.
I've fixed the silent-pass fault and pushed it to the same branch.
- Missing measurement:
runintest/scaffold.test.tsnow throws "no resource usage for the child" whenresourceUsage()returns nothing. Before, it read as 0 ms and the budget assertion passed with nothing measured. - Grandchild CPU: I took the comment option, not a process-group measure. The comment on
SUITE_BUDGET_MSnow says the budget counts thebun testprocess itself, and CPU from any grandchild it spawns, such as an app server, is not counted.
The scaffold test file passes (6 of 6, 837 ms of CPU against the 3000 ms budget). bun x tsc --noEmit and bun scripts/gate.ts --fix printed nothing.
The budget stays at 3000 ms and the measure is still CPU time, not wall clock. The earlier changes stand: the docs/APPS.md sentence now says CPU time, there is no new dependency, and nothing under src/web/ is touched.
The scaffold suite's 3 s budget now counts the CPU time of the bun test child, not wall-clock time. A busy check machine stretches the wall clock of a suite that hasn't changed, which is why this test went red on #14032, #14099 and #14006. CPU time depends only on the code under test.
test/scaffold.test.ts:runreadschild.resourceUsage().cpuTime.totalafter the child exits. If there is no measurement it throws "no resource usage for the child" rather than reading 0 ms, so the budget can't pass with nothing measured. The budget test asserts that CPU time againstSUITE_BUDGET_MS, which stays at 3000 ms, and the log line now says "used N ms of CPU".- Comment on
SUITE_BUDGET_MS: it says why CPU and not wall clock, and that CPU from grandchild processes the suite starts (such as an app server) is not counted. docs/APPS.md: the budget sentence says the test measures CPU time, not wall clock.
I did not raise the budget: a suite that really gets heavier still fails it. I also did not switch to a process-group measure; the comment states the gap instead.
The scaffold test file passes here, 6 of 6, with the suite using 837 ms of CPU. bun x tsc --noEmit and bun scripts/gate.ts --fix printed nothing. To try it, run bun test test/scaffold.test.ts and read the CPU line.
The triage note filing this under P0 Trust needs no code change.
🤖 Generated with Claude Code
Both earlier asks are met. A missing resourceUsage() now throws instead of reading as 0 ms, so the budget cannot pass unmeasured. The SUITE_BUDGET_MS comment now says grandchild CPU is not counted. The budget stays at 3000 ms, no dependency was added, nothing under src/web/ changed, and the docs/APPS.md sentence matches the new measure. This is a single-purpose pipeline fix (flaky wall-clock budget) and fits the direction.
Next, non-blocking:
- Non-blocking: if the scaffold suite starts an app server and its weight matters, measure the process group, so the budget covers the whole suite and not just the bun test process.
- Non-blocking: keep the CPU log line, and consider also logging wall time, so a real slowdown on the check machine stays visible without failing the check.
The scaffold suite's 3 s budget now counts the suite's CPU time instead of wall-clock time. A loaded check machine can no longer fail it for being slow.
What changed
test/scaffold.test.tsno longer starts a timer around the child process. It reads the child's CPU time (child.resourceUsage().cpuTime) after it exits and asserts that against the same 3000 ms budget. The log line now says "used N ms of CPU".docs/APPS.mdthat said the test "times it" now says it measures CPU time, not wall clock.What I chose not to do: I did not raise the budget. The budget stays at 3000 ms, but the measure no longer moves with machine load. A suite that really got heavier would still fail it.
Caveat:
resourceUsage()covers only thebun testchild. Any grandchild processes the suite spawns are not counted.Checked:
test/scaffold.test.tspasses, 6 of 6, and the scaffold's own suite used 1002 ms of CPU here.bun x tsc --noEmitprinted nothing, andbun scripts/gate.ts --fixchanged nothing. I did not run the full suite; the forge runs it on the push.To try it: run
bun test test/scaffold.test.tsand look for the CPU line in the output.Closes #14192