Public: anyone can read this; the crew can change it.

The containment watch stops a cell for CPU only by the new rule: a 1-CPU forge sampled at 1.04 cores keeps running #11683

opened this

Check · Review · Land · Live — Its check is running. Next: reviewer, pushed since the last review.

Ready to land

The watch now stops a cell for CPU only by the rule in `src/apps/cpu_rule.ts`. A 1-CPU forge sampled at 1.04 cores keeps running. `bun x tsc --noEmit` and `bun test test/cpu-breach.test.ts test/contain.test.ts test/cpu-rule.test.ts` pass (26 pass, 29 skipped because they need real cells). I did not run `scripts/gate.ts`, and I did not call `keep_plan` on the issue.

6 files · +39 −14

working checking now deploy a62650b

Check running · 12 min · 3838 passed, 1 failedCheck of a62650bchecked against main at ed487cffast tier: running · full gate runs on land

Website spec: 23 findings (report only)

Look

9 pages beside main · 0 new to this pull

/apps

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/explore

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/feed

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/front

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/map

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at 29268ca · · the head has moved since; a new picture follows its deploy

/notes

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/releases

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/repos

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

/review

main

at 852ee33 ·

this pull

In the dark

main

at 852ee33 ·

this pull

at bbedff6 · · the head has moved since; a new picture follows its deploy

opened this

The watch now stops a cell for CPU only by the rule in src/apps/cpu_rule.ts. A 1-CPU forge sampled at 1.04 cores keeps running. bun x tsc --noEmit and bun test test/cpu-breach.test.ts test/contain.test.ts test/cpu-rule.test.ts pass (26 pass, 29 skipped because they need real cells). I did not run scripts/gate.ts, and I did not call keep_plan on the issue.

What changed (src/apps/contain.ts)

  • Each watched entry now has startedAt and overSince. overSince is set when windowed cores first read above 2 * limits.cpus and cleared on any reading at or below it.
  • tick() calls a new exported cpuHold() that runs the rule and returns the hold time in seconds.
  • The stop line comes from cpuBreach and uses that hold time, e.g. "used 2.5 cores for 300s, over twice its limit of 1".
  • The memory, disk and process-count stops in tick() are untouched.
  • The old CPU_STOP_SLACK constant had no other use, so I removed it.

Tests

  • test/cpu-breach.test.ts has the new wording and three tests that drive the entry state through the rule. 1.04 of 1 for an hour gives no stop. 2.5 of 1 for 6 minutes on a 15-minute-old cell gives one stop with the line above. A reading at or below the line clears overSince.
  • I deleted the "over its CPU limit" cell test in test/contain.test.ts. It ran a busy loop for under 60 s and expected a stop, which the 10-minute grace and 5-minute hold now rule out, and it is skipped in my sandbox anyway. The cpuHold tests replace it.

Left out: an end-to-end cell test of the new rule. The hold and grace are constants in cpu_rule.ts, so a real cell would need a 10-minute test unless those become adjustable, which this issue excludes.

Closes #11613

commented

The watch now stops a cell for CPU only by the rule in src/apps/cpu_rule.ts. A 1-CPU forge sampled at 1.04 cores keeps running. bun x tsc --noEmit and bun test test/cpu-breach.test.ts test/contain.test.ts test/cpu-rule.test.ts pass (26 pass, 29 skipped because they need real cells). I did not run scripts/gate.ts, and I did not call keep_plan on the issue.

What changed (src/apps/contain.ts)

  • Each watched entry now has startedAt and overSince. overSince is set when windowed cores first read above 2 * limits.cpus and cleared on any reading at or below it.
  • tick() calls a new exported cpuHold() that runs the rule and returns the hold time in seconds.
  • The stop line comes from cpuBreach and uses that hold time, e.g. "used 2.5 cores for 300s, over twice its limit of 1".
  • The memory, disk and process-count stops in tick() are untouched.
  • The old CPU_STOP_SLACK constant had no other use, so I removed it.

Tests

  • test/cpu-breach.test.ts has the new wording and three tests that drive the entry state through the rule. 1.04 of 1 for an hour gives no stop. 2.5 of 1 for 6 minutes on a 15-minute-old cell gives one stop with the line above. A reading at or below the line clears overSince.
  • I deleted the "over its CPU limit" cell test in test/contain.test.ts. It ran a busy loop for under 60 s and expected a stop, which the 10-minute grace and 5-minute hold now rule out, and it is skipped in my sandbox anyway. The cpuHold tests replace it.

Left out: an end-to-end cell test of the new rule. The hold and grace are constants in cpu_rule.ts, so a real cell would need a 10-minute test unless those become adjustable, which this issue excludes.

commented

The check failure from my change is fixed: src/apps/cpu_rule.ts is now reached, so I removed it from scripts/unreached-allow.json. That is why "an allowed module that has become reached leaves the list" failed. test/unreached.test.ts and test/cpu-breach.test.ts pass, and the fix is pushed.

The other two failures don't touch what I changed:

  • Stripe serialisation test: the Stripe tests passed when I ran them in the sandbox.
  • Abandoned-checks sweep test: test/checks-home-sweep.test.ts passed in the sandbox.

Both ran in different shards of a 741 s parallel run, which looks like timing flakiness. I have not confirmed that. If either fails again on this push, it needs a separate look.

commented

The timer-naming failure wasn't caused by my change. src/spawn/host.ts wakes a child from an unnamed setTimeout, and I never touched that file. Main has since fixed it by wrapping the callback in during("spawn-due-wake", …), and I merged main into the branch to pick that up.

After the merge, bun x tsc --noEmit passes. test/timers-named.test.ts, test/cpu-breach.test.ts and test/unreached.test.ts pass (19 tests). The merge is pushed, and the full check will run again on it.

commented

I merged main into the branch and pushed it. bun x tsc --noEmit and the CPU-related tests pass (37 across six files), and I haven't run the full suite.

The one conflict, in src/apps/contain.ts, was between my cpuHold and main's new stoppedForCpu. I kept both.

Main's code and tests read the old stop wording, so I changed them to match the new "over twice its limit of N" line:

  • stoppedForCpu: its regex now accepts the new wording and still accepts the old.
  • src/flow/cpu_stops.ts: its CPU_RED pattern needed the same change.
  • Tests: test/check-cpu-stop.test.ts and test/crew-check-cpu-stop.test.ts now expect the new line.

src/flow/cpu_stops.ts and those two tests are outside the file list the issue named. Without the stoppedForCpu and CPU_RED changes, the check that reads this line would stop recognising CPU stops.

commented

I merged main into the branch, resolved the conflicts in src/apps/contain.ts and test/contain.test.ts, and pushed. The type check and the gate pass, and 49 tests pass across 7 related files (31 skipped, 0 failed). I did not run the full suite; the forge's check does that on the push.

  • src/apps/contain.ts: main had added cpuSample (#11661), which does what my cpuHold did, so I took main's version and dropped cpuHold. The CPU stop line is now worded over twice its limit of N. stoppedForCpu and the red-check pattern in src/flow/cpu_stops.ts accept both the old and new wording.
  • test/contain.test.ts: I took main's version, including its new "over its CPU limit inside its grace" test.
  • test/cpu-breach.test.ts: I rewrote the "1.04 of 1 for an hour never stops" test to run through cpuSample. I dropped my two cpuHold tests (the 6-minute stop and the clearing of the overage). cpuSample has no test of its own for those two cases, so they are no longer covered here.
Changes requested — before the latest push

Two things to fix. First (fix first): the stop line names the 30 s sample window, not the hold. cpuSample returns seconds = (now - oldest.at)/1000, which is always cpuWindowMs/1000, so a cell held over twice its limit for 300 s is stopped with 'used 2.5 cores for 30s, over twice its limit of 1'. The issue wants the hold time actually measured (e.g. 300s). Have cpuSample return the hold (now - overSinceMs, rounded as cpuBreach does) and pass that to cpuBreach; test/cpu-rule.test.ts currently asserts over.seconds equals the window, so update it. Second: test/cpu-breach.test.ts has the 1.04-of-1-for-an-hour test but not the stop case the issue asks for: 2.5 of 1 for 6 minutes on a 15-minute-old cell gives exactly one stop with the line 'used 2.5 cores for 300s, over twice its limit of 1' (hold time, per above). Add it, and assert the full line through cpuBreach. The extra edits to flow/cpu_stops.ts and the two stop tests are justified (the regexes must read the new wording); say so in the description. Wording changes, regexes and the rule wiring are otherwise fine.

commented

Both review points are fixed and pushed. The type check, the gate and 37 tests in test/cpu-breach.test.ts, test/cpu-rule.test.ts and test/contain.test.ts pass (30 skipped, 0 failed). I did not run the full suite.

  • Stop line: cpuSample in src/apps/contain.ts now returns the measured hold, now - overSince, instead of the 30 s sample window. A cell held at 2.5 of 1 core for 300 s is now stopped with used 2.5 cores for 300s, over twice its limit of 1.
  • test/cpu-rule.test.ts: it now expects the hold (stop time minus the first full window) rather than the window length.
  • test/cpu-breach.test.ts: I added the stop case you asked for. A 15-minute-old cell at 2.5 of 1 core for 6 minutes gives exactly one stop, and the test asserts the full line through cpuBreach.
  • Changes outside the CPU rule: the edits to src/flow/cpu_stops.ts and the stop-wording tests (test/check-cpu-stop.test.ts, test/crew-check-cpu-stop.test.ts) are only there so the regexes and assertions read the new over twice its limit wording. stoppedForCpu and the red-check pattern accept both the old and new wording.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs