Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

GET /data?from&to&step[&compare=1] and ?turn=<id> serve the facts as series for owner and moderators (src/data-series.ts, src/data-api.ts) #550

Merged opened this

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

GET /data is in and pushed. It gives the owner and moderators the facts as series, and a single turn when you pass ?turn= . The bun --cwd Software/so101 test run before my last fix showed 2 failures, both of which that fix addressed.

Notes

shipped merged

opened this

GET /data is in and pushed. It gives the owner and moderators the facts as series, and a single turn when you pass ?turn=<id>. The bun --cwd Software/so101 test run before my last fix showed 2 failures, both of which that fix addressed. After it I re-ran only the facts, data and relay tests, which pass (53), and the type check shows nothing new from this change. I did not re-run the whole suite after that fix. The type check does report errors in other files (hops.ts, servo-baseline.ts and some UI and test files) that I didn't touch.

I couldn't copy src/data-series.ts and src/data-api.ts from 74967da as they were. That branch was written against a different facts.ts (separate metrics, hops and joints), so I rewrote both against the facts.ts on main.

  • src/data-series.ts: the top of the file says how each number is made. Sums are added before dividing, a step's p95 is the p95 of its turns' p95s, and uptime is live minutes over minutes passed. I dropped what main's facts don't record: hop and joint series, latency, the worst-turns list and the Pi-version markers. Version markers are now just each change of app commit.
  • src/data-api.ts: 403 to anonymous visitors and plain users, 200 to the owner and moderators. A bad from, to, step, compare or turn is a 400 that says what to send, and an unknown turn is a 404. ?compare=1 adds the previous period. ?turn=<id> returns the turn plus the arm-related events and samples from a minute either side.
  • src/facts.ts: turns(), events() and samples() now take an optional from/to. I added turnById() and oldest(), and exported EventRow and SampleRow.
  • src/relay.ts: routes /data to serveData.
  • Tests: the two test files are rewritten for these shapes. Several of the old cases (hops, joints, worst turns, the /admin/data page) have no counterpart on main and are gone.

Check the GET /data route in src/relay.ts. I did not exercise it against a running preview.

🤖 Generated with Claude Code

Closes #529

3 earlier comments · 1 event
commented

GET /data is in and pushed. It gives the owner and moderators the facts as series, and a single turn when you pass ?turn=<id>. The bun --cwd Software/so101 test run before my last fix showed 2 failures, both of which that fix addressed. After it I re-ran only the facts, data and relay tests, which pass (53), and the type check shows nothing new from this change. I did not re-run the whole suite after that fix. The type check does report errors in other files (hops.ts, servo-baseline.ts and some UI and test files) that I didn't touch.

I couldn't copy src/data-series.ts and src/data-api.ts from 74967da as they were. That branch was written against a different facts.ts (separate metrics, hops and joints), so I rewrote both against the facts.ts on main.

  • src/data-series.ts: the top of the file says how each number is made. Sums are added before dividing, a step's p95 is the p95 of its turns' p95s, and uptime is live minutes over minutes passed. I dropped what main's facts don't record: hop and joint series, latency, the worst-turns list and the Pi-version markers. Version markers are now just each change of app commit.
  • src/data-api.ts: 403 to anonymous visitors and plain users, 200 to the owner and moderators. A bad from, to, step, compare or turn is a 400 that says what to send, and an unknown turn is a 404. ?compare=1 adds the previous period. ?turn=<id> returns the turn plus the arm-related events and samples from a minute either side.
  • src/facts.ts: turns(), events() and samples() now take an optional from/to. I added turnById() and oldest(), and exported EventRow and SampleRow.
  • src/relay.ts: routes /data to serveData.
  • Tests: the two test files are rewritten for these shapes. Several of the old cases (hops, joints, worst turns, the /admin/data page) have no counterpart on main and are gone.

Check the GET /data route in src/relay.ts. I did not exercise it against a running preview.

🤖 Generated with Claude Code

commented

I pushed a likely fix for the failing GET /data test, but I could not reproduce the failure, so the cause is a guess. The data-api tests passed on every run here, including six at once. I have not run the full suite since the fix.

My best explanation is a one-millisecond boundary. The default range ends at the request time and excludes its last millisecond. The test writes its turn and STOP event and then requests /data straight away. If both land in the same millisecond, those rows fall outside the range. That would break the sessions check on a fast machine.

serveData now defaults its now to Date.now() + 1, so a row written in the current millisecond is counted. Explicit to values are unchanged.

If #550's preview fails on this test again, its log will show which assertion failed, and I'll look at that instead.

🤖 Generated with Claude Code

commented

I merged main into the branch and pushed; the only conflict was in Software/so101/src/facts.ts.

On main, samples() now returns the optional piTemp and piUndervolt fields. On this branch, events() and samples() take an optional from/to range. I kept the range-taking versions and added the two Pi fields to SampleRow, so both sets of changes work together. The Pi columns on main are still read and written as before.

Only the facts, data and relay tests were re-run after the merge; they pass (56). The type check shows nothing new in the files I touched. I did not re-run the whole suite, so the forge's check on this push will be the first full run.

🤖 Generated with Claude Code

Changes requested — before the latest push

The route and series code are sound. Anonymous gets 403 on the preview, identity goes through lobby.who and lobby.moderates, the parameters are validated with clear 400s, and the tests cover the behaviour. One item on the checklist is missing.

Fix first: the README does not mention GET /data. It needs a short section giving the parameters (from, to, step, compare=1, turn=<id>), who may call it (owner and moderators, 403 otherwise), and an example response shape. The relay.ts header comment is not a substitute, because the README is where a person looks for how to use it.

Two smaller points, not blocking if you disagree. The issue is a JSON read, so I did not ask for a tty command. If the owner expects one, say so in the pull. Also, the whole suite was not re-run after the merge, so wait for the forge check log.

commented

I added a "Reading the facts: GET /data" section to Software/so101/README.md and pushed it. No code changed, so I ran no tests for this commit.

The section gives:

  • the parameters from, to, step, compare=1 and turn=<id>, with defaults and limits;
  • who may call it: the owner and moderators, 403 for anyone else;
  • the 400, 404 and 405 answers;
  • the keys of a range answer and of a ?turn= answer.

There is no tty command for /data. It is a JSON read, and the issue asked for nothing more.

I have not re-run the whole suite after the merge. I'm waiting on the forge's check log for that.

🤖 Generated with Claude Code

Approved

Approved. Preview: /data answers 403 JSON with the security headers to an anonymous caller (all I can reach without the forge identity). The code gates owner and moderators through lobby.moderates, validates every parameter with a 400 that says what to send, caps steps at 400, returns 404 for an unknown turn, 405 for POST, and names no one on a turn timeline. Both test files cover the behaviour and the README documents it.

Two things I would do next:

  1. Add a test for HEAD and for a moderator on ?turn=. Only the owner is tried there.
  2. In serveData the 405 check runs after the 403, so a guest learns nothing about methods. Keep that order, but add a test that pins it.

No tty door: the issue asks for a JSON read only and the README says so. That is acceptable here.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs