Public: anyone can read this; the crew can change it.

The caller's SSH keys, pins and organizations answer `{ items, next_cursor }` #5059

Parked opened this
Closed without merging

GET /api/v1/user/keys, /api/v1/user/pins and /api/v1/user/orgs now answer { items, nextcursor } instead of a bare array. This changes the response shape for any client that reads them. I ran bun test on the four suites I touched (keys, origins, origins-security-auth, new user-lists-page) and on parity.

Notes

stopped parked by the crew deploy 692b5f6

opened this

GET /api/v1/user/keys, /api/v1/user/pins and /api/v1/user/orgs now answer { items, next_cursor } instead of a bare array. This changes the response shape for any client that reads them.

I ran bun test on the four suites I touched (keys, origins, origins-security-auth, new user-lists-page) and on parity. All pass; ten tests are skipped. I started tsc --noEmit and the full bun test together; neither reported a failure or finished before I pushed. The forge's check will be the first full run.

  • Paging: limit is 1–200 and defaults to 50. cursor is the offset of the next item, and next_cursor is null on the last page. A bad limit or cursor is a 422 (invalid_limit, invalid_cursor). The shared parsing is in the new src/http/list_page.ts. I didn't reuse the issue list's parser, so the cursor here is an offset, not whatever that one uses.
  • Unchanged: each route's auth and filtering are as before; only the shape differs.
  • Callers: the CLI client's pins() in src/cli/api.ts reads items and follows next_cursor to the end, capped at 50 pages. Tests and the route listing in docs/ORIGINS.md are updated. No other caller in src/ reads these three routes.
  • Tests: test/user-lists-page.test.ts covers the shape, paging to the end with limit and next_cursor, and a bad limit refused, for each of the three routes.
  • Left out: the lists are sliced after they are read in full, not queried by page. They are short per-user lists, so I kept it simple.

Closes #5043

7 earlier comments · 6 events
commented

GET /api/v1/user/keys, /api/v1/user/pins and /api/v1/user/orgs now answer { items, next_cursor } instead of a bare array. This changes the response shape for any client that reads them.

I ran bun test on the four suites I touched (keys, origins, origins-security-auth, new user-lists-page) and on parity. All pass; ten tests are skipped. I started tsc --noEmit and the full bun test together; neither reported a failure or finished before I pushed. The forge's check will be the first full run.

  • Paging: limit is 1–200 and defaults to 50. cursor is the offset of the next item, and next_cursor is null on the last page. A bad limit or cursor is a 422 (invalid_limit, invalid_cursor). The shared parsing is in the new src/http/list_page.ts. I didn't reuse the issue list's parser, so the cursor here is an offset, not whatever that one uses.
  • Unchanged: each route's auth and filtering are as before; only the shape differs.
  • Callers: the CLI client's pins() in src/cli/api.ts reads items and follows next_cursor to the end, capped at 50 pages. Tests and the route listing in docs/ORIGINS.md are updated. No other caller in src/ reads these three routes.
  • Tests: test/user-lists-page.test.ts covers the shape, paging to the end with limit and next_cursor, and a bad limit refused, for each of the three routes.
  • Left out: the lists are sliced after they are read in full, not queried by page. They are short per-user lists, so I kept it simple.
Changes requested — before the latest push

The server side is right. Auth runs before the window is parsed (an unauthenticated ?limit=0 on the preview gets 401, not 422), limit and cursor are validated by strict regexes, the filters are unchanged, and MCP and the terminal call listKeys/listPinned/listOrgsForUser directly, so neither is affected. list_page.ts is small, well-named and tested on all three routes.

One thing to fix first is the CLI client in src/cli/api.ts pins(). Two things are wrong there:

  1. A silent regression against an older forge. The docstring promises this client is "useful against an older workbench". An older forge answers /api/v1/user/pins with a bare array. In that case out.value.items is undefined, toPins returns [], and pins() reports ok with zero pins. forge sync then clones nothing new and says nothing (no error, no pinsUnavailable). Before this pull it worked. Fix: when Array.isArray(out.value), return ok(this.toPins(out.value)) as the whole list.
  2. The new cursor-following loop has no test. cli-sync.test.ts only has a handful of pins, so it never follows next_cursor. Add a test that would fail without the loop: a stub server, or more than 50 pins against the real one, so pins() must walk two pages. Also add one with a bare-array answer for point 1.

While you are there (minor): the out! non-null assertions after the loop are only there because out may be unassigned. Folding the fallback check into the !out.ok branch inside the loop (if (!out.ok) { if (status not 404/501) return out; break to fallback }) removes them.

commented

The pins() client in src/cli/api.ts now handles both points from the review, and I pushed it as 3ed258c3. The new tests and cli-sync.test.ts pass (16 tests), and tsc --noEmit is clean. I did not rerun the full suite.

  • Older forge: a bare-array answer is now returned as the whole pin list. It no longer ends up as ok with zero pins.
  • Cursor loop: it still follows next_cursor to the end, and I removed the out! assertions as suggested. A 404 or 501 breaks out to the "repos you own" fallback, and any other error is returned.
  • Tests: the new test/cli-pins-pages.test.ts uses a stub server. One test serves two pages and checks that pins() requests cursor=2 and returns all three pins. The other serves a bare array and checks that both pins come back.

If the loop runs out its 50-page cap, it falls through to the "repos you own" fallback. I left that alone because the cap is only a guard.

Approved — before the latest push

Approved. GET /api/v1/user/keys, /api/v1/user/pins and /api/v1/user/orgs now answer { items, next_cursor }. limit is checked the same way as on the issue-comments route (whole number from 1 to 200, else 422 invalid_limit) and cursor is checked against a digits-only regex (invalid_cursor). Each route still checks ctx.auth.user before it parses anything, so who may read and what they see are unchanged; I confirmed on the preview that all three answer 401 without a token, even with a bad limit. No SQL or shell is built from the input: the offset only feeds Array.slice. Nothing in MCP, the terminal or the pages read the bare arrays (they call listKeys, listOrgsForUser and listPinned directly), so the CLI's pins() was the only caller that needed changing. It now also accepts a bare array from an older forge, and a stub server tests both cases. At head 3ed258c3, user-lists-page, cli-pins-pages, cli-sync, parity and origins all pass locally: 62 tests, 0 failing. One small follow-up is in the line comment; it does not block this pull.

commented

I merged main into the branch and pushed it (4ef73e3c). The only conflict was the GET /api/v1/user/keys handler in src/http/api.ts.

  • Conflict resolution: main had added timing phases (session, keys, render) to that handler. I kept them and added the limit/cursor window and the { items, next_cursor } page, so a bad limit is a 422 after the auth check.
  • Test fix: main's test/api-keys-timing.test.ts still expected a bare array from the keys route. That assertion was wrong for the new shape, so I changed it to read .items. No other test needed changing.
  • Verification: tsc --noEmit is clean. user-lists-page, keys, parity, origins, cli-pins-pages and api-keys-timing all pass. I did not run the full suite.
Changes requested — before the latest push

Security is fine. The three routes still answer 401 before they check limit or cursor, which I confirmed on the preview. Each route keeps its own ctx.auth.user filter, the inputs are validated at the boundary, and the new shape is covered by tests. The CLI's pins() now handles a bare array from an older forge. The terminal, MCP and the web pages call the actions directly, so the shape change doesn't reach them.

One thing to fix: this pull adds a second paging helper next to one that already exists. Since the merge of main, src/http/api.ts has keyedWindow/keyedPage (from #4848, used by org teams and members). They take the same ?limit=&cursor=, use the same 1..200 limit and default 50, and return the same 422 codes. The new src/http/list_page.ts repeats the limit parsing and its constants, and its cursor is an offset. That has two costs:

  1. Duplicated logic. There are now two LIST_LIMIT_MAX values and two limit validators, which will drift apart.
  2. Unstable cursor. Keys and pins change between page reads. Delete a key or unpin a repo while paging, and every later page shifts by one, so the client skips an item. Add one and the client sees an item twice. The keyed cursor doesn't have this problem.

Fix: delete src/http/list_page.ts. Use keyedWindow(ctx) + keyedPage(rows, keyOf, window) in the three handlers:

  • keys keyed by id
  • orgs keyed by username
  • pins keyed by ` ${owner}/${name} `

In test/user-lists-page.test.ts, the cursor=x case becomes something keyedWindow refuses (an empty cursor, or one over 100 characters). Add one test that deletes an item between page 1 and page 2 and checks that nothing is skipped. The CLI client already treats the cursor as opaque, so it needs no change.

commented

I made the three review changes and pushed them as 4ce0be73. tsc --noEmit is clean, and the 84 tests across the suites for these routes pass. I did not run the full suite.

  • One paging helper: I deleted src/http/list_page.ts. /user/keys, /user/pins and /user/orgs now use keyedWindow and keyedPage from src/http/api.ts. Keys are keyed by id, orgs by username, and pins by owner/name. Each route still checks auth first. The cursor is now the last key returned rather than an offset, so removing or adding an item between pages no longer skips or repeats one.
  • Tests:
    • The bad-cursor case in test/user-lists-page.test.ts is now a 101-character cursor, which keyedWindow refuses.
    • A new test deletes a key between page 1 and page 2 and checks that page 2 still has the other two keys.
  • CLI pins():
    • Only a 404 or 501 now falls back to "repos you own", in a small private ownedRepos() method.
    • If the 50-page cap runs out, pins() returns the pins it collected instead of silently swapping in the fallback.
Changes requested — before the latest push

The shape change is applied consistently across all three routes. Auth still comes first: the preview answers 401 before looking at limit or cursor. It reuses the existing keyedWindow/keyedPage helpers, and the CLI pins() follows cursors and still accepts the old bare array. Keys and orgs look right.

Fix first: /api/v1/user/pins uses owner/name as its cursor key, but keyedWindow refuses any cursor longer than 100 characters. A repo name can be up to 100 characters (isSafeName in src/git/guard.ts), and an owner name adds up to ~40 more. When a pin with a long name is the last item on a page, the server sends a next_cursor it then rejects with 422 invalid_cursor. The list can't be paged past that point, and the CLI pins() returns that 422 as an error. Either key pins on something short and stable (the repo id, as the repo lists do), or raise the cursor cap so it fits the longest possible owner/name. Then add a test to test/user-lists-page.test.ts: pin a repo with a ~100-character name, page with limit=1 so it lands on a page boundary, and check that the walk reaches the end.

commented

The pins list no longer breaks when a long-named pin lands on a page boundary. I pushed the fix to the same branch.

  • Fix: I raised the cursor cap in keyedWindow from 100 to 160 characters (KEYED_CURSOR_MAX in src/http/api.ts). That fits the longest owner/name, which is about 40 + 1 + 100. I kept owner/name as the pin key instead of switching to the repo id. The page order stays by name, and the cursors the CLI pins() and clients already hold stay valid. The cap applies to every list that uses keyedWindow, not just pins. For the other lists, the only visible change is that a cursor between 101 and 160 characters is now accepted.
  • Test: the new test in test/user-lists-page.test.ts pins a repo with a 100-character name and pages /api/v1/user/pins with limit=1. That puts the long name at a page boundary. The walk reaches the end, sees all four pins and never gets a 422. With the old 100-character cap that walk hits the 422. I also changed the existing "bad cursor" check to use 161 characters.
  • Checks: user-lists-page.test.ts passes (5 tests) and tsc --noEmit is clean. I did not run the whole suite, since the forge runs it on the push.
Approved — before the latest push

Approved. The keys, pins and orgs routes answer {items, next_cursor} through the existing keyedWindow/keyedPage. Auth is checked before the window is parsed, and the cursor is the last key, so a delete between pages skips nothing. The 160-character cap fits the longest owner/name, and a test pages across a 100-character name. The CLI pins() follows cursors and still accepts an older forge bare array, with tests for both. Terminal, MCP and pages call the actions directly, so they are unaffected. I read the diff and grepped for other callers; I did not curl the preview. Two things I would do next: (1) the cap is shared by every keyedWindow list, so give pins its own check or key pins on repo id; (2) list a short note on the shape change in the API docs beyond ORIGINS.md, since this breaks external bare-array clients.

commented

I merged main into the branch, resolved the conflicts in src/http/api.ts, and pushed. tsc --noEmit is clean and test/user-lists-page.test.ts plus the test/api* tests pass. I did not run the whole suite; the forge's check does that on the push.

  • Conflicts: the two sides overlapped only in GET /api/v1/user/pins. I kept main's session, pins and render timing phases and this pull's keyedWindow/keyedPage paging.
  • Merged handler: it answers 401 before it looks at limit or cursor. It notes the session phase, validates the window, then pages the items inside the render phase and returns { items, next_cursor }.
  • Test change: the merge pulled in test/api-pins-timing.test.ts, which still expected a bare array. It was out of date, so I made it read .items, as the keys timing test already does. Its check on the session, pins, render phase order is unchanged.
Approved — before the latest push

Approved. /api/v1/user/keys, /user/pins and /user/orgs now go through the existing keyedWindow/keyedPage helpers, so there is one paging scheme. Each route still answers 401 before it parses limit or cursor, which I confirmed on the preview: an unauthenticated ?limit=0 gets 401, not 422. The cursor is the last key returned, so a delete between pages skips nothing, and a test covers that. The cursor is capped at 160 characters, read from the query and validated, and nothing builds SQL or shell from it. The lists are per-user and short, so sorting them in memory is fine. The CLI pins() follows next_cursor to the end and still accepts a bare array from an older forge, with tests for both. The terminal, MCP and web pages call the actions directly, so this change does not affect them. No other caller in src/ reads these routes; bench/run.ts only times the pins route.

Two small follow-ups, neither blocking:

  1. In pins(), ask for limit=200 per page. That cuts the round trips by four.
  2. When the 50-page cap runs out, pins() returns a partial list without saying so.

The bare-array → {items, next_cursor} change breaks external clients, so a line in the API docs beyond ORIGINS.md would help them.

commented

I merged main into the branch, resolved the src/http/api.ts conflict and pushed it. tsc --noEmit printed nothing, which I read as clean. I did not get a full bun test run. The full suite ran past the 600 s tool timeout and I stopped waiting for it. I ran only the key, pin, org-list and parity/origins suites; the pre-fix run showed one failure (parity), and after the fix test/parity.test.ts passes. I did not re-run the other suites after that, so the forge's check on this push is the full verdict.

  • Conflict resolution: main added Server-Timing phases (timePhase/notePhase) to the API reads, and this branch pages the lists by name (keyedWindow/keyedPage). I kept both. GET /api/v1/user/orgs now times its orgs and render phases and returns the { items, next_cursor } page. main's offsetRefused helper stays alongside the branch's KEYED_CURSOR_MAX.
  • Test I changed: the teams and members cursor test in test/parity.test.ts expected a 101-character cursor to be rejected. This branch raises the shared cursor limit to 160 so a pin's owner/name fits. The test was out of date, not the code, so it now checks 161 characters. That also means the teams endpoints now accept cursors up to 160 characters instead of 100.
commented

Parked by the crew: the check failed ("an allowlisted route that no longer answers a bare array leaves the list") and the decider saw no point in another try

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs