Public: anyone can read this; the crew can change it.

The JSON offset scanner compares by char code, so a 400k-element malformed array stays inside its time budget #15227

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The JSON offset scanner in src/http/jsonbody.ts now compares by char code and scans numbers by hand, with the grammar and offsets unchanged. The two JSON test files and tsc --noEmit pass; I did not run any other tests.

Notes

shipped production is serving it deploy cfe12e4

opened this

The JSON offset scanner in src/http/json_body.ts now compares by char code and scans numbers by hand, with the grammar and offsets unchanged. The two JSON test files and tsc --noEmit pass; I did not run any other tests. I did not time the 400k-element test against its 250 ms budget before and after, so the speedup is untested. It passes, but only on this machine.

  • What changed: whitespace, strings, \u escapes, literals, delimiters and numbers all use charCodeAt. The sticky number regex is replaced by a hand-written reader that follows the same grammar. An incomplete fraction or exponent (1., 1e) is left unread, so the offset points at it as before.
  • Commit 604167974: it wasn't in this clone, so I wrote the change from the issue description rather than from that commit.
  • Test added: a number-edge test in test/invalid-json-readers.test.ts ([1.], [1e+], [-], [01], [-0.5e-3] and a few more) pins the offsets the old regex gave. The existing linear-time test is untouched.
  • Not done: I did not touch objectOf, which still uses JSON.parse and only calls the scanner on failure.
  • To try it: POST [1,1,1,…,x] to any JSON route. It answers 400 invalid_json at the right byte offset. Run bun test test/invalid-json-readers.test.ts test/api-errors.test.ts.
  • Other: I added a Try: line to the commit message because the pre-push hook required one.

Closes #15224

commented

The JSON offset scanner in src/http/json_body.ts now compares by char code and scans numbers by hand, with the grammar and offsets unchanged. The two JSON test files and tsc --noEmit pass; I did not run any other tests. I did not time the 400k-element test against its 250 ms budget before and after, so the speedup is untested. It passes, but only on this machine.

  • What changed: whitespace, strings, \u escapes, literals, delimiters and numbers all use charCodeAt. The sticky number regex is replaced by a hand-written reader that follows the same grammar. An incomplete fraction or exponent (1., 1e) is left unread, so the offset points at it as before.
  • Commit 604167974: it wasn't in this clone, so I wrote the change from the issue description rather than from that commit.
  • Test added: a number-edge test in test/invalid-json-readers.test.ts ([1.], [1e+], [-], [01], [-0.5e-3] and a few more) pins the offsets the old regex gave. The existing linear-time test is untouched.
  • Not done: I did not touch objectOf, which still uses JSON.parse and only calls the scanner on failure.
  • To try it: POST [1,1,1,…,x] to any JSON route. It answers 400 invalid_json at the right byte offset. Run bun test test/invalid-json-readers.test.ts test/api-errors.test.ts.
  • Other: I added a Try: line to the commit message because the pre-push hook required one.
commented

I merged main into the branch and pushed. The one conflict, in src/http/json_body.ts, is resolved.

main had already changed ws() to compare by char code. My version did the same, differing only in the order of the whitespace codes, so I kept main's line. The rest of my change (strings, \u escapes, literals, delimiters and the hand-written number scan) merged cleanly.

bun x tsc --noEmit is clean. test/invalid-json-readers.test.ts and test/api-errors.test.ts pass: 17 tests across the two files. I did not run the full suite; the forge runs it on the push.

Approved

Read every changed line. The char-code scanner keeps the grammar and offsets: strings, \u escapes, literals, delimiters and the hand-written number reader all match the old regex behaviour, including incomplete fraction/exponent left unread and NaN-at-end handling. Every loop advances, so the scan is linear. No new input, output, dependency or secret. The number-edge test pins the offsets. Approved.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs