Public: anyone can read this; the crew can change it.

A .pcap capture shows its packets as a table: time, source, destination, protocol, length #14572

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A .pcap file in a repository now opens as a packet table. I haven't opened one in a running app. I ran only the render, kinds, fuzz and guide tests plus tsc, which pass.

Notes

shipped production is serving it deploy 9bdd6af

opened this

A .pcap file in a repository now opens as a packet table. I haven't opened one in a running app. I ran only the render, kinds, fuzz and guide tests plus tsc, which pass. The forge runs the full suite on push.

What changed

  • Parser: src/web/render/pcap.ts parses classic pcap by hand, in both byte orders and with microsecond or nanosecond stamps.
  • Table: it shows the first 500 packets with relative time, source, destination, protocol and length. IPv4 and IPv6 endpoints carry ports for TCP and UDP, and ICMP, ICMPv6 and ARP are named. A line says how many packets were not shown.
  • Link types: Ethernet (with VLAN tags), Linux cooked, loopback and raw IP are read.
  • Fallbacks: pcapng, an unknown link type, a bad signature and a cut-off header all fall back to the plain file view with one sentence, not an error page. A file that ends inside a packet keeps the packets before the cut and says so.
  • Bounds: the file is capped at 5 MB by RENDER_CAP, every read is checked against the bytes left, and the walk stops at 2 million records. A hostile record length is refused.
  • Wiring: a new pcap kind in kinds.ts and renderers.ts. It matches .pcap, .pcapng and .cap files that start with a capture's magic number. The file-kinds table in docs/features/file-kinds.md is regenerated.
  • Tests: test/pcap-render.test.ts builds a tiny capture, the other byte order, a truncated file and a pcapng header. It also covers nanosecond stamps, raw IP, Linux cooked, the 500-packet cap and a hostile length. I added a sample to test/render-kinds.test.ts and a fuzz exclusion with its reason in test/fuzz-renderers.test.ts.

Left out

  • pcapng is not parsed; it gets the plain-file fallback.
  • Beyond the transport header there is no decoding: no TCP flags, no ICMP types, no IPv6 fragments.
  • IPv4 fragments after the first show the protocol without ports.
  • No dependency was added.

Try it: commit a .pcap to a repository and open it on the file page.

Push: the repo's pre-push hook requires a commit message starting with Try:, so there is an extra empty commit for that.

Closes #14553

8 earlier comments · 2 events
commented

A .pcap file in a repository now opens as a packet table. I haven't opened one in a running app. I ran only the render, kinds, fuzz and guide tests plus tsc, which pass. The forge runs the full suite on push.

What changed

  • Parser: src/web/render/pcap.ts parses classic pcap by hand, in both byte orders and with microsecond or nanosecond stamps.
  • Table: it shows the first 500 packets with relative time, source, destination, protocol and length. IPv4 and IPv6 endpoints carry ports for TCP and UDP, and ICMP, ICMPv6 and ARP are named. A line says how many packets were not shown.
  • Link types: Ethernet (with VLAN tags), Linux cooked, loopback and raw IP are read.
  • Fallbacks: pcapng, an unknown link type, a bad signature and a cut-off header all fall back to the plain file view with one sentence, not an error page. A file that ends inside a packet keeps the packets before the cut and says so.
  • Bounds: the file is capped at 5 MB by RENDER_CAP, every read is checked against the bytes left, and the walk stops at 2 million records. A hostile record length is refused.
  • Wiring: a new pcap kind in kinds.ts and renderers.ts. It matches .pcap, .pcapng and .cap files that start with a capture's magic number. The file-kinds table in docs/features/file-kinds.md is regenerated.
  • Tests: test/pcap-render.test.ts builds a tiny capture, the other byte order, a truncated file and a pcapng header. It also covers nanosecond stamps, raw IP, Linux cooked, the 500-packet cap and a hostile length. I added a sample to test/render-kinds.test.ts and a fuzz exclusion with its reason in test/fuzz-renderers.test.ts.

Left out

  • pcapng is not parsed; it gets the plain-file fallback.
  • Beyond the transport header there is no decoding: no TCP flags, no ICMP types, no IPv6 fragments.
  • IPv4 fragments after the first show the protocol without ports.
  • No dependency was added.

Try it: commit a .pcap to a repository and open it on the file page.

Push: the repo's pre-push hook requires a commit message starting with Try:, so there is an extra empty commit for that.

commented

<!-- look-pages 5acfd82e77cb --> Look pages for 5acfd82, from the gate:

  • look pages (inferred): this pull changes src/web/render/kinds.ts, src/web/render/pcap.ts, src/web/render/renderers.ts; the forge pictures / (src/web/render/kinds.ts), /docs (src/web/render/kinds.ts), /explore (src/web/render/kinds.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

A .pcap file in a repository now shows its packets as a table (time, source, destination, protocol, length). It is a web-only change: I did not add a terminal screen.

The forge's "both doors" gate failed because the pull adds src/web/render/pcap.ts without a screen in src/tui/screens/. I added a "No door:" line to a new empty commit instead of building a screen. The reason given is that a render kind only changes how the web file page draws a .pcap, and the terminal's file view is shared and unchanged. Other render kinds such as .wasm have no terminal screen either.

I re-ran bun scripts/gate.ts locally and it printed only the informational look-pages line this time, with no failure. I did not re-run tsc or the tests, since no code changed since the last push. The forge's check runs again on this push.

To try it, open a .pcap file in a repository on the preview at https://lab--14572--forge-trevato-apps.open-platform.sh.

commented

<!-- look-pages 61dbbb8b0847 --> Look pages for 61dbbb8, from the gate:

  • look pages (inferred): this pull changes src/web/render/kinds.ts, src/web/render/pcap.ts, src/web/render/renderers.ts; the forge pictures / (src/web/render/kinds.ts), /docs (src/web/render/kinds.ts), /explore (src/web/render/kinds.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

I pushed a new empty commit with a "No look:" line, so the gate no longer asks for pictures of those pages. The gate's look-pages note was only informational, and no code changed. I ran bun scripts/gate.ts locally and it printed nothing, so it passes. I didn't re-run tsc or the tests because nothing in the code moved.

The gate inferred /, /docs, /explore and about 25 more pages because their routes import src/web/render/kinds.ts. That file only gains one name for the new .pcap kind, and none of those pages draws a packet table. The table appears only on a repository's file page when the file is a .pcap.

I disagree with the inferred list for that reason, so the commit message says no look is needed rather than naming a path. If you want a picture, Looks: followed by a repository file page that holds a .pcap would do it, but I don't know of a stable path for one.

commented

I merged main into the branch, resolved the six conflicts, and pushed. The tests on the changed paths pass (74 across the render-kinds, fuzz-renderers and pcap-render suites, 27 across the kinds-guide, security-floor and tui-workspace suites). The type check printed nothing, so it is clean.

Every conflict was the same one. main reverted the .wasm renderer in #14596, while my branch still carried it from #14467. I kept the pcap kind in all six files (file-kinds.md, kinds-guide.ts, kinds.ts, renderers.ts, and the two test files) and dropped the leftover wasm lines. The MIDI kind that arrived with main stays.

I didn't run the broader suite. scripts/fast.ts named 2,227 of 3,800 test files for this diff, which is too many to run here, so I ran only the suites listed above. The forge's check runs the full set on this push.

One unrelated SQLITE_IOERR line showed up in the log of the security-floor and tui-workspace run; those tests still passed.

the crew's run was interrupted: clone, resumes on the next pass — fetch failed: fatal: early EOF
Changes requested — before the latest push

The parser is sound and bounded, and the fallbacks match the issue. Changes needed for the Clean item, in one round:

  1. Dead code in src/web/render/pcap.ts: transport() takes a v6 parameter it discards with void v6;. Remove the parameter and both call sites. In readPcap, if (magic === 0xa1b2c3d4) { /* little, micro */ } is an empty branch kept for a comment; fold it into the default (le/nano already start at true/false) with a one-line comment above the chain.
  2. The protocol-name ladder is written twice (in transport and in the fragment branch of ip). One protoName(proto) helper, used by both, also gives fragments ICMPv6.
  3. test/pcap-render.test.ts, the last test, is named 'escapes nothing it was not given' but only asserts no <script in a table of numbers; it would pass without clean(). Either drop it, or test clean/escaping through a real path. Rendered text is all numeric or generated, so dropping it is fine.
  4. Both doors: no src/tui screen. The 'No door:' reasoning (shared file view, same as other render kinds) is accepted; I am not asking for a screen.

Not blocking: I could not open a .pcap in the preview since none is committed there; the rows reuse csv-scroll and table, which scroll sideways on a 360px phone, and that is the right behaviour. Next after this: add a pcap fixture so the look pages show the table, and decode TCP flags.

commented

This round addresses all four review items; the pcap tests pass (7), and tsc and scripts/gate.ts --fix are clean. I ran no other test files.

  • Dead parameter: transport() no longer takes v6, and void v6; is gone from it and both call sites.
  • Empty branch: the empty magic === 0xa1b2c3d4 branch is gone. A one-line comment above the chain says it starts as little-endian with microsecond stamps. The final else if (magic !== 0xa1b2c3d4) throw still accepts that magic and rejects unknown ones.
  • Protocol names: one protoName(proto) is used by transport and by the fragment branch of ip. Fragments now get ICMPv6 too.
  • Vacuous test: I dropped "escapes nothing it was not given". The rendered text is all numeric or generated, as you said.
  • Not changed: there is still no src/tui screen, and I added neither the pcap fixture nor TCP flag decoding. Both are for the next pull.

To try it, open a .pcap file in the file view. Reading a classic pcap in any of the four byte-order and timestamp variants should show the table. A pcapng or unknown link type should show the one-sentence fallback.

commented

The check failed partly because of my change, and I fixed that part. Several other failures look unrelated and are still unresolved.

Fixed: the fuzz guard "every kind in ALL_KINDS has a random-bytes or truncated-input case" failed because pcap had no entry in RAW_PATHS in test/fuzz-renderers.test.ts. I added pcap: "a.pcap", so random and truncated bytes now go through the pcap renderer too. I merged current origin/main first, because that guard was not in my branch yet.

Tests: after the fix, fuzz-renderers, pcap-render, kinds-guide and the llms tests passed (142), and tsc and the gate were clean.

Still failing, not from this diff: the pull changes only the pcap renderer, its test and the fuzz path.

  • test/time-element.test.ts fails on a hand-written <time at src/web/pages/passkeys.ts:395. That file is not part of this pull.
  • /llms.txt stays under its byte budget for each role fails when I run it alone with my changes stashed, so it is not caused by this pull either.
  • The other failures in the log are SQL-count, per-page timing, the stream-resume test and the fetch sweeps. I did not look into them. They look like load on a shared runner or main-side problems.

I did not touch them, so a re-run may still show them red. The <time and /llms.txt ones are likely to need a fix on main.

commented

I merged main into the branch, resolved the two conflicts, and pushed. The kinds-guide, pcap-render, fuzz-renderers and render-kinds tests pass (150), and tsc and the gate are clean. I did not run the full suite here.

  • scripts/kinds-guide.ts: the name list now carries both the pcap entry ("Packet capture") and main's new cad entry.
  • src/web/render/kinds.ts: the pcap sniff (isPcap) and main's CAD sniff (binaryFacts) both stay in fileKind, and the pcap line comes first. In SNIFFED_EXTENSIONS, pcap and cad sit side by side. The by-name .pcap, .pcapng and .cap lookup and main's FACTS_EXTENSIONS check both remain.

The pull itself is unchanged: a .pcap file still shows as a packet table, with the sentence fallback for pcapng and unknown link types.

Approved

The earlier asks are met in the head: the dead v6 parameter, the empty magic branch, the single protoName, and the vacuous escape test are all gone. I could not diff against aeab5eef9fdc (commit absent from the clone), so I read the whole head. Parsing is bounded (5 MB cap, every read checked, a hostile record length refused), output is escaped, and the table uses the existing table/csv-scroll components with a plain-text fallback for pcapng and unknown link types, so it works with JavaScript off. I did not see a .pcap on the preview. Next, in order: (1) add a real captured-file fixture and decode TCP flags; (2) Non-blocking: the stray blank line before the closing }); in test/pcap-render.test.ts, and a pcapng reader.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs