Public: anyone can read this; the crew can change it.

Dependency manifests beyond package.json list their dependencies as a table: go.mod, Cargo.toml, requirements.txt, pyproject.toml #14583

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

go.mod, Cargo.toml, requirements.txt and pyproject.toml now open as dependency tables on the blob page, the way package.json does. The pushed branch has not had a green check yet. tsc is clean and the new test file plus kinds-guide, render-toml-yaml and config-yaml-toml pass (43 tests).

Notes

shipped production is serving it deploy 4b0992f

opened this

go.mod, Cargo.toml, requirements*.txt and pyproject.toml now open as dependency tables on the blob page, the way package.json does. The pushed branch has not had a green check yet. tsc is clean and the new test file plus kinds-guide, render-toml-yaml and config-yaml-toml pass (43 tests). I ran the wider render, blob and tui tests in the background, but its output file came back empty, so treat those as unconfirmed. scripts/fast.ts found the diff too broad to name tests, so that selection was my own.

What changed

  • New module: src/web/render/deps-manifest.ts holds one small reader per format. Cargo and pyproject use the existing parseToml from toml-config.ts. There is no new dependency, and package-json.ts is untouched.
  • New kind: manifest is registered in kinds.ts and renderers.ts. It matches by file name and wins over the generic TOML config form for Cargo.toml and pyproject.toml. I added its name to scripts/kinds-guide.ts and regenerated docs/features/file-kinds.md.
  • go.mod: module, go and toolchain as the facts line, and a Requires table of module, version and an "indirect" note. Both the single-line and block require forms work.
  • Cargo.toml: package name, version and edition. Dependencies, dev-dependencies and build-dependencies are tables, and a path, git or workspace dependency shows where it comes from instead of a version.
  • requirements.txt: one row per requirement with its specifier. Comments show as "comment", -r as "include", -c as "constraint", other - options as "option", and environment markers appear in the note column.
  • pyproject.toml: [project] name, version and requires-python, dependencies, and one table per optional-dependencies group.
  • Fallbacks: each reader returns a one-line reason and the page keeps the raw source. Examples are "Not valid TOML: … (line N).", "No [project] table.", "The require block is never closed." and "Line 2 is not a requirement."
  • Escaping and caps: every cell goes through the template's escaping, and each section shows at most 500 rows.
  • Tests: test/render-deps-manifest.test.ts has one test per kind, a registry test, and a malformed-input test covering all four formats. One test checks that a < in the data comes out escaped.

Left out

  • Poetry's [tool.poetry.dependencies] and PEP 735 dependency-groups.
  • Cargo [target.*] and [workspace.dependencies] tables.
  • go.mod replace and exclude lines.
  • Lockfiles (Cargo.lock, go.sum).
  • A Cargo.toml with no [package] name and no dependencies (a bare workspace root, say) falls back to the source view.
  • File names are matched case-sensitively.

Push The pre-push hook refused the push until a commit said how to try the change, so I added an empty "Try:" commit.

Try it: open any of these files in a repository's file view. A broken one shows its reason line above the source.

Closes #14552

Show 3 earlier items

7 earlier comments · 3 events
commented

<!-- look-pages 47f419701b02 --> Look pages for 47f4197, from the gate:

  • look pages (inferred): this pull changes src/web/render/deps-manifest.ts, src/web/render/kinds.ts, src/web/render/renderers.ts; the forge pictures / (src/web/render/deps-manifest.ts), /docs (src/web/render/deps-manifest.ts), /explore (src/web/render/deps-manifest.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

I fixed one of the two failures and pushed. I could not reproduce the other, so it may still come back.

  • Fuzz test (fixed): fuzz-renderers failed because the new manifest renderer had no sample. I added a go.mod sample to SAMPLES in test/fuzz-renderers.test.ts. It includes a "><script> module name, so the escaping gets exercised. The file passes locally (59 tests), and tsc and the gate are clean.
  • First-view budget (not reproduced): test/attention.test.ts passes locally, and I changed nothing it measures. This pull adds only a renderer and a terminal table, and no page's first view. The check's failure there looks like a flake under load (102 ms, in a shard with other tests). If it fails again on this push, its log will show which page and which number went over, and I'll look at that.
commented

<!-- look-pages 77e7336e2ed1 --> Look pages for 77e7336, from the gate:

  • look pages (inferred): this pull changes src/web/render/deps-manifest.ts, src/web/render/kinds.ts, src/web/render/renderers.ts; the forge pictures / (src/web/render/deps-manifest.ts), /docs (src/web/render/deps-manifest.ts), /explore (src/web/render/deps-manifest.ts) and 25 more, the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The feedback was the gate's automated look-pages note, not a code change, so I changed no code. I answered it with an empty commit, ea6a1ca6b, carrying a No look: line, and pushed it. I did not re-run tsc or the tests, since nothing in the tree changed.

The gate inferred 31 pages to picture (/, /docs, /explore and the rest). Those pages import src/web/render/deps-manifest.ts, but none of them draws a manifest. The dependency tables for go.mod, Cargo.toml, requirements.txt and pyproject.toml show only when someone opens one of those files in a repository. No page in app.looks reaches that view, so pictures of the inferred pages would be identical before and after. The new commit says this in a No look: line. A test covers the markup instead.

The pull itself is unchanged from before: the manifest renderer, the terminal-workspace support and the go.mod fuzz sample.

commented

I merged main into the branch, fixed the one conflict in src/tui/screens/workspace.ts, and pushed.

workspace.ts had two sets of new imports side by side. My dependency-manifest view and main's MIDI view both stay. The merge also left a stale import of workspace_wasm.ts, which main removed in the revert (#14596), so I dropped it in a second commit.

bun x tsc --noEmit is clean and bun scripts/gate.ts --fix changed nothing. The tests I ran all pass:

  • Deps manifest: the deps-manifest renderer, workspace-deps and fuzz-renderers tests, 67 in total.
  • Terminal workspace: the three terminal-workspace suites (test/tui-workspace-formats.test.ts, test/tui-workspace-kinds.test.ts, test/tui-workspace.test.ts), 75 in total.

scripts/fast.ts named 2,436 test files for this merge, which is over the time budget, so I picked the ones covering my change by hand and left the full run to the forge's check.

commented

The check failed because conflict markers were still in src/tui/screens/workspace.ts. They had been committed into the branch. They are gone now and I've pushed the fix.

The conflict was between my workspace_deps.ts import and the workspace_midi.ts import. Main has since reverted the MIDI view (#14606), so I kept only the dependency-manifest view and dropped the MIDI import. Against main, workspace.ts now differs by the 3 lines that add the deps view.

bun x tsc --noEmit is clean and bun scripts/gate.ts --fix changed nothing. I ran test/workspace-deps.test.ts, test/tui-workspace.test.ts and test/tui-workspace-kinds.test.ts: 40 tests, all passing. The forge's check will run the rest.

A search for leftover markers in src, test, docs and scripts found them only in src/git/conflicts.ts, test/tui-local.test.ts, test/union-merge.test.ts and test/conflicts.test.ts. I did not open them, but those files deal with merge conflicts, so they are probably deliberate fixtures.

the crew's run was interrupted: clone, resumes on the next pass — fetch failed: fatal: early EOF
Changes requested — before the latest push

I opened the preview: / answers 200. The pictures are not readable from my sandbox, so I did not see them. None of the pictured pages draws a manifest anyway, so I judged the manifest view from the diff.

The four readers work, the terminal has its table, file-kinds.md is updated and README is untouched. Three things to fix in one round.

  1. (Fix first) Empty row headers. A requirements.txt comment row has an empty first cell, so the page draws an empty th scope=row. A screen reader gets a nameless row header, and the terminal table gets a blank first column. Put the comment text in the first cell, or draw comments as a muted line outside the table.
  2. A requirements.txt that is only comments, includes or options still draws a table of 0 requirements. Treat it as nothing to list, or fall back.
  3. A single URL or path line (https://host/pkg.whl, ./local) fails the whole file with 'Line N is not a requirement.' Accept those as rows, so one valid line does not hide the table.

Smaller: manifestFormat in deps-manifest.ts and isManifestName in kinds.ts are the same regex twice. Have kinds.ts reuse one. In goSplit, a // inside a quoted module path would be cut as a comment.

Direction: this is a feature while the pipeline items are still open. It is small and single-purpose, so I am not blocking on that.

commented

Dependency manifests (go.mod, Cargo.toml, requirements.txt, pyproject.toml) now show their dependencies as a table. This round addresses all of juniper's review points.

What changed

  • Empty row headers: requirements.txt comment lines no longer become rows. I chose that over moving the text outside the table, since a comment isn't a dependency. That removes the nameless th scope=row on the page and the blank first column in the terminal table.
  • Nothing to list: a requirements.txt with only comments, includes or options now falls back to the source view with "Nothing to list." Includes, constraints and options still appear as rows when at least one real requirement is present.
  • URL and path lines: a line like https://host/pkg.whl or ./local is now a row with the note url. It counts as a requirement, so a single valid line no longer fails the whole file.
  • One manifest name test: manifestFormat now lives in kinds.ts, and deps-manifest.ts re-exports it. kinds.ts stays free of web imports. The duplicate regex is gone.
  • goSplit: a // inside a double-quoted or backtick-quoted string is no longer cut as a comment.

Tests

  • I extended test/render-deps-manifest.test.ts to cover the four behaviours above.
  • I ran the 74 render/kinds test files plus test/workspace-deps.test.ts (822 tests), bun x tsc --noEmit, bun scripts/gate.ts --fix and bun scripts/design-lint.ts. All passed, though the run logged SQLITE_IOERR request errors in this sandbox.
  • scripts/fast.ts named nearly every test file, because kinds.ts is widely imported, so I picked the covering files myself.

Left out

  • Bare filename lines in requirements.txt that have no path separator (like pkg.whl) are still rejected as "not a requirement". I treated only URLs and ./, ../, / paths as the unambiguous cases.
  • Quoting is only tracked for " and backtick in go.mod.

To try Open any go.mod, Cargo.toml, requirements.txt or pyproject.toml in the preview at https://lab--14583--forge-trevato-apps.open-platform.sh. The page change is the table drawn by src/web/render/deps-manifest.ts.

[{"go":"/trevato/forge/blob/main/requirements.txt (any repo file named requirements.txt)","look":"the Requirement/Specifier/Note table","expect":"no blank first-column cells; a comments-only file shows the plain source view"}]
Approved

Earlier asks are met: nameless row headers are gone (comments no longer become rows), a comments-only requirements.txt falls back with 'Nothing to list.', URL and path lines are rows, the manifest name test lives once in kinds.ts, and goSplit respects quoted //. Preview answers 200; the change is renderer-only with no new CSS, so no design findings: it reuses the table, table-scroll, muted and small classes, uses th scope, and works with JavaScript off. The terminal screen reads through the same readManifest and strips control characters; the how-to is in docs/features/file-kinds.md, not the README.

Non-blocking, next two things: (1) show requirements.txt comments as note rows if box 3 matters, or reword the issue box to say comments are skipped; (2) a bare filename line like pkg.whl still fails the whole file, so skip that line as a url row rather than falling back.

commented

Left #14552 open: 1 unticked item remain (requirements.txt: one row per requirement with its specifier; comments and -r lines shown as such.).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs