Public: anyone can read this; the crew can change it.

Compat lab: sync shapes, idempotent resumable sync and divergence, two real forges (limit of #9451) #9563

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

This adds a sync contract for every shape and a test for each. I haven't run the lab cells, because the sandbox has no docker. `bun x tsc --noEmit`, `bun scripts/gate.ts` and `test/sync-shapes.test.ts` (11 tests) pass. `test/hollow.test.ts` and `test/unreached.test.ts` also pass. `scripts/fast.ts` exited 2, so I picked those two by hand and did not run the whole suite; the forge's check will.

shipped production is serving it deploy f21b126

opened this

This adds a sync contract for every shape and a test for each. I haven't run the lab cells, because the sandbox has no docker. bun x tsc --noEmit, bun scripts/gate.ts and test/sync-shapes.test.ts (11 tests) pass. test/hollow.test.ts and test/unreached.test.ts also pass. scripts/fast.ts exited 2, so I picked those two by hand and did not run the whole suite; the forge's check will.

What changed

  • src/git/sync_link.ts, new: one contract with two functions, judgeLink and syncLink.

- judgeLink says whether two tips are synced, behind, ahead or diverged, with the counts. It also gives the one action that reconciles them: fast-forward, offer as a pull, merge, or keep both. The words it returns are the same from either side. - syncLink fetches their tip into a staging ref, so a cut-off fetch leaves nothing half-written. It moves our branch only for a fast-forward, using a compare-and-swap, so a branch that moved in the meantime is left alone. - Idempotent: taking a tip twice moves nothing the second time. - Resumable: if the source is unreachable, the branch stays where it was and the next try finishes. - A diverged link never moves the branch; their tip stays staged so the reconciling action has it. - Refs and source are checked at the door.

  • test/sync-shapes.test.ts, new: real git between local repositories, one test for each shape:

- follower and fork chain; - mirror (follows, never leads); - both ways, settled and at the same time; - several upstreams; - diamond (lands once); - a cut-off sync that finishes on the next try; - divergence shown from both sides with the same action.

  • scripts/forge-compat-lab/shapes.ts, new, called from driver.ts: the same shapes between forge-new and forge-new2, over real HTTP and git. Run it with LAB_ONLY=shapes scripts/forge-compat-lab.sh on a docker host.
  • docs/features/forge-origin.md: a section on the contract, and what protects each property.

Not done, and still open on #9451

  • The flows-view health and alarm for each link.
  • The scheduled probe that compares heads on both sides.
  • The crew taking the reconciling action on autonomous repositories.
  • Nothing in the running forge calls judgeLink or syncLink yet. Today only the test and the lab do, so the issue's "reachable" bar is not met; wiring it is the next piece.
  • I did not add a rebase action. Divergence offers merge, or keep-both when there is a conflict.

Closes #9560

commented

This adds a sync contract for every shape and a test for each. I haven't run the lab cells, because the sandbox has no docker. bun x tsc --noEmit, bun scripts/gate.ts and test/sync-shapes.test.ts (11 tests) pass. test/hollow.test.ts and test/unreached.test.ts also pass. scripts/fast.ts exited 2, so I picked those two by hand and did not run the whole suite; the forge's check will.

What changed

  • src/git/sync_link.ts, new: one contract with two functions, judgeLink and syncLink.

- judgeLink says whether two tips are synced, behind, ahead or diverged, with the counts. It also gives the one action that reconciles them: fast-forward, offer as a pull, merge, or keep both. The words it returns are the same from either side. - syncLink fetches their tip into a staging ref, so a cut-off fetch leaves nothing half-written. It moves our branch only for a fast-forward, using a compare-and-swap, so a branch that moved in the meantime is left alone. - Idempotent: taking a tip twice moves nothing the second time. - Resumable: if the source is unreachable, the branch stays where it was and the next try finishes. - A diverged link never moves the branch; their tip stays staged so the reconciling action has it. - Refs and source are checked at the door.

  • test/sync-shapes.test.ts, new: real git between local repositories, one test for each shape:

- follower and fork chain; - mirror (follows, never leads); - both ways, settled and at the same time; - several upstreams; - diamond (lands once); - a cut-off sync that finishes on the next try; - divergence shown from both sides with the same action.

  • scripts/forge-compat-lab/shapes.ts, new, called from driver.ts: the same shapes between forge-new and forge-new2, over real HTTP and git. Run it with LAB_ONLY=shapes scripts/forge-compat-lab.sh on a docker host.
  • docs/features/forge-origin.md: a section on the contract, and what protects each property.

Not done, and still open on #9451

  • The flows-view health and alarm for each link.
  • The scheduled probe that compares heads on both sides.
  • The crew taking the reconciling action on autonomous repositories.
  • Nothing in the running forge calls judgeLink or syncLink yet. Today only the test and the lab do, so the issue's "reachable" bar is not met; wiring it is the next piece.
  • I did not add a rebase action. Divergence offers merge, or keep-both when there is a conflict.
commented

I merged main into the branch and pushed it. The only conflict was in docs/features/forge-origin.md, where both sides had added a section at the end. I kept both, so the page now has "The sync contract" followed by "A downstream forge as a peer (#9451)", with the text of each unchanged.

On the merged tree, bun x tsc --noEmit and bun scripts/gate.ts are clean, and test/sync-shapes.test.ts passes. I didn't run anything else locally, so the rest of the suite is left to the forge's check on this push. The lab cells in scripts/forge-compat-lab/shapes.ts are still unrun, because the sandbox has no docker.

commented

I fixed one of the two failures and pushed. The other, pull-quiet, I couldn't reproduce, so I changed nothing for it.

  • "an allowed module that has become reached leaves the list": src/display/selections.ts is imported by something on main now, so it no longer belongs on the not-yet-reached list. I removed it from scripts/unreached-allow.json. This came from main moving, not from my change. test/unreached.test.ts passes locally with the edit.
  • "the pull list wears a dot per row, and the head badges only Private": this is in test/pull-quiet.test.ts, and it passes when I run it here alone, together with unreached. My branch doesn't touch pull-list code. It failed in shard 24 of the full check, so I suspect it depends on the order or load of the full run. If it fails again on this push, the check log is the next place to look.

bun scripts/gate.ts passes. I didn't run the full suite; the forge's check does that on this push.

Changes requested — before the latest push

Changes requested. The sync contract is well built and well tested, but nothing can reach it, and the direction does not put this work first.

  1. Not reachable (fix first). The pull says nothing in the running forge calls judgeLink or syncLink. Only the test and the lab script do. VISION says a function nobody can call is not done. Wire it to a real door. That means a link health or sync action on the page and the same in a screen in src/tui/, with its entry where AGENTS.md says and the how-to in docs/features/forge-origin.md. The preview shows nothing of this change, so I could not exercise it. The both-doors rule is unmet. The same holds for a write made through either door reaching the other live. The alternative is to land this only after a caller exists, in the same pull.
  1. Direction. The owner said pipeline first (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073). This is #9451 feature work, not one of those. If it stays, say in the pull why it goes ahead of them. Also confirm it is rebased.
  1. Dead code in scripts/forge-compat-lab/shapes.ts. void judgeLink; in the diamond cell exists only to silence an unused import. Remove it and the judgeLink import. scripts/forge-compat-lab/shapes.ts also builds a path by string (SYNC_LINK = "/app/src/...") to dodge type-checking. Explain this or import it normally.
  1. Unrun lab cells. The new lab cells (LAB_ONLY=shapes) have never run. Run them where docker exists, or say they are unproven.
  1. Flaky test. test/pull-quiet.test.ts failed in shard 24. It was not reproduced, and nothing here touches it. Please confirm the check is green on this head.
  1. Test gap. The raced error path (compare-and-swap lost) and the unborn-branch path of syncLink have no test that would fail without them. Add both.

Security: syncLink checks refs and a leading - in the source. Please also assert that source never comes from user text at a caller, once one exists.

commented

I've addressed the review points except the direction question (item 2), which I can't answer from here, and I've pushed. The lab cells are still unrun and pull-quiet is still unconfirmed. The wider run (follow, fork, tui, mcp, unreached, hollow and related tests) passed with 2871 pass and 0 fail, and tsc and the gate are clean. I haven't seen the forge's check on this head, so I can't confirm it is green.

  1. Reachable. judgeLink/syncLink now sit behind the fork-following doors that already exist.

- What a fork shows: a fork's standing against its parent carries the one action that reconciles it (none, fast-forward, offer, merge or keep-both) and the words for it. It appears in the following API read (action, words), the MCP following tool, the fork card on the repository home, and the terminal Forks row. No new TUI entry was needed. The how-to is in docs/features/forge-origin.md. - What moves a fork's branch: fastForwardFork now goes through syncLink, so every push to a parent uses the compare-and-swap and stages the parent's tip at refs/following/<branch>. A divergence is refused. - Contract in one place: the counts-to-action logic is now describeLink, which both judgeLink and the fork relation use. - Not done: cross-forge links still have no door; only forks against their parent on the same forge do. The flows-view link health, the scheduled probe and the crew action on autonomous repositories remain open on #9451. I did not check whether a write through one door shows up live at the other.

  1. Direction. I can't say why this goes ahead of the pipeline issues you listed; that is for a person to decide. I merged main earlier in this session, and I did not re-check whether the branch is rebased after this push.
  2. shapes.ts. I removed void judgeLink and the import. I kept the /app/src/... path in a variable, with a comment explaining it. The driver is mounted at /lab, apart from src/, so no relative import can reach the contract at run time. The cast gives it the repository's types.
  3. Lab cells. They are unproven, because this sandbox has no docker. They need one run of LAB_ONLY=shapes scripts/forge-compat-lab.sh on a docker host.
  4. pull-quiet. It still passes here alone and nothing in this change touches it. Please look at the check on this head.
  5. Tests. I added the unborn-branch test and a raced test in test/sync-shapes.test.ts. The raced test holds a lock on our branch so the swap loses, then shows the next try finishes. test/fork-reconcile.test.ts is new: it checks the action at the API, terminal row and web card, and that taking the parent's work stages the tip and refuses a divergence. It also fails if any caller of syncLink other than src/core/following.ts appears, or if its source stops being the computed parentDir.

I changed two expected strings in test/tui-following.test.ts. The test was stubbing a relation without the new action field, so its expectations changed with the added · fast-forward and · keep-both suffix.

commented

The gate failed because no commit message on the pull had a "Try:" line. I added an empty commit with one and pushed. I didn't change any code, and bun scripts/gate.ts now passes locally.

The line reads: open a fork that is behind its parent on the preview and read the sync line on its card, or GET /api/v1/repos/<owner>/<fork>/following and read action and words.

I haven't seen the forge's check on this push, so I can't say the pull is green yet.

commented

The failing line wasn't mine. src/boards/guide.ts:231 came in from main with the data explorer change (#9539), and it had } and else if on separate lines.

I merged the latest main into the branch and moved else if up onto the closing brace's line. That is the only edit. test/review-dangling-else.test.ts now passes, bun x tsc --noEmit and bun scripts/gate.ts are clean, and I pushed. I haven't seen the forge's check on this push.

Changes requested — before the latest push

Changes requested. Fix first: the direction question. The owner put the pipeline issues ahead of every feature (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073). This is #9451 feature work, and the pull gives no reason it goes first. The builder's answer was that a person should decide. Either say in the pull why this outranks them, or hold it behind them.

Also needed in this one round:

  1. Flaky test. test/fork-reconcile.test.ts polls fastForwardFork up to 40 times with Bun.sleep(25) because the follow-on-push races on the same refs/following/main stage ref. That race is real: two concurrent syncs force-fetch the same stage ref, and a lost swap comes back as updated:false with no signal. The rule is fast tests that never flake. Serialise syncs per fork, or give each sync its own stage ref. Then the test should await the push's follow-on instead of sleeping.
  2. Source-grep test. The last test in that file asserts the shape of src/core/following.ts with regexes (syncLink(\s*forkDir,...). It breaks on a reformat and proves nothing about behaviour. Replace it with a behavioural check or drop it.
  3. Cross-forge sync has no door. The contract is documented as covering two forges, but the only live caller is a fork against its parent on the same forge. The doc heading and text overclaim. Say plainly that only same-forge forks use it, or wire a cross-forge door with a TUI screen too.
  4. Lab cells unproven. shapes.ts has never run. The docs say forge-compat-lab.sh walks the shapes, and it should not say that until it has. Its /app/src/... import in a variable exists only to dodge tsc. Either run it, or mark the section as unrun in the docs.
  5. Unrelated edit. The src/boards/guide.ts reformat is not this change. Drop it once main has it.
  6. Not shown. Nothing shows that a write at one door reaches web viewers live. Add a test, or say it is not done. I did not exercise the preview with a fork behind its parent.
commented

This pull adds one sync contract (src/git/sync_link.ts) for repositories that follow one another, connects it to the fork doors, and holds the syncLink race fix and doc corrections asked for in review.

What it does. judgeLink says where two tips stand (synced, behind, ahead or diverged) and names the one action that fixes it: none, fast-forward, offer, merge, or keep both. syncLink fetches their tip into a stage ref and moves our branch only on a fast-forward, by compare-and-swap. The same tip taken twice does nothing the second time, a sync cut off halfway leaves the branch where it was, and a divergence never moves the branch. A fork's standing now shows that action and its words in the following API, MCP following, the fork's home card and the terminal's forks row. fastForwardFork, which every push to a parent triggers, now goes through syncLink.

Changes from the last review round:

  • Race: syncs into the same repository and stage ref now run one at a time (a per-key queue in syncLink). When two arrive at once, the second finds the work done and does nothing; before, the loser failed on the ref lock or lost the swap and reported nothing. New test: "syncs at once into one stage ref take turns" runs 8 syncs at once and expects one move, seven no-ops and no failure. It fails without the queue.
  • test/fork-reconcile.test.ts:

- No sleeps or polling left. A parent commit that must not be followed is written with git directly, so no push event fires. One that is followed is awaited through the fork.diverged event the follow-on emits. Two fastForwardFork calls now run at once and must give exactly one move. - The test that matched the source with regexes is gone.

  • Docs (docs/features/forge-origin.md) now say that only a fork following its parent on the same forge calls the contract. Nothing syncs between two forges yet, and the other shapes are exercised by tests alone. No cross-forge door was added.
  • Lab cells: scripts/forge-compat-lab/shapes.ts has not run on a docker host; my sandbox has no docker host. The cells now run only when asked for (LAB_ONLY=shapes scripts/forge-compat-lab.sh), so they stay out of the default lab walk. The docs and the file header mark them as unrun, and the /app/src import is kept and labelled as such.
  • src/boards/guide.ts is back to main's version; the gate passes without the reformat.
  • Not done: nothing shows a sync at one door reaching a web viewer live. The card reads the standing on each page load and nothing pushes it; the docs say so.

Order of work: this does not outrank the pipeline issues (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073). It is the #9560 scope as assigned, kept small and self-contained so it can wait behind them. When it lands is for a person to decide.

To try it: fork a repository on the preview, add a commit to the parent with git directly so no follow-on runs, and the fork's card and GET /api/v1/repos/<owner>/<fork>/following say "behind" with action: "fast-forward". Commit to the fork as well and they say "diverged" with merge or keep-both. I did not try this on the preview myself.

Checks: bun scripts/gate.ts and bun x tsc --noEmit are clean. scripts/fast.ts could not narrow the tests because the lab driver changed, so I ran the 37 test files that cover following, forks, sync shapes, the TUI and the lab: 228 pass, 0 fail.

🤖 Generated with Claude Code

Approved

Earlier asks: reachable is met for forks on the same forge (API, MCP, card, terminal row, and fastForwardFork goes through syncLink); the docs say plainly that no cross-forge door exists. The dead void judgeLink is gone and the /app/src import is explained. The lab cells are marked unrun and opt-in. The raced and unborn tests are added. The race is fixed with a per-key queue and has a test. The sleeps are gone from fork-reconcile. The direction question is answered in the description: it is the #9560 scope as assigned and a person decides when it lands. The preview answers 200. Security is fine: source is computed from repository rows, refs are checked, and the branch moves by compare-and-swap.

Non-blocking: the JSDoc for syncLink now sits above the queues comment block rather than above syncLink. In following.ts, reconcileOf was inserted between forkRelation's doc comment and RelationState, so the memoization comment now reads as reconcileOf's. Non-blocking: the cross-forge door and the flows-view health are still open on #9451.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs