Public: anyone can read this; the crew can change it.
Forgify a business: origins mirror its repos in, extract pulls them apart, the crew writes the sync and pull adapters, and the result ships as its own forge #10032
Check · Review · Land · Live done — Landed and live in production.
Merged `main` into the branch and resolved the conflicts in `src/apps/manifest.ts`, `src/apps/process.ts`, `src/apps/runtime.ts`, `src/core/schema.ts` and `src/web/studio/app.ts`, keeping what both sides meant. The adapter feature now sits on top of main's newer layout.
shipped production is serving it deploy 84c60a0
Merged main into the branch and resolved the conflicts in src/apps/manifest.ts, src/apps/process.ts, src/apps/runtime.ts, src/core/schema.ts and src/web/studio/app.ts, keeping what both sides meant. The adapter feature now sits on top of main's newer layout.
- Schema: main now keeps one file per migration, so the numbered
adapter_pullentry this branch had added toschema.tsis nowsrc/core/migrations/20261109120000-adapter-pull.ts. The table itself is unchanged. Its pin was written byscripts/pin-migrations.ts.schema.tsis main's version. - Manifest:
adaptersits beside main'sservice,looksAsOwner,pages,launch,egress,limitsandtoolbar. It is now optional (adapter?: AdapterManifest | null), like main's newer fields, so the existing test fixtures that don't set it stay valid. All readers already usemanifest.adapter?.. - Process:
adapterHosts(which setsFORGE_ADAPTER_HOSTS) is passed into main's restructured, cell-aware spawn.featureis passed alongside it. - Runtime: the pull ticker is kept next to main's
placement,storageEveryMsandgateoptions.adapterOfis kept next to main'scellForandnoteCell. - Studio: the "Syncs …" adapter line renders under main's asking and policy lines, inside its
splitlayout. The merge had dropped thetimeimport from../ui.ts, which the adapter line needs, so it is restored.
bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts pass. 156 tests across 12 files pass: adapter, the adapter template, apps, schema migrations, pin edits, app env, studio and process tests. scripts/fast.ts reported the diff too broad to name (exit 2), so I picked those files myself and left the full suite to the forge's check.
Try: open an adapter app's page on the preview; its "Syncs …" line names the source, the last and next pull, and the cursor.
🤖 Generated with Claude Code
This pull gives the forge a shape for sync adapters (#390). An app adds an adapter block to its forge.json, and the forge then pulls an external system into that app's own state on a schedule.
- Parsing:
src/apps/adapter.tsreads the block when the manifest is loaded and checks it. The block names the source and the hosts it may reach, how often to pull, and typed config with env names and defaults. - Pulling:
src/apps/pull.tsruns the pulls. A timer insrc/apps/runtime.ts(every 15 s) starts one only in production, only for a running app, and only one at a time per app. - Recording: each pull is saved in
src/core/adapter_pulls.ts(migration20261109120000-adapter-pull), and the app page shows the last one. - Reach: the app process gets its declared hosts as
FORGE_ADAPTER_HOSTS, andhostAllowedrefuses any host not on the list. - Template:
examples/adapter-template/is a working adapter with its own tests, for the crew to start from.docs/ADAPTERS.mddescribes all of this and is now linked fromdocs/REFERENCE.md.
The last check failed on 5 tests:
- Real failure:
docs/ADAPTERS.mdwasn't linked from REFERENCE.md's "Where the rest is" section. That's fixed. - Not caused by this pull, as far as I can tell: the 3 tests in
test/traffic-truth.test.ts(shard 15) and the 1 intest/tui-system.test.ts(shard 47). None of them touch the adapter code, and both files pass here every time I run them. That check run reported only about 17% disk free, so I expect them to pass on a re-run; if they fail again, they need a closer look.
I also moved the sweep docstring in runtime.ts back onto sweep; the earlier change had left it sitting above pullOwed.
Locally, bun scripts/gate.ts, bun x tsc --noEmit, and the tests in test/adapter.test.ts, test/apps.test.ts, test/readme-front-door.test.ts, test/traffic-truth.test.ts and test/tui-system.test.ts all pass. I didn't run bun scripts/design-lint.ts, although this pull changes src/web/studio/app.ts.
Not done here: the first real adapters (the owner chooses which), and Atlas turning a description into apps and adapters.
To try it: copy examples/adapter-template/ into a repository and deploy it as an app. The app page shows the last pull once the first interval has passed.
🤖 Generated with Claude Code
Changes requested. Fix first: the forge reads an adapter's whole reply into memory before capping it, and a sleeping-sweep guard is wrong for manifests without the field. Full list:
- Direction: the owner put the development pipeline first (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073) before any feature. This is a new feature (adapters, #390) and none of those items is touched. Say why it jumps the queue, or park it.
- Both doors: the adapter line exists only on the web page. There is no screen in src/tui/ (entry where AGENTS.md says), and no docs/features/<name>.md how-to; docs/ADAPTERS.md is a contract, not the how-to. Add both.
- Tests missing: nothing covers pullOwed (production only, one at a time, due-ness, stop waiting for pulls), the sweep skip for adapters, FORGE_ADAPTER_HOSTS reaching the process env, or adapterLine rendering and escaping (source name, hosts and cursor come from the repo / the adapter). Each added behaviour needs a test that fails without it.
- Security: see line comments (unbounded reply read, cursor shown to every viewer, host list advertised as enforced but not enforced by the forge, unbounded table growth).
- Design: the adapter line is one long run-on sentence with a code fragment, reusing policy-line with a data-tone the stylesheet may not define. Show it as a plain
dlrow (Source, Last pull, Next pull) per principles 1, 7 and 8, with no new class. - Clean: hostAllowed in src/apps/adapter.ts is used only by its own test in the forge (dead code); the real guard is the template's copy. Remove it, or use it in the forge.
This pull makes an app with an adapter block in its forge.json a sync adapter (#390). The forge calls the app's POST /pull on its schedule, records each pull, and shows the result to the app's owner on the web and in the terminal. This push addresses the review.
Fixes
- Reply size:
src/apps/pull.tsnow reads the reply a chunk at a time and stops past 64 KiB, recording that as a failed pull. Before, it read the whole reply into memory and only then cut it short. - Control characters: adapter error text has them replaced by spaces when the reply is parsed, because it now reaches a terminal.
- Pull history:
recordPullkeeps each app's newest 200 pulls and deletes the rest in the same transaction. - Sleep check: the idle sweep tested
!== null, so it kept every app without anadapterfield awake. It now uses!= null, and only an adapter's production is kept awake; previews never pull, so they sleep. - Allowed hosts: an adapter's declared hosts are added to the app's
egresslist inparseManifest. The egress proxy already enforces that list for an app running in a cell, so the forge now enforces it too. I removed the unusedhostAllowedand rewordeddocs/ADAPTERS.mdto match. The template'sguardedFetchstays as an extra check inside the app.
Where it shows
- App page: the run-on line is gone. Owners and admins (
canSeeOwnerFacts) see a plaindlwith Source, Last pull and Next pull. Guests and readers see none of it, and the cursor is shown to no one. There is no new class and nodata-tone. The next-pull time comes from a small shared module,src/apps/adapter_status.ts, instead of a string comparison. - Terminal: the Deploys screen shows the same three lines to the same people (
adapterLines). - How-to:
docs/features/adapters.mdis new and linked fromdocs/REFERENCE.md.
Tests
test/adapter-runtime.test.ts(new) covers:
- pulls run only in production, one at a time, and only when due;
- stop waits for a pull already running;
- the sweep keeps an adapter's production awake and lets other apps and adapter previews sleep;
- the app page escapes adapter text and hides it from readers and guests;
- the terminal lines appear for the owner only.
test/adapter.test.tsadds an endless reply that has to fail early, control-character stripping, pruning, the merge intoegress, andFORGE_ADAPTER_HOSTSreaching the process.
I fixed one of the new tests itself: it expected egress to be undefined for an app without an adapter, but the manifest already gives such apps an empty list.
Checks run: bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. scripts/fast.ts named 1,859 test files, too many to run here, so I ran the adapter, app-env, egress, sleep/busy, app-page and tui-deploys* suites by hand; all pass.
Pipeline-first order (review point 1): I didn't park this. It is assigned issue #390, and most of this push secures behaviour the pull already adds. Whether adapters should wait behind the pipeline items is the owner's call; if they should, this pull can wait as it stands.
Not done here: the first real adapters (the owner chooses which), and Atlas turning a description into apps and adapters.
To try it: copy examples/adapter-template/ into a repository and make it an app. Once production runs, its app page and Deploys screen show Source, Last pull and Next pull to the owner.
🤖 Generated with Claude Code
This pull makes an app with an adapter block in forge.json a sync adapter (#390). The forge calls the app's POST /pull on production when a pull is due, records each pull, and shows the result to the app's owner on the web and in the terminal.
Merge with main: main had added its own docs/ADAPTERS.md (#10056), a contract written before any code existed. The merge keeps that contract and changes the code to follow it:
schedule.everyis whole seconds from 60 to 86400, not"15m".- Every source host must also be listed in
egress, or the block is invalid. The forge no longer adds the source hosts toegressitself. - The answer must have exactly four fields.
errorsis a count from 0 to 1,000,000, and the cursor is at most 200 printable characters. The forge rejects an answer that breaks a rule and names the field, e.g.pulled is not an integer from 0 to 1000000. - A good answer is recorded with its counts, even when
errorsis above zero. A bad answer keeps the last good cursor (lastGoodCursor).
Docs: the contract's Status section now says what runs and points to docs/features/adapters.md and examples/adapter-template/. Pulls are kept in their own table, not as runs, and the Status section says why. A new last section lists what else the forge holds an adapter to:
- only production pulls, one at a time;
- the forge sends the app's identity key as a bearer token;
- the process gets
FORGE_ADAPTER_HOSTS; - answers over 64 KiB are bad and a pull times out after five minutes;
- each app keeps its newest 200 pulls;
- only the owner and admins see Source, Last pull and Next pull, and no one sees the cursor.
The optional config block is documented as part of the adapter block. The REFERENCE.md conflict keeps main's line and adds the how-to link.
Template: it now declares "every": 900 and lists its host in egress. It counts errors and logs the reason instead of returning error strings.
Checks run: the gate, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. These pass: test/adapter.test.ts, test/adapter-runtime.test.ts, the template's own tests, test/readme-front-door.test.ts, test/apps.test.ts, test/app-env.test.ts, test/here-manifest.test.ts, test/confirm-app-stop.test.ts, and the deploys and egress terminal tests. Two expectations in my own new tests were wrong and I corrected them: egress lowercases its entries, and ManifestError needs a kind check before reading message.
Before / after:
- The direction point from the earlier review still stands as I answered it: the pull serves issue #390, and parking it is the owner's call.
- Not done here: the first real adapters and Atlas.
To try it: copy examples/adapter-template/ into a repository and make it an app. Its owner then sees Source, Last pull and Next pull on the app page and the Deploys screen.
🤖 Generated with Claude Code
This pull makes an app with an adapter block in forge.json a sync adapter (#390). The forge calls the app's POST /pull on production when a pull is due, records each pull, and shows the result to the app's owner on the web and in the terminal. This push merges main, which had added its own adapter parser.
One parser: main's src/apps/adapter.ts (#10055) reads { schedule: { every } }, gives undefined when the block is absent, and refuses unknown keys. That is now the base. This branch's source and config are optional keys beside schedule, and unknown keys inside them are refused the same way. A block with only a schedule parses exactly as it does on main, and the interval error sentences are main's word for word.
The rest of the branch: src/apps/manifest.ts is main's version plus the rule that every source host must be in egress. These now read main's shape, through a new everyMs(spec) helper where they need the interval:
- the pull scheduler in
runtime.ts; - the shared status in
adapter_status.ts, which says "not declared" for an adapter with no source; FORGE_ADAPTER_HOSTSinprocess.ts.
One test of main's changed: in test/apps-adapter.test.ts, the sentence for an unknown key now reads adapter takes only schedule, source and config, not "url". The test was right for main; the change follows from allowing the two new keys.
Docs: docs/APPS.md and docs/ADAPTERS.md say source and config are optional and that any other key is refused.
Tests: test/adapter.test.ts adds checks that a schedule-only block parses, that unknown keys in source and config are refused, and what the owner reads for an adapter without a source.
Checks run: the gate, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. These pass: test/adapter.test.ts, test/adapter-runtime.test.ts, test/apps-adapter.test.ts, test/apps.test.ts, test/readme-front-door.test.ts, test/tui-deploys.test.ts, test/tui-deploys-egress.test.ts, test/app-env.test.ts, test/here-manifest.test.ts, test/confirm-app-stop.test.ts, and the template's own tests.
Not done here: the first real adapters (the owner chooses which), and Atlas turning a description into apps and adapters.
To try it: copy examples/adapter-template/ into a repository and make it an app. Its owner then sees Source, Last pull and Next pull on the app page and the Deploys screen.
🤖 Generated with Claude Code
I made no new commit and pushed nothing. scout/390 already contains main through 490e78b8, and the forge is running its check on the current head (ee15f987). Since my last run, the work has moved into smaller pulls: #10055 and #10056 are merged, #10058 (pull runs) and #10059 (example adapter) are open and green, and #10043 (the scheduler) is open with no one assigned. Adding to this branch now would only widen those overlaps.
Here locally, the gate, bun x tsc --noEmit and bun scripts/design-lint.ts all pass. The adapter tests pass 31 of 31, but bun test reported only 3 files when I listed 4, and I didn't check which one it skipped.
---
This builds the adapter app shape from #437 on top of main's adapter block (src/apps/adapter.ts, #10055) and the contract in docs/ADAPTERS.md (#10056):
- Manifest:
forge.jsoncan nameadapter.source(a name and its hosts) andadapter.config, alongsideschedule.every. Every source host must also be listed inegress. - Pull runs:
src/apps/pull.tschecks eachPOST /pullanswer as it arrives and stops reading the reply past its size limit.src/core/adapter_pulls.tsstores each pull in theadapter_pulltable, added by migration20261109120000-adapter-pullwith its pin committed, and removes old rows. - Scheduling:
src/apps/runtime.tspulls each running adapter once it is due. It never wakes a sleeping app, and two pulls never run at once for the same app. - Status:
src/apps/adapter_status.tsshows Source, Last and Next on the app page and on the terminal's deploys screen, to the owner only. - Template:
examples/adapter-template/is a zero-dependency adapter to copy, with its own tests.
What to look at: this branch overlaps three smaller pieces of the same work:
- #10058 stores pull runs in a separate
pull_runtable, but its migration has the same timestamp as this branch's. - #10059 also adds
examples/adapter-template/, with the source hosts kept in its ownsrc/source.ts. - #10043, still open, is the scheduler this branch already has in
runtime.ts.
Whichever lands first, the others must be rebased onto it. If the split pulls land, this pull reduces to the parts nothing else covers: source and config on the manifest, egress joining, and the owner-only status. I left those smaller pulls alone and didn't touch #438 (which waits for the owner to name the two systems) or #439.
What I left out: the app reads secrets from its own environment; the manifest does not reference them. Adapters have no manual "pull now" button.
How to try it: deploy examples/adapter-template as an app. Its page shows Source, Last and Next to the owner, and each scheduled pull appears there with its counts and cursor. Tests: bun test test/adapter.test.ts test/adapter-runtime.test.ts test/apps-adapter.test.ts examples/adapter-template/test/adapter.test.ts.
The repository and issues held no instructions to reach other hosts or repositories.
🤖 Generated with Claude Code
I made no new commit and pushed nothing: nothing has changed since my last run. scout/390 is still current with main, and the check on its head (ee15f987) is still running. #10058 and #10059 are still open and green, and #10043 (the scheduler) is still open with no one assigned. Adding to this branch would only widen the overlap with those smaller pulls. I didn't rerun the checks this time; last run, the gate, bun x tsc --noEmit and bun scripts/design-lint.ts passed here, and the adapter tests passed 31 of 31.
---
This builds the adapter app shape from #437 on top of main's adapter block (src/apps/adapter.ts, #10055) and the contract in docs/ADAPTERS.md (#10056):
- Manifest:
forge.jsoncan nameadapter.source(a name and its hosts) andadapter.config, alongsideschedule.every. Every source host must also be listed inegress. - Pull runs:
src/apps/pull.tschecks eachPOST /pullanswer as it arrives and stops reading the reply past its size limit.src/core/adapter_pulls.tsstores each pull in theadapter_pulltable, added by migration20261109120000-adapter-pullwith its pin committed, and removes old rows. - Scheduling:
src/apps/runtime.tspulls each running adapter once it is due. It never wakes a sleeping app, and two pulls never run at once for the same app. - Status:
src/apps/adapter_status.tsshows Source, Last and Next on the app page and on the terminal's deploys screen, to the owner only. - Template:
examples/adapter-template/is a zero-dependency adapter to copy, with its own tests.
What to look at: this branch overlaps three smaller pieces of the same work:
- #10058 stores pull runs in a separate
pull_runtable, but its migration has the same timestamp as this branch's. - #10059 also adds
examples/adapter-template/, with the source hosts kept in its ownsrc/source.ts. - #10043, still open, is the scheduler this branch already has in
runtime.ts.
Whichever lands first, the others must be rebased onto it. If the split pulls land, this pull reduces to the parts nothing else covers: source and config on the manifest, egress joining, and the owner-only status. I left those smaller pulls alone and didn't touch #438 (which waits for the owner to name the two systems) or #439.
What I left out: the app reads secrets from its own environment; the manifest does not reference them. Adapters have no manual "pull now" button.
How to try it: deploy examples/adapter-template as an app. Its page shows Source, Last and Next to the owner, and each scheduled pull appears there with its counts and cursor. Tests: bun test test/adapter.test.ts test/adapter-runtime.test.ts test/apps-adapter.test.ts examples/adapter-template/test/adapter.test.ts.
The repository and issues held no instructions to reach other hosts or repositories.
🤖 Generated with Claude Code
I made no new commit and pushed nothing: nothing has changed since my last run. scout/390 is still current with main, and the check on its head (ee15f987) is still running. #10058 and #10059 are still open and green, and #10043 (the scheduler) is still open with no one assigned. Adding to this branch would only widen the overlap with those smaller pulls. I didn't rerun the checks this run; two runs ago, the gate, bun x tsc --noEmit and bun scripts/design-lint.ts passed here, and the adapter tests passed 31 of 31.
---
This builds the adapter app shape from #437 on top of main's adapter block (src/apps/adapter.ts, #10055) and the contract in docs/ADAPTERS.md (#10056):
- Manifest:
forge.jsoncan nameadapter.source(a name and its hosts) andadapter.config, alongsideschedule.every. Every source host must also be listed inegress. - Pull runs:
src/apps/pull.tschecks eachPOST /pullanswer as it arrives and stops reading the reply past its size limit.src/core/adapter_pulls.tsstores each pull in theadapter_pulltable, added by migration20261109120000-adapter-pullwith its pin committed, and removes old rows. - Scheduling:
src/apps/runtime.tspulls each running adapter once it is due. It never wakes a sleeping app, and two pulls never run at once for the same app. - Status:
src/apps/adapter_status.tsshows Source, Last and Next on the app page and on the terminal's deploys screen, to the owner only. - Template:
examples/adapter-template/is a zero-dependency adapter to copy, with its own tests.
What to look at: this branch overlaps three smaller pieces of the same work:
- #10058 stores pull runs in a separate
pull_runtable, but its migration has the same timestamp as this branch's. - #10059 also adds
examples/adapter-template/, with the source hosts kept in its ownsrc/source.ts. - #10043, still open, is the scheduler this branch already has in
runtime.ts.
Whichever lands first, the others must be rebased onto it. If the split pulls land, this pull reduces to the parts nothing else covers: source and config on the manifest, egress joining, and the owner-only status. I left those smaller pulls alone and didn't touch #438 (which waits for the owner to name the two systems) or #439.
What I left out: the app reads secrets from its own environment; the manifest does not reference them. Adapters have no manual "pull now" button.
How to try it: deploy examples/adapter-template as an app. Its page shows Source, Last and Next to the owner, and each scheduled pull appears there with its counts and cursor. Tests: bun test test/adapter.test.ts test/adapter-runtime.test.ts test/apps-adapter.test.ts examples/adapter-template/test/adapter.test.ts.
The repository and issues held no instructions to reach other hosts or repositories.
🤖 Generated with Claude Code
This pull now builds the adapter app shape from #437 on top of what main already has: the adapter block (#10055), the docs/ADAPTERS.md contract (#10056) and the pull-run record from #10058.
Since #10058 landed, this branch no longer has its own pull storage. The adapter_pull table, its migration and pin, and src/core/adapter_pulls.ts are gone. Every pull is checked by main's parsePullAnswer and saved by main's recordPull in the pull_run table.
What changed
src/apps/pull_call.ts(new) is the caller. It sendsPOST /pullover loopback with the app's identity key as a bearer token and stops reading the reply past 64 KB. A timeout or an unreachable adapter is recorded as a failed pull. It also holdspullDue: a pull's start time is taken ascreatedAt − durationMs.src/core/runs.tsgainsprunePullRuns, so each app keeps its newest 200 pull runs.src/apps/runtime.tspulls each running adapter once it is due:
- production only, never a preview;
- never two pulls at once for the same app;
- a sleeping app is never woken, and an adapter's production is not put to sleep;
- stop waits for a pull already in flight.
- Manifest:
forge.jsoncan declareadapter.source(a name and its hosts) andadapter.config, next toschedule.every. Every source host must also be listed inegress. - Status:
src/apps/adapter_status.tsshows Source, Last pull and Next pull on the app page and on the terminal's Deploys screen, to the owner only. - Template:
examples/adapter-template/is a zero-dependency adapter to copy, with its own tests.
What to look at
- I changed one test assertion, because the test was wrong under main's rules rather than the code. In
test/adapter-runtime.test.ts, a second pull that returns the same cursor is now expected as a repeat: nothing written, the records counted as skipped. - #10059 also adds
examples/adapter-template/. Whichever lands second has to be rebased onto the other. - #10043 (the scheduler) describes what this branch already does in
runtime.ts; it can close when this lands. - I didn't touch #438, which waits for the owner to name the two systems, or #439.
Left out: the app reads secrets from its own environment; the manifest does not reference them. Adapters have no manual "pull now" button.
Checks run here: bun scripts/gate.ts, bun x tsc --noEmit and bun scripts/design-lint.ts pass. These tests pass:
test/adapter.test.ts,test/adapter-runtime.test.ts,test/apps-adapter.test.tsandtest/apps-pull.test.ts: 38 tests;test/schema-additive.test.tsandtest/schema-migrations.test.ts: 33 tests;- the template's own suite, run from
examples/adapter-template: 6 tests.
How to try it: deploy examples/adapter-template as an app. Its page shows Source, Last pull and Next pull to the owner, and each scheduled pull is recorded as a pull run with its counts and cursor.
The repository and issues held no instructions to reach other hosts or repositories.
🤖 Generated with Claude Code
This pull makes an app with an adapter block in forge.json a sync adapter (#390). The forge calls the app's POST /pull on production when a pull is due, records each pull, and shows the result to the app's owner on the web and in the terminal. This push merges main, which has added the pull record (#10058) and the adapter template (#10059).
What's kept from main:
- The pull record:
parsePullAnswerchecks the answer, andrecordPullsaves it as a run of kindpullin thepull_runtable. This branch's ownadapter_pulltable, its migration and its pin are gone. - The template:
examples/adapter-template/ismain's, taken whole. Only an admin signed by the forge may pull, and the source lives insrc/source.ts. This branch's template and itsguard.tsare gone.
What this branch adds on top:
- The caller:
main'spull.tsleft "the caller that sends the request" for later;runPullis it.
- It signs the forge's call as forge|forge|admin|<expiry> with the app's identity key, which main's template accepts. The bearer token is gone.
- It reads at most 64 KiB, and an endless body is cut off early.
- It gives up after five minutes, and records every outcome through recordPull.
- Pruning:
recordPullkeeps each app's newest 200 pull runs (prunePullRunsinsrc/core/runs.ts). - Readers: the scheduler,
pullDue, and the Source / Last pull / Next pull lines on the app page and the Deploys screen now read pull runs. A pull's start time is its record time minus its duration.
Tests changed:
test/adapter.test.tschecks that the template'swho()accepts the forge's signed call as admin, along with failure reasons, the last good cursor surviving a failure, the size cap and pruning.- In
test/adapter-runtime.test.ts, a second pull with the same cursor is nowmain's repeat: recorded, with nothing written. That test's expectation was wrong undermain's rules, so I changed it.
Docs: docs/ADAPTERS.md now says what runs. Its template section says the forge signs its call and that the parser accepts source. docs/features/adapters.md is updated to match.
Branch history: another of my runs had already pushed d76dd23f, a merge of #10058 alone. It is merged here with its changes left out (-s ours), because this tree merges both #10058 and #10059.
Checks run: the gate, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. 127 tests pass across test/adapter.test.ts, test/adapter-runtime.test.ts, test/apps-adapter.test.ts, test/apps-pull.test.ts, test/adapter-template.test.ts, test/apps.test.ts, test/schema-additive.test.ts, test/readme-front-door.test.ts, test/tui-deploys.test.ts, test/tui-deploys-egress.test.ts, test/app-env.test.ts, test/here-manifest.test.ts, test/confirm-app-stop.test.ts and the template's own tests.
Not done here: the first real adapters (the owner chooses which), and Atlas turning a description into apps and adapters.
To try it: copy examples/adapter-template/ into a repository, set its source in src/source.ts, and make it an app. Production is pulled on its schedule, and its owner sees Source, Last pull and Next pull on the app page and the Deploys screen.
🤖 Generated with Claude Code
Changes requested. Fix first: the direction. The owner put the pipeline first (#6534, #6611, #5276, #3316, #6217, #6428, #1242, #1073) before any feature, and this pull is a new feature (adapters, #390) that touches none of them. It also adds 860 lines when its own pieces (#10043 scheduler, split pulls) were meant to land small and single-purpose. Park it behind the pipeline items, or split it so each piece is one purpose, and say in the pull why it jumps the queue. A bare 'the owner decides' is not an answer.
Also fix, in the same round:
- stop() awaits in-flight pulls, and a pull may run 5 minutes (PULL_TIMEOUT_MS). A rollout or shutdown can stall that long, which works against #6611 (queue survives a rollout). Abort in-flight pulls on stop with an AbortSignal and record them as failed, rather than waiting.
- Web page, principles 1, 7 and 8: the Last pull line reads 'ok — pulled 3, skipped 0 (12 ms)', a sentence with a dash and milliseconds jammed into one dd. Say it in plain words (docs/WRITING.md), one fact per dt/dd row. Confirm the bare dl is the named fact-row component of docs/DESIGN.md and that it sits in the page's existing rhythm. Run design-lint again after the merge.
- Clean: src/apps/runtime.ts has a stray blank line between imports, an odd
Promise.all(this.pulling.size === 0 ? [] : [this.pullsDone()]), andinFlight/pullsDonedeclared after the method that uses them. MakepullingandinFlightone set, declared with the other fields. test/adapter.test.ts has doubled blank lines. - adapter_status.ts:
duefor a last pull that is already late gives nextAt undefined, which the pages print as 'due now'. That is fine, butNumber.isFinite(due)is dead since both operands are finite. Remove it. - Tests: the runtime test pokes private
envsandsweepby cast. Expose a small seam rather than casting. The stop-waits test only passes when stop waits, so it will need to change with point 1.
I made no new commit and pushed nothing. Another run of this issue had already merged #10058 and #10059 into this branch (4a84093c, f4f3bea5), and the branch is current with main. Its check is running on f4f3bea5. Here, the gate, bun x tsc --noEmit and bun scripts/design-lint.ts pass, as do 71 tests across the six adapter, pull and schema files and the template's own 14 tests. #10043 (the scheduler) now has torque assigned to it, and this branch already does that work in runtime.ts.
---
The forge now pulls adapters by itself. This is built on what main already has: the adapter block (#10055), the docs/ADAPTERS.md contract (#10056), the pull-run record (#10058) and the template adapter (#10059). This branch adds the part that sends the pull, the timer that runs it, and what the owner sees about it.
What changed
- The caller,
runPullinsrc/apps/pull.ts. It sendsPOST /pullover loopback and signs the request as an admin with the app's identity key, using the same headers the proxy signs. That matters because the template lets only an admin pull. It stops reading the reply past 64 KB and gives up after five minutes. Every outcome goes through main'sparsePullAnswerandrecordPull, and a timeout or an unreachable adapter is recorded as a failed pull. - Keeping history bounded.
recordPullnow keeps each app's newest 200 pull runs, usingprunePullRunsinsrc/core/runs.ts. - The timer,
pullOwedinsrc/apps/runtime.ts. It pulls each running adapter once it is due:
- production only, never a preview;
- never two pulls at once for the same app;
- a sleeping app is never woken, and an adapter's production is not put to sleep;
- stop waits for a pull already in flight.
- Manifest:
forge.jsoncan declareadapter.source(a name and its hosts) andadapter.config, next toschedule.every. Every source host must also be listed inegress, and the app process receives them asFORGE_ADAPTER_HOSTS. - Status:
src/apps/adapter_status.tsshows Source, Last pull and Next pull on the app page and on the terminal's Deploys screen, to the owner only, with the text escaped.
What to look at
- #10043 (assigned to torque) describes what
pullOwedalready does. If this lands first, #10043 can close. If not, one of the two has to give way. - In
test/adapter-runtime.test.ts, a second pull that returns the same cursor is now expected to be recorded as a repeat: nothing written, the records counted as skipped. That is main's rule, so the old assertion was wrong, not the code. - I didn't touch #438, which waits for the owner to name the two systems, or #439.
Left out: the app reads secrets from its own environment; the manifest does not reference them. Adapters have no manual "pull now" button.
How to try it: deploy examples/adapter-template as an app. Its page shows Source, Last pull and Next pull to the owner, and each scheduled pull is recorded as a pull run with its counts and cursor. Tests: bun test test/adapter.test.ts test/adapter-runtime.test.ts test/apps-adapter.test.ts test/apps-pull.test.ts, and bun test inside examples/adapter-template.
The repository and issues held no instructions to reach other hosts or repositories.
🤖 Generated with Claude Code
This pull now does one thing: it lets the adapter block in forge.json declare source and config. On main, docs/ADAPTERS.md describes adapter.source and its rule that every source host is also in egress, and the template README waits for "the parser accepts source". But src/apps/adapter.ts on main refuses source as an unknown key, so an adapter written to the contract fails its deploy. This pull removes that contradiction.
What changed:
src/apps/adapter.ts:main's parser is the base:schedule.every, the same error sentences, and stray keys refused at every level. It now also reads two optional keys:
- source: a name and 1 to 8 bare lowercase hosts.
- config: up to 32 typed fields, each named by an environment variable. A secret names a variable and never holds a value. A field may not name a variable the forge sets.
src/apps/manifest.ts: a source host thategressdoes not list makes the manifest invalid, with the host named, as the contract says.docs/ADAPTERS.mdanddocs/APPS.md: they now saysourceandconfigare optional, that any other key is refused, and that the parser reads the block.- Tests: they are in
test/apps-adapter.test.tsand fail onmain's parser. One sentence ofmain's in that file changed: an unknown key now readsadapter takes only schedule, source and config, not "url", becausesourceandconfigare allowed keys now.
The split: the review asked to park or split this pull, and I split it. The pull drops from about 860 added lines to 203.
- The scheduler leaves for #10043. That covers
runPull,pullOwed, the sweep exemption, pruning andFORGE_ADAPTER_HOSTS. #10043 is assigned to torque, and the plan showed it held because "an open pull already proposes it", which was this pull. - The Source / Last pull / Next pull lines on the app page and the terminal leave for #10037.
- Their code stays in this branch's history at
f4f3bea5for whoever takes those issues. - Review points 1 to 5 were about the scheduler, the page lines and their tests. That code is gone from this pull, so I have not changed it. #10043 and #10037 should still follow them: abort pulls on stop, plain-word rows, one set for in-flight pulls, a test seam instead of casts.
Why this goes ahead of the pipeline items: it is not a pipeline item, and it no longer competes with one. It is now one small fix: main ships a contract that its own parser rejects. Leaving it open would still leave a written contract the code refuses. Removing the scheduler also releases #10043.
Checks run: the gate, bun x tsc --noEmit and bun scripts/design-lint.ts are clean. 56 tests pass across test/apps-adapter.test.ts, test/apps-pull.test.ts, test/adapter-template.test.ts, test/apps.test.ts, test/readme-front-door.test.ts and the template's own tests.
To try it: add "source": { "name": "Example", "hosts": ["api.example.com"] } to the template's adapter block. The manifest parses while egress lists api.example.com and fails, naming the host, once it doesn't.
🤖 Generated with Claude Code
I made no new commit and pushed nothing. Another run of this issue had already narrowed this pull (1834eedc) after review asked for a split, and the branch is still current with main. The forge's check is running on that head. Here, bun scripts/gate.ts and bun x tsc --noEmit pass, and so do the 26 tests in test/apps-adapter.test.ts and test/apps-pull.test.ts.
---
forge.json can now declare where an adapter reads from and what settings it takes. docs/ADAPTERS.md, docs/APPS.md and the template's README already described adapter.source, but src/apps/adapter.ts on main refused it. This pull makes the parser accept it, plus adapter.config:
source(optional):nameis the external system as people read it, andhostslists 1 to 8 bare lowercase host names. Every source host must also be listed inegress, so a manifest that doesn't open a host fails and names that host.config(optional): the settings the adapter reads, each named by the environment variable it comes from.
- Each has a type of string, number, boolean or secret, and is either required or has a default.
- A secret names a variable and never holds a value.
- A setting may not name a variable the forge sets itself.
- An unknown key at any level is refused, as main's parser already does for
schedule.
src/apps/manifest.ts carries the parsed block. docs/ADAPTERS.md and docs/APPS.md now say the parser reads source and config. Tests are in test/apps-adapter.test.ts.
What to look at
- The scheduler (sending
POST /pull, the timer, keeping the newest pull runs) left this pull for #10043, assigned to torque. The Source / Last pull / Next pull lines on the app page and terminal left it for #10037. Their earlier code is still in this branch's history at f4f3bea5 for whoever takes them. - I didn't touch #438, which waits for the owner to name the two systems, or #439.
Left out: nothing reads config at runtime yet, so the app still reads its settings from its own environment.
How to try it: add "adapter": {"schedule": {"every": 900}, "source": {"name": "Shop API", "hosts": ["api.shop.example"]}} to an app's forge.json. Without api.shop.example in egress, the deploy refuses the manifest and names the host. Tests: bun test test/apps-adapter.test.ts.
The repository and issues held no instructions to reach other hosts or repositories.
🤖 Generated with Claude Code
Approved. I read only the diff since f4f3bea5; I did not re-check the preview or screenshots. This round shrinks the pull to the manifest parser, the egress rule, docs and tests. Nothing here changes a page, a screen or a request path.
Earlier asks:
- Direction: met. The pull is now one small fix, because main ships a contract that its own parser rejected. The scheduler and the status lines are out.
- Both doors and how-to: met by scope. No person-facing feature remains, README is untouched, and docs/ADAPTERS.md and docs/APPS.md say what the parser reads.
- Tests: met for what remains. test/apps-adapter.test.ts covers source and config, bare hosts, secrets without values, reserved variables, default types, and the egress rule. Those tests would fail on main parser. The scheduler tests left with the scheduler, so #10043 must bring its own.
- Security: met. The unbounded read, the cursor shown to viewers, the host list, and the table growth all left with the code they were about. Here, secrets are names and never values, forge-owned variables are refused, and hosts must be bare lowercase hostnames that egress also lists.
- Design: nothing to review, because no web code changed.
- Dead code: met. hostAllowed is gone.
Non-blocking, two things I would do next:
- Refuse a config field that sets both required: true and a default. The two contradict, and the parser accepts both today. Also refuse duplicate entries in source.hosts.
- When #10043 and #10037 pick up the scheduler and the status lines, they must meet the earlier asks that were left open: abort a pull on stop, show plain-word dl rows to the owner only, escape adapter text, and keep the cursor hidden.
Merged
maininto the branch and resolved the conflicts insrc/apps/manifest.ts,src/apps/process.ts,src/apps/runtime.ts,src/core/schema.tsandsrc/web/studio/app.ts, keeping what both sides meant. The adapter feature now sits on top of main's newer layout.adapter_pullentry this branch had added toschema.tsis nowsrc/core/migrations/20261109120000-adapter-pull.ts. The table itself is unchanged. Its pin was written byscripts/pin-migrations.ts.schema.tsis main's version.adaptersits beside main'sservice,looksAsOwner,pages,launch,egress,limitsandtoolbar. It is now optional (adapter?: AdapterManifest | null), like main's newer fields, so the existing test fixtures that don't set it stay valid. All readers already usemanifest.adapter?..adapterHosts(which setsFORGE_ADAPTER_HOSTS) is passed into main's restructured, cell-aware spawn.featureis passed alongside it.placement,storageEveryMsandgateoptions.adapterOfis kept next to main'scellForandnoteCell.splitlayout. The merge had dropped thetimeimport from../ui.ts, which the adapter line needs, so it is restored.bun scripts/gate.ts,bun x tsc --noEmitandbun scripts/design-lint.tspass. 156 tests across 12 files pass: adapter, the adapter template, apps, schema migrations, pin edits, app env, studio and process tests.scripts/fast.tsreported the diff too broad to name (exit 2), so I picked those files myself and left the full suite to the forge's check.Try: open an adapter app's page on the preview; its "Syncs …" line names the source, the last and next pull, and the cursor.
🤖 Generated with Claude Code
Continues #10032