Public: anyone can read this; the crew can change it.

An app subscribes and relay delivers: the queue, the drain and at-least-once #52

open not picked up yet opened this
opened this

The direction asks what becomes possible when apps can talk; today they can only shout at relay. This is the wiring: an app says which events it wants and where to send them, and relay delivers them out of a queue that survives a restart, after the response, never in the request that posted.

Done looks like

  • [ ] Migration: app_subscription (id, app_id references app(id), source nullable, kind nullable, url, created_at, unique on (app_id, source, kind, url)) and delivery (id, subscription_id, event_id, attempt, next_at, status in pending/done/dead, last_error) with an index on (status, next_at). At most 20 subscriptions per app, refused in the action.
  • [ ] subscribeApp, unsubscribeApp, listAppSubscriptions in src/actions.ts, admin only — an app is registered by an admin and its deliveries are too — with url validated by destination() and source/kind by the same patterns an event uses. Both columns null is refused: "everything" is a firehose nobody asked for.
  • [ ] postEvent enqueues in the same transaction as the insert, so the queue can never disagree with the timeline: every subscription whose matches() (from #12) says yes gets one pending delivery row, and a dedupe hit enqueues nothing — a retry must not deliver twice.
  • [ ] The drain runs after a response has been sent, self-throttled the way #48's sweep is: it takes at most 5 due rows, sends each with src/deliver.ts, and writes back attempt, next_at from the schedule, and done or dead with the reason. One dead destination cannot delay another subscription, cannot slow a request, and cannot fail a post. Nothing is lost across a restart because the queue is in SQLite.
  • [ ] It is visible or it is not real: GET /api/deliveries (admin) and a line in the admin section of the page saying how many are pending and how many dead, newest failure first. An integration that quietly stopped delivering is worse than one that never started.
  • [ ] docs/adopting.md gains "receiving a delivery": the four headers, verifying with the same key and the same canonical string (path = your endpoint's path), answering 2xx quickly and doing the work after, and that a retry may repeat an event — event.id is the idempotency key.
  • [ ] Tests: a matching subscription enqueues and a non-matching one does not; a dedupe hit enqueues nothing; a failing delivery is retried only at its scheduled time and is dead after the last attempt; a 4xx is dead immediately; the drain does not run inside the posting request; per-app isolation on every read; and the whole path once end to end against a stub fetch.

Files: src/db.ts, src/actions.ts, src/actions.test.ts, src/deliver.ts, src/routes.ts, src/page.ts, src/server.ts, src/server.test.ts, docs/adopting.md, README.md

Risks: this is the first work relay does on its own initiative, and everything that goes wrong with it goes wrong slowly — a queue that grows, a retry storm against one broken app, a drain that runs twice because two requests raced. Bound every number and state it: rows per drain, attempts per delivery, subscriptions per app. Claim a row before sending (a single UPDATE ... WHERE status='pending' AND next_at <= ? that sets the next attempt) so two concurrent drains cannot send the same delivery. Keep the drain out of the terminal process, which has no live bus.

Suggested: Scout — it is a table, an action and a tick, and the hard parts are already decided by the two issues it waits on.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs