Public: anyone can read this; the crew can change it.

Paste a lead notification email (Zillow, Realtor.com, a website form) and the lead's name, email, phone, source and question are filled in to check and add #48

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

mason/speed-to-lead:mason/46-paste-a-lead-notification-email-zillow-r → main 3 commits fork 2 threads · 2 unresolved

  • Merged. mason/46-paste-a-lead-notification-email-zillow-r was merged into main as 0c396da.

Review

Sign in to review this change.
15 files changed +684 −46
README.md +37
@@ -67,6 +67,30 @@ what you see right after a restart. Do step 1 first.
6767 for someone else; the seven made-up inquiries come in turn (some "Text
6868 OK", some "Email only"), and restarting the samples clears them.
6969
70 10. Paste a lead email. Real leads reach you as notification emails. Copy
71 this made-up one, press "Paste a lead email" beside "Log a lead by
72 hand", paste it and press "Read the email":
73
74 ```
75 From: Zillow Premier Agent <[email protected]>
76 Subject: New Zillow lead: 12 Example St
77
78 You have a new contact on Zillow.
79
80 Name: Priya Raman
81 Email: [email protected]
82 Phone: (206) 555-0143
83 Message: Is 12 Example St still available? Can we see it this weekend?
84
85 Reply to this lead within five minutes for the best chance of a showing.
86 ```
87
88 Nothing is added yet. The desk comes back with the hand form filled in
89 (Priya Raman, her email and phone, Zillow, her question) and a line naming
90 anything the email did not hold. "Text OK" stays unticked: consent is never
91 read from an email. Check it and press "Add to the desk": her reply is
92 drafted.
93
7094 Nothing is really sent in this prototype: "send" records the message in the
7195 lead's history and goes no further. To run the tour again, restart the
7296 samples (step 1).
@@ -78,6 +102,19 @@ samples (step 1).
78102 terminal desk. They go through the same path as `POST /leads`, for anyone who
79103 may add a lead. The turn is read off the desk, so it needs no extra state.
80104
105 **Pasting a lead email.** `readLeadEmail` (`src/intake.ts`) reads the
106 labelled lines of a Zillow, Realtor.com or website-form notification (Name,
107 Email, Phone, Message and the usual variants) into a new lead's fields and
108 sets the source from the sender (Realtor.com is filed under "Website form":
109 the desk has no source of its own for it). It only prefills: nothing is
110 added until the person presses the add button, a field it cannot find is left
111 out, and consent is never taken from an email. The same read is `POST
112 /leads/parse` with `{"text": …}` (at most 10,000 characters; JSON back, nothing
113 written) and the MCP `parse_lead_email` tool, for whoever may add a lead. The
114 terminal has it too: `v` on the desk takes the pasted
115 text, ctrl-d reads it, and the add screen opens with what was found (and a
116 line naming what was not) for you to check; consent starts as n.
outdated

The terminal must have this too (both doors). Add it to src/tty.ts and document how to use it here.

117
81118 **Fixing a lead's details.** A website form brings in a misspelled name or a
82119 mistyped phone, so an open lead's page has **Edit details** (name, email,
83120 phone; checked as when a lead is added, and one of email or phone must
contract/openapi.json +75
@@ -205,6 +205,81 @@
205205 }
206206 }
207207 },
208 "/leads/parse": {
209 "post": {
210 "summary": "Read a pasted lead notification email (Zillow, Realtor.com, a website form) into a new lead's fields. Only what the email labels is returned, never consent. Nothing is written; add the lead with POST /leads once checked. Needs the same role as adding a lead.",
211 "operationId": "postLeadsParse",
212 "requestBody": {
213 "required": true,
214 "content": {
215 "application/json": {
216 "schema": {
217 "type": "object",
218 "properties": {
219 "text": {
220 "type": "string",
221 "maxLength": 10000
222 }
223 },
224 "required": [
225 "text"
226 ]
227 }
228 }
229 }
230 },
231 "responses": {
232 "200": {
233 "description": "Read a pasted lead notification email (Zillow, Realtor.com, a website form) into a new lead's fields. Only what the email labels is returned, never consent. Nothing is written; add the lead with POST /leads once checked. Needs the same role as adding a lead.",
234 "content": {
235 "application/json": {
236 "schema": {
237 "type": "object",
238 "properties": {
239 "name": {
240 "type": "string",
241 "minLength": 1,
242 "maxLength": 120
243 },
244 "email": {
245 "type": "string",
246 "format": "email"
247 },
248 "phone": {
249 "type": "string"
250 },
251 "source": {
252 "enum": [
253 "zillow",
254 "website",
255 "open_house",
256 "referral"
257 ]
258 },
259 "message": {
260 "type": "string",
261 "minLength": 1,
262 "maxLength": 2000
263 }
264 },
265 "description": "Only the fields the email held."
266 }
267 }
268 }
269 },
270 "default": {
271 "description": "{ error, message }: why not",
272 "content": {
273 "application/json": {
274 "schema": {
275 "$ref": "#/components/schemas/Error"
276 }
277 }
278 }
279 }
280 }
281 }
282 },
208283 "/leads/{id}": {
209284 "get": {
210285 "summary": "One lead, with every message: sent, drafted, and the follow-ups to come. Ask for application/json.",
src/actions.ts +19
@@ -57,6 +57,7 @@ import { emit } from "./live.ts";
5757 import { currentAgent, storeAgent } from "./profile.ts";
5858 import { nextInquiry, seed } from "./sample.ts";
5959 import { goesAt, isZone } from "./quiet.ts";
60 import { MAX_EMAIL_TEXT, readLeadEmail } from "./intake.ts";
6061 import { MAX_ADDRESS, timeOf } from "./showings.ts";
6162 import type { Who } from "./who.ts";
6263
@@ -296,6 +297,24 @@ function add(
296297 return ok(added);
297298 }
298299
300 /**
301 * A pasted notification email read into the fields of a new lead, to check
302 * before adding. Same cell as adding (lead create); writes nothing, so
303 * there is nothing to audit.
304 */
305 export function parseLeadEmail(
306 who: Who,
307 input: Record<string, unknown>,
308 ): Result<Partial<NewLead>> {
309 if (permit(who, "lead", "create") !== "yes")
310 return refuse(who, "Sign in to add a lead.", "You may not add a lead.");
311 if (typeof input.text !== "string" || input.text.trim() === "")
312 return fail("invalid", "Paste the text of the email.");
313 if (input.text.length > MAX_EMAIL_TEXT)
314 return fail("invalid", `The email is at most ${MAX_EMAIL_TEXT} characters.`);
315 return ok(readLeadEmail(input.text));
316 }
317
299318 /** The fields of a lead's details, in the order they are told. */
300319 const CONTACT_FIELDS = ["name", "email", "phone"] as const;
301320 export type ContactField = (typeof CONTACT_FIELDS)[number];
src/contract.test.ts +3
@@ -24,6 +24,7 @@ import {
2424 cancelHeld,
2525 cancelShowing,
2626 logLeadReply,
27 parseLeadEmail,
2728 saveProfile,
2829 setState,
2930 } from "./actions.ts";
@@ -108,6 +109,8 @@ const doors: Record<
108109 const added = addLead(db, who, NEW).ok;
109110 // The sample inquiry is the same cell, through its own door.
110111 expect(addSampleInquiry(db, who).ok).toBe(added);
112 // So is reading a pasted lead email (POST /leads/parse, the MCP tool parse_lead_email).
113 expect(parseLeadEmail(who, { text: "Name: Pat" }).ok).toBe(added);
111114 return added;
112115 },
113116 // The spreadsheet is the same cell through its own door (GET /leads.csv, the MCP tool export_leads, `x` in the terminal).
src/fixtures/lead-emails.ts added +35
@@ -0,0 +1,35 @@
1 /** Three notification emails as portals and forms send them. The people are made up. */
2
3 export const ZILLOW = `From: Zillow Premier Agent <[email protected]>
4 Subject: New Zillow lead: 12 Example St
5
6 You have a new contact on Zillow.
7
8 Name: Priya Raman
9 Email: [email protected]
10 Phone: (206) 555-0143
11 Message: Is 12 Example St still available? Can we see it this weekend?
12
13 Reply to this lead within five minutes for the best chance of a showing.`;
14
15 export const REALTOR = `From: Realtor.com Leads <[email protected]>
16 Subject: Realtor.com lead for 48 Sample Ave
17
18 Lead details
19 Contact Name: Tomas Okafor
20 E-mail: [email protected]
21 Telephone: 425-555-0178
22 Comments: We are relocating in June.
23 Does the house on Sample Ave allow dogs?
24
25 Lead ID: 99120`;
26
27 export const FORM = `Subject: New message from your website contact form
28
29 Someone sent a message through your website.
30
31 Full name: Jo Whitfield
32 Email address: [email protected]
33 Message: Hi, I would like a price opinion on my condo.
34
35 Sent from the contact form at example-realty.test`;
src/intake.test.ts added +55
@@ -0,0 +1,55 @@
1 import { expect, test } from "bun:test";
2 import { readLeadEmail } from "./intake.ts";
3 import { FORM, REALTOR, ZILLOW } from "./fixtures/lead-emails.ts";
4
5 test("a Zillow notification gives name, email, phone, source and question", () => {
6 expect(readLeadEmail(ZILLOW)).toEqual({
7 name: "Priya Raman",
8 email: "[email protected]",
9 phone: "(206) 555-0143",
10 source: "zillow",
11 message: "Is 12 Example St still available? Can we see it this weekend?",
12 });
13 });
14
15 test("a Realtor.com notification reads its wording and a question over two lines", () => {
16 expect(readLeadEmail(REALTOR)).toEqual({
17 name: "Tomas Okafor",
18 email: "[email protected]",
19 phone: "425-555-0178",
20 source: "website",
21 message: "We are relocating in June.\nDoes the house on Sample Ave allow dogs?",
22 });
23 });
24
25 test("a website form gives what it labels and leaves out the phone it does not have", () => {
26 const read = readLeadEmail(FORM);
27 expect(read).toEqual({
28 name: "Jo Whitfield",
29 email: "[email protected]",
30 source: "website",
31 message: "Hi, I would like a price opinion on my condo.",
32 });
33 expect("phone" in read).toBe(false);
34 });
35
36 test("consent is never read from an email", () => {
37 const read = readLeadEmail(`${ZILLOW}\nConsent: yes\nText OK: yes`);
38 expect("consent" in read).toBe(false);
39 });
40
41 test("text with no recognisable fields gives an empty object", () => {
42 expect(readLeadEmail("Hello,\nsee you Tuesday.\nThanks")).toEqual({});
43 expect(readLeadEmail("")).toEqual({});
44 });
45
46 test("a bad phone or email is left out, not guessed", () => {
47 expect(readLeadEmail("Phone: call me\nEmail: not an address")).toEqual({});
48 });
49
50 test("long lines of colons and spaces are read in linear time", () => {
51 const started = performance.now();
52 readLeadEmail(`${" ".repeat(9_000)}:${":".repeat(900)}`);
53 readLeadEmail(`Email: ${"a@".repeat(4_000)}`);
54 expect(performance.now() - started).toBeLessThan(500);
55 });
src/intake.ts added +148
@@ -0,0 +1,148 @@
1 /**
2 * Reading a lead notification email: the text a portal or a website form
3 * sends the agent ("Name: …", "Email: …", "Phone: …", "Message: …") becomes
4 * the fields of a new lead. Plain text in, plain data out; nothing is
5 * written and nothing is guessed: a field the email does not label is left
6 * out, and consent to be texted is never among them. The person checks what
7 * was found before it becomes a lead.
8 *
9 * Only labelled lines are read, by splitting at the first colon, so no
10 * pattern here can backtrack.
11 */
12
13 import type { NewLead } from "./leads.ts";
14 import type { Source } from "./drafts.ts";
15
16 /** The most a pasted email may hold. */
17 export const MAX_EMAIL_TEXT = 10_000;
18
19 type Field = "name" | "email" | "phone" | "message" | "source";
20
21 const LABELS: Readonly<Record<string, Field>> = {
22 name: "name",
23 "full name": "name",
24 "contact name": "name",
25 "lead name": "name",
26 "customer name": "name",
27 email: "email",
28 "e-mail": "email",
29 "email address": "email",
30 "contact email": "email",
31 phone: "phone",
32 "phone number": "phone",
33 telephone: "phone",
34 mobile: "phone",
35 cell: "phone",
36 "contact phone": "phone",
37 message: "message",
38 comments: "message",
39 comment: "message",
40 question: "message",
41 inquiry: "message",
42 "their message": "message",
43 "customer message": "message",
44 source: "source",
45 };
46
47 /** The longest label, in characters before the colon; a longer line is prose. */
outdated

This comment does not describe MAX_LABEL (the longest label before the colon). Replace it.

48 const MAX_LABEL = 30;
49
50 const labelled = (line: string): { field: Field | undefined; value: string } | undefined => {
51 const colon = line.indexOf(":");
52 if (colon < 1 || colon > MAX_LABEL) return undefined;
53 const label = line.slice(0, colon).trim().toLowerCase();
54 if (label === "" || label.includes("http")) return undefined;
55 return { field: LABELS[label], value: line.slice(colon + 1).trim() };
56 };
57
58 const CONTROL = /[\x00-\x08\x0b-\x1f\x7f-\x9f]/g;
59
60 const emailIn = (value: string): string | undefined => {
61 for (const token of value.split(/[\s<>(),;:"'[\]]+/)) {
62 const at = token.indexOf("@");
63 if (at < 1 || at !== token.lastIndexOf("@")) continue;
64 const dot = token.lastIndexOf(".");
65 if (dot > at + 1 && dot < token.length - 1 && token.length <= 254)
66 return token.toLowerCase();
67 }
68 return undefined;
69 };
70
71 const phoneIn = (value: string): string | undefined => {
72 const phone = value.trim();
73 if (phone.length > 20 || phone.length === 0) return undefined;
74 let digits = 0;
75 for (const ch of phone) {
76 if (ch >= "0" && ch <= "9") digits++;
77 else if (!" ()+-.".includes(ch)) return undefined;
78 }
79 return digits >= 7 && digits <= 15 ? phone : undefined;
80 };
81
82 /** Where the email came from, by the lines that say so: its head, sender and source lines. */
83 const sourceOf = (lines: readonly string[], named: string | undefined): Source | undefined => {
84 const head = lines
85 .filter((l, i) => i < 8 || /^(from|sender|reply-to|subject|via)\s*:/i.test(l))
86 .join("\n")
87 .toLowerCase();
88 const said = named?.toLowerCase().replace(/[\s-]+/g, "_").replace(/_form$/, "");
89 if (said === "zillow" || said === "open_house" || said === "referral" || said === "website")
90 return said;
91 if (head.includes("zillow")) return "zillow";
92 // The desk has no Realtor.com source: it is a website the lead came in through.
93 if (head.includes("realtor.com")) return "website";
94 if (/website|web form|contact form|form submission|new inquiry from your/.test(head))
95 return "website";
96 return undefined;
97 };
98
99 export function readLeadEmail(text: string): Partial<NewLead> {
100 const lines = text
101 .replace(/\r\n?/g, "\n")
102 .replace(CONTROL, " ")
103 .split("\n")
104 .map((l) => l.replace(/^[\s>]+/, "").trimEnd());
105 const found: Partial<Record<Field, string>> = {};
106 for (let i = 0; i < lines.length; i++) {
107 const read = labelled(lines[i]!);
108 if (read === undefined || read.field === undefined || found[read.field] !== undefined)
109 continue;
110 if (read.field === "message") {
111 // A question runs on until the next label or a blank line after it.
112 const body = [read.value];
113 let j = i + 1;
114 for (; j < lines.length; j++) {
115 const line = lines[j]!;
116 if (labelled(line) !== undefined) break;
117 if (line === "" && body.join("").trim() !== "") break;
118 body.push(line);
119 }
120 i = j - 1;
121 const message = body.join("\n").trim();
122 if (message !== "") found.message = message.slice(0, 2000);
123 } else if (read.value !== "") found[read.field] = read.value;
124 }
125
126 const out: { -readonly [K in keyof NewLead]?: NewLead[K] } = {};
127 const name = found.name?.slice(0, 120).trim();
128 if (name) out.name = name;
129 const email = found.email === undefined ? undefined : emailIn(found.email);
130 if (email !== undefined) out.email = email;
131 const phone = found.phone === undefined ? undefined : phoneIn(found.phone);
132 if (phone !== undefined) out.phone = phone;
133 const source = sourceOf(lines, found.source);
134 if (source !== undefined) out.source = source;
135 if (found.message !== undefined) out.message = found.message;
136 return out;
137 }
138
139 const FOUND_FIELDS = ["name", "email", "phone", "source", "message"] as const;
140
141 /** Which of the fields the email did not hold, in a line. */
142 export const missingLine = (found: Partial<NewLead>): string => {
143 const missing = FOUND_FIELDS.filter((f) => found[f] === undefined);
144 if (missing.length === FOUND_FIELDS.length)
145 return "Nothing was found in that email. Fill the form in by hand.";
146 if (missing.length === 0) return "Found everything. Check it, then add the lead.";
147 return `Not found in the email: ${missing.join(", ")}. Check what was found and fill in the rest.`;
148 };
src/mcp.ts +13
@@ -24,9 +24,11 @@ import {
2424 bookShowing,
2525 cancelShowing,
2626 logLeadReply,
27 parseLeadEmail,
2728 type Result,
2829 } from "./actions.ts";
2930 import { MAX_NOTE, OUTCOMES } from "./calls.ts";
31 import { MAX_EMAIL_TEXT } from "./intake.ts";
3032 import { MAX_ADDRESS } from "./showings.ts";
3133 import { SOURCES } from "./drafts.ts";
3234 import { title } from "./env.ts";
@@ -111,6 +113,17 @@ export const tools: readonly Tool[] = [
111113 },
112114 run: (db, who, args) => addLead(db, who, args),
113115 },
116 {
117 name: "parse_lead_email",
118 description:
119 "Read a pasted lead notification email (Zillow, Realtor.com, a website form) into the fields of a new lead: name, email, phone, source, message. Only what the email labels is returned, never consent; nothing is added. Check the fields, then call add_lead.",
120 inputSchema: {
121 type: "object",
122 properties: { text: { type: "string", maxLength: MAX_EMAIL_TEXT } },
123 required: ["text"],
124 },
125 run: (db, who, args) => parseLeadEmail(who, args),
126 },
114127 {
115128 name: "new_sample_inquiry",
116129 description:
src/pages/desk.ts +27 −11
@@ -10,7 +10,8 @@ import { recordLine } from "../clock.ts";
1010 import { isAnswer } from "../drafts.ts";
1111 import { goesLabel, RECORD_NOTE } from "../quiet.ts";
1212 import { showingWhen } from "../showings.ts";
13 import type { DeskLead, ResponseRecord, View } from "../leads.ts";
13 import { MAX_EMAIL_TEXT, missingLine } from "../intake.ts";
14 import type { DeskLead, NewLead, ResponseRecord, View } from "../leads.ts";
1415 import type { Agent } from "../env.ts";
1516 import type { Who } from "../who.ts";
1617
@@ -25,6 +26,8 @@ export function deskPage(input: {
2526 mayAdd: boolean;
2627 mayReset: boolean;
2728 notice?: string | undefined;
29 /** What a pasted email was read into: the add form opens with it filled in. */
30 prefill?: Partial<NewLead> | undefined;
2831 }): string {
2932 const { who, agent, leads, notice } = input;
3033 if (leads === undefined)
@@ -85,7 +88,7 @@ export function deskPage(input: {
8588 ${list}
8689 <p class="export"><a href="${escape(exportHref(show, q))}" download>Download leads (CSV)</a></p>
8790 </div>
88 ${input.mayAdd ? `<form class="reset" method="post" action="/sample/inquiry"><button class="button button-quiet">Bring in a new inquiry</button></form>${addForm}` : ""}
91 ${input.mayAdd ? `<form class="reset" method="post" action="/sample/inquiry"><button class="button button-quiet">Bring in a new inquiry</button></form>${pasteForm}${addForm(input.prefill)}` : ""}
8992 ${
9093 input.mayReset
9194 ? `<form class="reset" method="post" action="/sample/reset"><button class="button button-quiet">Restart the sample leads from now</button></form>`
@@ -180,22 +183,35 @@ const row = (lead: DeskLead, now: number, zone: string): string => `<li>
180183 </a>
181184 </li>`;
182185
183 const addForm = `<details class="card add">
186 const pasteForm = `<details class="card add">
187 <summary>Paste a lead email</summary>
188 <form class="stack" method="post" action="/leads/parse">
189 <label>The notification email, from Zillow, Realtor.com or a website form <textarea class="input" name="text" required maxlength="${MAX_EMAIL_TEXT}" rows="6"></textarea></label>
190 <button class="button">Read the email</button>
191 </form>
192 </details>`;
193
194 const addForm = (prefill?: Partial<NewLead>): string => `<details class="card add"${prefill === undefined ? "" : " open"}>
184195 <summary>Log a lead by hand</summary>
196 ${prefill === undefined ? "" : `<p class="muted" role="status">${escape(missingLine(prefill))} Text OK is never taken from an email: tick it only with written consent.</p>`}
185197 <form class="stack" method="post" action="/leads">
186 <label>Name <input class="input" name="name" required maxlength="120" autocomplete="off"></label>
187 <label>Email <input class="input" name="email" type="email" required maxlength="254" autocomplete="off"></label>
188 <label>Phone <input class="input" name="phone" type="tel" maxlength="20" autocomplete="off"></label>
198 <label>Name <input class="input" name="name" required maxlength="120" autocomplete="off" value="${escape(prefill?.name ?? "")}"></label>
199 <label>Email <input class="input" name="email" type="email" required maxlength="254" autocomplete="off" value="${escape(prefill?.email ?? "")}"></label>
200 <label>Phone <input class="input" name="phone" type="tel" maxlength="20" autocomplete="off" value="${escape(prefill?.phone ?? "")}"></label>
189201 <label>Source
190202 <select class="input" name="source">
191 <option value="website">Website form</option>
192 <option value="zillow">Zillow</option>
193 <option value="open_house">Open house</option>
194 <option value="referral">Referral</option>
203 ${SOURCE_OPTIONS.map(([value, label]) => `<option value="${value}"${prefill?.source === value ? " selected" : ""}>${label}</option>`).join("\n ")}
195204 </select>
196205 </label>
197 <label>What they asked <textarea class="input" name="message" required maxlength="2000" rows="3"></textarea></label>
206 <label>What they asked <textarea class="input" name="message" required maxlength="2000" rows="3">${escape(prefill?.message ?? "")}</textarea></label>
198207 <label class="check"><input type="checkbox" name="consent" value="yes"> They gave written consent to be texted</label>
199208 <button class="button">Add to the desk</button>
200209 </form>
201210 </details>`;
211
212 const SOURCE_OPTIONS = [
213 ["website", "Website form"],
214 ["zillow", "Zillow"],
215 ["open_house", "Open house"],
216 ["referral", "Referral"],
217 ] as const;
src/parse.test.ts added +112
@@ -0,0 +1,112 @@
1 import { afterAll, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { FORM, ZILLOW } from "./fixtures/lead-emails.ts";
6 import { serve } from "./server.ts";
7 import { sign } from "./who.ts";
8
9 const KEY = "a-key-only-the-forge-holds";
10 const server = serve({ port: 0, dataDir: mkdtempSync(join(tmpdir(), "app-")), identityKey: KEY });
11 const base = `http://${server.hostname}:${server.port}`;
12 afterAll(() => server.stop(true));
13
14 // The per-person ration is shared by every test file in the process, so each
15 // call is a fresh person of its own and never spends another file's tokens.
16 let people = 0;
17 const as = (id: string, role: "user" | "admin" = "user"): Record<string, string> => {
18 const identity = `${id}-parse${++people}|${id}|${role}|${Date.now() + 60_000}`;
19 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
20 };
21 const alice = () => as("alice");
22 const post = (text: string, who: Record<string, string>, headers: Record<string, string> = {}) =>
23 fetch(`${base}/leads/parse`, {
24 method: "POST",
25 headers: { "content-type": "application/json", ...who, ...headers },
26 body: JSON.stringify({ text }),
27 });
28 const pasteForm = (text: string, who: Record<string, string>) => {
29 const f = new FormData();
30 f.set("text", text);
31 return fetch(`${base}/leads/parse`, { method: "POST", body: f, headers: who });
32 };
33 const root = () => as("root", "admin");
34 const rows = async () => {
35 const desk = (await (await fetch(`${base}/leads`, { headers: { ...root(), accept: "application/json" } })).json()) as { counts: { all: number } };
36 const audit = (await (await fetch(`${base}/api/audit`, { headers: root() })).json()) as unknown[];
37 return [desk.counts.all, audit.length];
38 };
39
40 test("the API returns the fields found, and writes no lead and no audit row", async () => {
41 const before = await rows();
42 const r = await post(ZILLOW, alice());
43 expect(r.status).toBe(200);
44 expect(await r.json()).toEqual({
45 name: "Priya Raman",
46 email: "[email protected]",
47 phone: "(206) 555-0143",
48 source: "zillow",
49 message: "Is 12 Example St still available? Can we see it this weekend?",
50 });
51 await pasteForm(FORM, alice());
52 expect(await rows()).toEqual(before);
53 });
54
55 test("the desk comes back with the add form filled in, consent unticked, and names what is missing", async () => {
56 const r = await pasteForm(FORM, alice());
57 expect(r.status).toBe(200);
58 const page = await r.text();
59 expect(page).toContain('value="Jo Whitfield"');
60 expect(page).toContain('value="[email protected]"');
61 expect(page).toContain("Not found in the email: phone.");
62 expect(page).toContain('<option value="website" selected>');
63 expect(page).toMatch(/name="consent" value="yes">/);
64 expect(page).not.toMatch(/name="consent"[^>]*checked/);
65 });
66
67 test("an email with nothing recognisable says nothing was found", async () => {
68 const r = await post("Hello there, see you Tuesday.", alice());
69 expect(await r.json()).toEqual({});
70 const page = await (await pasteForm("Hello there, see you Tuesday.", alice())).text();
71 expect(page).toContain("Nothing was found in that email.");
72 });
73
74 test("10,001 characters is a 422, 10,000 is read", async () => {
75 expect((await post("x".repeat(10_001), alice())).status).toBe(422);
76 expect((await post("x".repeat(10_000), alice())).status).toBe(200);
77 expect((await pasteForm("x".repeat(10_001), alice())).status).toBe(422);
78 expect((await post(" ", alice())).status).toBe(422);
79 });
80
81 test("HTML in the pasted text is escaped in the prefilled form", async () => {
82 const page = await (
83 await pasteForm('Name: "><script>alert(1)</script>\nMessage: </textarea><img src=x onerror=alert(1)>', alice())
84 ).text();
85 expect(page).not.toContain("<script>alert(1)");
86 expect(page).not.toContain("<img src=x");
87 expect(page).toContain("&lt;script&gt;alert(1)&lt;/script&gt;");
88 });
89
90 test("anon is refused, and a cross-site post is refused", async () => {
91 expect((await post(ZILLOW, {})).status).toBe(401);
92 const cross = await post(ZILLOW, alice(), { origin: "https://evil.example", "sec-fetch-site": "cross-site" });
93 expect(cross.status).toBe(403);
94 });
95
96 test("the MCP tool returns the same fields, and anon is refused", async () => {
97 const call = (who: Record<string, string>) =>
98 fetch(`${base}/mcp`, {
99 method: "POST",
100 headers: { "content-type": "application/json", ...who },
101 body: JSON.stringify({ jsonrpc: "2.0", id: 1, method: "tools/call", params: { name: "parse_lead_email", arguments: { text: ZILLOW } } }),
102 }).then((r) => r.json() as Promise<any>);
103 const viaApi = await (await post(ZILLOW, alice())).json();
104 const mine = await call(alice());
105 expect(JSON.parse(mine.result.content[0].text)).toEqual(viaApi as object);
106 expect(JSON.stringify(await call({}))).toContain("signin_required");
107 });
108
109 test("/openapi.json documents it", async () => {
110 const doc = (await (await fetch(`${base}/openapi.json`, { headers: alice() })).json()) as { paths: Record<string, any> };
111 expect(doc.paths["/leads/parse"].post.summary).toContain("lead notification email");
112 });
src/readme.test.ts +7
@@ -1,5 +1,6 @@
11 import { expect, test } from "bun:test";
22 import { readFileSync } from "node:fs";
3 import { ZILLOW } from "./fixtures/lead-emails.ts";
34 import { SAMPLES } from "./sample.ts";
45
56 const readme = readFileSync(new URL("../README.md", import.meta.url), "utf8");
@@ -59,3 +60,9 @@ test("what the tour says about each lead is what the samples hold", () => {
5960 test("the tour has no jargon", () => {
6061 for (const word of ["API", "JSON", "SQLite", "MCP"]) expect(tour.includes(word), word).toBe(false);
6162 });
63
64 test("the tour shows the Zillow sample email, as the fixture holds it", () => {
65 const indented = ZILLOW.split("\n").map((l) => (l ? ` ${l}` : l)).join("\n");
66 expect(tour).toContain(indented);
67 expect(tour).toContain("Paste a lead email");
68 });
src/routes.ts +77 −31
@@ -25,6 +25,7 @@ import {
2525 listAudit,
2626 exportLeads,
2727 listLeads,
28 parseLeadEmail,
2829 resetSample,
2930 saveProfile,
3031 optOutTexts,
@@ -59,7 +60,8 @@ import {
5960 } from "./http.ts";
6061 import { MAX_NOTE, OUTCOMES } from "./calls.ts";
6162 import { MAX_ADDRESS, MAX_AHEAD_DAYS } from "./showings.ts";
62 import { MAX_INBOUND, MAX_QUERY, Q_HELP, VIEW_HELP, VIEWS } from "./leads.ts";
63 import { MAX_EMAIL_TEXT } from "./intake.ts";
64 import { MAX_INBOUND, MAX_QUERY, Q_HELP, VIEW_HELP, VIEWS, type NewLead } from "./leads.ts";
6365 import { events } from "./live.ts";
6466 import { rpc } from "./mcp.ts";
6567 import { deskPage } from "./pages/desk.ts";
@@ -452,41 +454,48 @@ const whole = (value: string | null): number | undefined =>
452454 const OPENAPI = new URL("../contract/openapi.json", import.meta.url);
453455 const openapi = existsSync(OPENAPI) ? readFileSync(OPENAPI, "utf8") : undefined;
454456
457 /** The desk page, with what a pasted email was read into when there is one. */
458 function desk(
459 c: Context,
460 pasted?: { found: Partial<NewLead>; status?: number; notice?: string },
461 ): Response {
462 const { notice, clear } = readNotice(c.request);
463 const show = c.url.searchParams.get("show");
464 let leads = listLeads(c.db, c.who, {
465 show,
466 q: c.url.searchParams.get("q"),
467 });
468 // A refused search still shows the desk, with the reason above it.
469 let refused: string | undefined;
470 if (!leads.ok && leads.error === "invalid") {
471 refused = leads.message;
472 leads = listLeads(c.db, c.who, { show });
473 }
474 return html(
475 deskPage({
476 who: c.who,
477 agent: currentAgent(c.db),
478 leads: leads.ok ? leads.value.leads : undefined,
479 record: leads.ok ? leads.value.record : undefined,
480 show: leads.ok ? leads.value.show : undefined,
481 q: leads.ok ? leads.value.q : undefined,
482 counts: leads.ok ? leads.value.counts : undefined,
483 mayAdd: permit(c.who, "lead", "create") === "yes",
484 mayReset: permit(c.who, "lead", "delete") === "yes",
485 notice: pasted?.notice ?? refused ?? notice,
486 prefill: pasted?.found,
487 }),
488 pasted?.status ?? 200,
489 clear,
490 );
491 }
492
455493 export const routes: readonly Route[] = [
456494 // The desk. Forms post and come back; the page stays live over /events.
457495 {
458496 method: "GET",
459497 path: "/",
460 handle: (c) => {
461 const { notice, clear } = readNotice(c.request);
462 const show = c.url.searchParams.get("show");
463 let leads = listLeads(c.db, c.who, {
464 show,
465 q: c.url.searchParams.get("q"),
466 });
467 // A refused search still shows the desk, with the reason above it.
468 let refused: string | undefined;
469 if (!leads.ok && leads.error === "invalid") {
470 refused = leads.message;
471 leads = listLeads(c.db, c.who, { show });
472 }
473 return html(
474 deskPage({
475 who: c.who,
476 agent: currentAgent(c.db),
477 leads: leads.ok ? leads.value.leads : undefined,
478 record: leads.ok ? leads.value.record : undefined,
479 show: leads.ok ? leads.value.show : undefined,
480 q: leads.ok ? leads.value.q : undefined,
481 counts: leads.ok ? leads.value.counts : undefined,
482 mayAdd: permit(c.who, "lead", "create") === "yes",
483 mayReset: permit(c.who, "lead", "delete") === "yes",
484 notice: refused ?? notice,
485 }),
486 200,
487 clear,
488 );
489 },
498 handle: (c) => desk(c),
490499 },
491500
492501 // Leads: the page for a browser and JSON for a script, at the same paths.
@@ -550,6 +559,43 @@ export const routes: readonly Route[] = [
550559 status: 201,
551560 },
552561 },
562 {
563 // Reads a pasted notification email into a new lead's fields and writes
564 // nothing. A script gets JSON; the desk's form gets the desk back with
565 // the add form filled in. Not under /leads/:id, so the door leaves the
566 // check to the action, which asks the lead-create cell.
567 method: "POST",
568 path: "/leads/parse",
569 handle: async (c) => {
570 const input = await readFields(c.request);
571 if (input === undefined)
572 return scripted(c.request) ? json({ error: "too_large" }, 413) : text("too large", 413);
573 const read = parseLeadEmail(c.who, input);
574 if (scripted(c.request)) return answer(read);
575 if (!read.ok) return desk(c, { found: {}, status: STATUS[read.error], notice: read.message });
576 return desk(c, { found: read.value });
577 },
578 api: {
579 summary:
580 "Read a pasted lead notification email (Zillow, Realtor.com, a website form) into a new lead's fields. Only what the email labels is returned, never consent. Nothing is written; add the lead with POST /leads once checked. Needs the same role as adding a lead.",
581 body: {
582 type: "object",
583 properties: { text: { type: "string", maxLength: MAX_EMAIL_TEXT } },
584 required: ["text"],
585 },
586 answer: {
587 type: "object",
588 properties: {
589 name: LEAD_FIELDS.name,
590 email: LEAD_FIELDS.email,
591 phone: { type: "string" },
592 source: LEAD_FIELDS.source,
593 message: LEAD_FIELDS.message,
594 },
595 description: "Only the fields the email held.",
596 },
597 },
598 },
553599 {
554600 method: "GET",
555601 path: "/leads/:id",
src/server.test.ts +1
@@ -286,6 +286,7 @@ test("the MCP server lists the tools and calls them as the caller", async () =>
286286 "export_leads",
287287 "get_lead",
288288 "add_lead",
289 "parse_lead_email",
289290 "new_sample_inquiry",
290291 "send_reply",
291292 "log_lead_reply",
src/terminal.test.ts +23
@@ -367,3 +367,26 @@ test("x on the desk shows the leads as CSV, for those who may list; esc goes bac
367367 guest.key("x");
368368 expect(read(guest)).not.toContain("as CSV");
369369 });
370
371 test("v reads a pasted email into the add screen, says what is missing, and adds with consent unticked", () => {
372 const db = fresh();
373 const t = terminal(db, alice);
374 t.key("v");
375 t.key("From: Zillow <[email protected]>\r\nName: Priya Raman\r\nEmail: [email protected]\r\nMessage: Is 12 Example St still available?\r\n");
376 t.key("\x04");
377 let screen = read(t);
378 expect(screen).toContain("Not found in the email: phone");
379 expect(screen).toContain("Name: Priya Raman█");
380 for (let i = 0; i < 6; i++) t.key("\r");
381 expect(read(t)).toContain("Priya Raman is on the desk");
382 const listed = listLeads(db, alice);
383 const added = listed.ok && listed.value.leads.find((l) => l.name === "Priya Raman");
384 expect(added && added.email).toBe("[email protected]");
385 expect(added && added.source).toBe("zillow");
386 expect(added && added.consent).toBe(false);
387 // Nothing readable says so, and esc writes nothing.
388 t.key("\x1b");
389 t.key("v");
390 t.key("\x04");
391 expect(read(t)).toContain("Paste the text of the email");
392 });
src/terminal.ts +52 −4
@@ -5,11 +5,12 @@
55 * may — the hints only leave out keys the table would refuse.
66 *
77 * Screens:
8 * desk j/k move · enter open · f view (needs/open/closed/all) · / find by name, email or phone · n log a lead · i new sample inquiry · p profile · a activity (admin) · R restart samples (admin) · esc quit
8 * desk j/k move · enter open · f view (needs/open/closed/all) · / find by name, email or phone · n log a lead · v paste a lead email · i new sample inquiry · p profile · a activity (admin) · R restart samples (admin) · esc quit
99 * lead s approve and send · e edit, then send · w they wrote back · ] next lead that needs you ·
1010 * q qualified · x lost · t asked to stop texts · D remove (admin) · esc back
1111 * edit type; enter is a new line · ctrl-d approve and send · esc cancel
1212 * wrote type what they wrote · ctrl-t text/email (text only with consent) · ctrl-d record it and draft the answer · esc cancel
13 * paste paste the notification email · ctrl-d read it, then check each field in the add screen (consent starts as n) · esc cancel
1314 * add one field at a time; enter next · esc cancel
1415 * profile the name, brokerage and license every message carries ·
1516 * e edit (admin): one field at a time, enter next, the last saves · esc back
@@ -42,8 +43,10 @@ import {
4243 cancelShowing,
4344 logLeadReply,
4445 nextLead,
46 parseLeadEmail,
4547 setState,
4648 } from "./actions.ts";
49 import { missingLine } from "./intake.ts";
4750 import { showingLine, showingWhen } from "./showings.ts";
4851 import { answers, callLine, OUTCOME_LABEL, OUTCOMES, type Outcome } from "./calls.ts";
4952 import { between, elapsed, recordLine, tone } from "./clock.ts";
@@ -129,7 +132,8 @@ type View =
129132 | { name: "showing"; id: number; step: 0 | 1; typed: string; when: string }
130133 | { name: "wrote"; id: number; text: string; channel: Channel }
131134 | { name: "details"; id: number; step: number; typed: string; fields: Record<string, string> }
132 | { name: "add"; step: number; typed: string; fields: Record<string, string> }
135 | { name: "paste"; text: string }
136 | { name: "add"; step: number; typed: string; fields: Record<string, string>; found?: Record<string, string>; note?: string }
133137 | { name: "confirm"; what: "remove" | "reset" | "opt-out"; id?: number; back: View };
134138
135139 /** The details the agent can correct, in the order asked. */
@@ -463,7 +467,8 @@ export function terminal(
463467 }
464468
465469 function addScreen(v: Extract<View, { name: "add" }>): string[] {
466 const lines = [bold("Log a lead by hand"), ""];
470 const lines = [bold(v.found === undefined ? "Log a lead by hand" : "Lead from the email"), ""];
471 if (v.note !== undefined) lines.push(dim(clean(v.note)), "");
467472 ADD_FIELDS.forEach((field, i) => {
468473 if (i < v.step) lines.push(`${field.label}: ${clean(v.fields[field.key] ?? "")}`);
469474 if (i === v.step) lines.push(`${field.label}: ${clean(v.typed)}█`);
@@ -472,6 +477,28 @@ export function terminal(
472477 return lines;
473478 }
474479
480 function pasteScreen(v: Extract<View, { name: "paste" }>): string[] {
481 const lines = [bold("Paste a lead email"), ""];
482 const shown = clean(v.text).split("\n").slice(-12);
483 lines.push(...shown.map((l, i) => (i === shown.length - 1 ? `${l}█` : l)));
484 lines.push("", dim("paste the notification email · ctrl-d read it · esc cancel"));
485 return lines;
486 }
487
488 /** What the email held, as the add screen's starting answers; consent is never one of them. */
489 function readEmail(text: string): void {
490 const read = parseLeadEmail(me, { text });
491 if (!read.ok) {
492 said = read.message;
493 view = { name: "desk" };
494 return;
495 }
496 const found: Record<string, string> = { consent: "n" };
497 for (const [key, value] of Object.entries(read.value))
498 if (typeof value === "string") found[key] = key === "source" ? String(SOURCES.indexOf(value as (typeof SOURCES)[number]) + 1) : value;
499 view = { name: "add", step: 0, typed: found.name ?? "", fields: {}, found, note: missingLine(read.value) };
500 }
501
475502 function send(id: number, body?: string): void {
476503 const zone = currentAgent(db).timeZone;
477504 const sent = approveReply(db, me, { id, body }, new Date(now()));
@@ -557,6 +584,8 @@ export function terminal(
557584 ]
558585 : view.name === "details"
559586 ? detailsScreen(view)
587 : view.name === "paste"
588 ? pasteScreen(view)
560589 : view.name === "add"
561590 ? addScreen(view)
562591 : [
@@ -590,6 +619,7 @@ export function terminal(
590619 show = VIEWS[(VIEWS.indexOf(show) + 1) % VIEWS.length]!;
591620 cursor = 0;
592621 } else if (input === "/") view = { name: "search", typed: q };
622 else if (input === "v") view = { name: "paste", text: "" };
593623 else if (input === "n")
594624 view = { name: "add", step: 0, typed: "", fields: {} };
595625 else if (input === "i") {
@@ -835,6 +865,19 @@ export function terminal(
835865 }
836866 return;
837867 }
868 case "paste": {
869 if (input === "\x1b") {
870 said = "Nothing read.";
871 view = { name: "desk" };
872 } else if (input === "\x04") readEmail(view.text);
873 else if (input === "\r") view = { ...view, text: `${view.text}\n` };
874 else {
875 // A paste arrives whole: keep its line breaks, drop other control keys.
876 const text = input.replace(/\r\n?/g, "\n").replace(/[\x00-\x09\x0b-\x1f\x7f]/g, "");
877 view = input === "\x7f" || input === "\b" ? { ...view, text: view.text.slice(0, -1) } : { ...view, text: view.text + text };
878 }
879 return;
880 }
838881 case "add": {
839882 if (input === "\x1b") {
840883 said = "Nothing logged.";
@@ -843,7 +886,12 @@ export function terminal(
843886 const field = ADD_FIELDS[view.step]!;
844887 const fields = { ...view.fields, [field.key]: view.typed.trim() };
845888 if (view.step + 1 < ADD_FIELDS.length)
846 view = { name: "add", step: view.step + 1, typed: "", fields };
889 view = {
890 ...view,
891 step: view.step + 1,
892 typed: view.found?.[ADD_FIELDS[view.step + 1]!.key] ?? "",
893 fields,
894 };
847895 else finishAdd(fields);
848896 } else {
849897 const next = typing(view.typed, input);

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs