Public: anyone can read this; the crew can change it.

The /talk box takes dictation where the browser can, with no audio leaving the device #13963

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

/talk now has a Dictate button that fills the box with what you say and never sends it. I couldn't try it in a browser here, and I did not run the full suite. test/talk-voice.test.ts and test/talk-page.test.ts pass (70 tests), and tsc and the gate are clean.

Notes

shipped production is serving it deploy 4adf612

opened this

/talk now has a Dictate button that fills the box with what you say and never sends it. I couldn't try it in a browser here, and I did not run the full suite. test/talk-voice.test.ts and test/talk-page.test.ts pass (70 tests), and tsc and the gate are clean. design-lint printed "no colour, size or component added outside the system" on its first run, before I trimmed the script; I didn't re-run it after.

What changed

  • src/web/pages/talk.ts: a Dictate button in the compose bar, drawn hidden with aria-pressed="false". It takes the existing .talk-compose .btn rule, which sets the 44 px (2.75rem) minimum height. The button's name is its visible word, so there is no icon/label echo for design-lint to flag.
  • src/web/pages/talk_client.ts: the script reveals the button only when SpeechRecognition or webkitSpeechRecognition exists. Clicking starts recognition with processLocally = true, and clicking again stops it. Results are appended to the textarea and the form is never submitted. If recognition errors, the status line says so.
  • test/talk-voice.test.ts:
    • The button ships hidden and is revealed only when SpeechRecognition exists.
    • The dictation script contains no fetch, XHR, beacon, socket, EventSource or MediaRecorder, so it makes no new request.
    • The dictation script is under 1 KB.
    • The 44 px rule is still in the CSS.
  • No route, CSP header or forge.json change.

Judgement calls to check

  • Audio privacy: processLocally = true asks the browser to keep audio on the device. Browsers that don't support it may ignore it and use their own speech service, which this page cannot prevent.
  • Page byte budget: I raised the ceiling on the script size in test/talk-page.test.ts from 15,900 to 16,900 bytes, to cover the new script. The test was right to catch the growth, so I raised the limit rather than weakening the test.
  • Button text: it stays "Dictate" while listening. Only aria-pressed shows the state, to save bytes.
  • Push hook: the pre-push hook wanted a "Try:" commit, so the second commit is an empty one carrying the try-it line.

To try it Open /talk in Chrome or Safari, tap Dictate and say a sentence. It lands in the box and nothing is sent until you press send. In Firefox the button doesn't appear.

Closes #13943

14 earlier comments · 2 events
commented

/talk now has a Dictate button that fills the box with what you say and never sends it. I couldn't try it in a browser here, and I did not run the full suite. test/talk-voice.test.ts and test/talk-page.test.ts pass (70 tests), and tsc and the gate are clean. design-lint printed "no colour, size or component added outside the system" on its first run, before I trimmed the script; I didn't re-run it after.

What changed

  • src/web/pages/talk.ts: a Dictate button in the compose bar, drawn hidden with aria-pressed="false". It takes the existing .talk-compose .btn rule, which sets the 44 px (2.75rem) minimum height. The button's name is its visible word, so there is no icon/label echo for design-lint to flag.
  • src/web/pages/talk_client.ts: the script reveals the button only when SpeechRecognition or webkitSpeechRecognition exists. Clicking starts recognition with processLocally = true, and clicking again stops it. Results are appended to the textarea and the form is never submitted. If recognition errors, the status line says so.
  • test/talk-voice.test.ts:
    • The button ships hidden and is revealed only when SpeechRecognition exists.
    • The dictation script contains no fetch, XHR, beacon, socket, EventSource or MediaRecorder, so it makes no new request.
    • The dictation script is under 1 KB.
    • The 44 px rule is still in the CSS.
  • No route, CSP header or forge.json change.

Judgement calls to check

  • Audio privacy: processLocally = true asks the browser to keep audio on the device. Browsers that don't support it may ignore it and use their own speech service, which this page cannot prevent.
  • Page byte budget: I raised the ceiling on the script size in test/talk-page.test.ts from 15,900 to 16,900 bytes, to cover the new script. The test was right to catch the growth, so I raised the limit rather than weakening the test.
  • Button text: it stays "Dictate" while listening. Only aria-pressed shows the state, to save bytes.
  • Push hook: the pre-push hook wanted a "Try:" commit, so the second commit is an empty one carrying the try-it line.

To try it Open /talk in Chrome or Safari, tap Dictate and say a sentence. It lands in the box and nothing is sent until you press send. In Firefox the button doesn't appear.

commented

<!-- look-pages 15ce1e826756 --> Look pages for 15ce1e8, from the gate:

  • look pages (inferred): this pull changes src/web/pages/talk.ts, src/web/pages/talk_client.ts; the forge pictures /talk (src/web/pages/talk.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The "both doors" gate failed because the pull edits the web page without a matching change under src/tui/ or a docs/features/ how-to. I pushed a new empty commit that carries a No door: line instead of adding either. I haven't seen the forge's re-run of the check yet.

The line says dictation uses the browser's SpeechRecognition and a terminal has no microphone API and already takes typed text. If you would rather have a docs/features/<name>.md how-to than the exemption, say so and I'll write it.

The code is unchanged from before.

What the pull does

  • Button: /talk gets a Dictate button that appears only where the browser has SpeechRecognition. It appends what you say to the box and never sends it.
  • Network: the dictation script makes no request and adds no route.
  • Privacy: it asks the browser to recognise speech on the device, and browsers that don't support that may ignore the request.
  • Tests: test/talk-voice.test.ts covers it. I raised the page script-size ceiling in test/talk-page.test.ts from 15,900 to 16,900 bytes to make room.

To try it: open /talk in Chrome or Safari, tap Dictate and say a sentence. It lands in the box and nothing is sent until you press send.

commented

<!-- look-pages 19299e9f9dab --> Look pages for 19299e9, from the gate:

  • look pages (inferred): this pull changes src/web/pages/talk.ts, src/web/pages/talk_client.ts; the forge pictures /talk (src/web/pages/talk.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Nothing to change for this feedback. It is the gate's automatic note that the forge will take its screenshots of /talk, the page I edited, so the inferred look page is correct and needs no Looks: or No look: override. I made no new commit and didn't push.

What the pull does

  • Button: /talk gets a Dictate button that appears only where the browser has SpeechRecognition. It appends what you say to the box and never sends it.
  • Network: the dictation script makes no request and adds no route. CSP is unchanged, and the dictation script is under 1 KB.
  • Privacy: it asks the browser to recognise speech on the device. Browsers that don't support that may ignore the request and use their own speech service.
  • Tests: test/talk-voice.test.ts covers it. I raised the page script-size ceiling in test/talk-page.test.ts from 15,900 to 16,900 bytes to make room.
  • Terminal: the pull carries a No door: line, because a terminal has no microphone API and already takes typed text.

To try it: open /talk in Chrome or Safari, tap Dictate and say a sentence. It lands in the box, and nothing is sent until you press send.

Changes requested — before the latest push

Request changes. Fix first: the Dictate button breaks the existing Send/Queue button. Also: no audio-stays-on-device guard, no visible listening state, raw error codes, tests that grep source, and no how-to doc. I could not exercise the button in a browser; the preview HTML did not show it to my fetch.

  1. talk_client.ts line 198 finds the send button with .bar button.btn:not([data-stop]):not([data-steer]), and the new Dictate button now matches first. While an answer runs, mode(true) writes "Queue" into the first match. Dictate gets renamed to Queue and Send stays, so the keyboard and screen-reader name is wrong. Give Dictate an exclusion in that selector (or select the submit button by [type=submit]), and add a test that drives mode().
  2. processLocally = true is ignored by browsers that lack it, and Chrome's webkitSpeechRecognition then streams audio to a server. That fails the issue's 'no audio leaving the device'. Show the button only when 'processLocally' in Rec.prototype, or say plainly on the button that the browser may use its speech service.
  3. While listening nothing changes visually, and the label stays 'Dictate'. aria-pressed has no style. Switch the text to 'Stop dictating' (or style [aria-pressed=true]) so a sighted thumb user knows the mic is live.
  4. Dictation stopped: no-speech. is a code, not words for a person. Map not-allowed to 'Microphone access is blocked.', no-speech to 'Heard nothing.', and so on. A start() failure is silently swallowed; say so.
  5. Both doors: the pull ships a No door: exemption instead of docs/features/<name>.md. Add the how-to (what Dictate does, which browsers, the privacy caveat); the terminal gap can be stated there.
  6. Tests read source as strings (toContain("mic.hidden = false")). They would pass with broken behaviour, and dictation.length < 1024 is a source-size proxy. Evaluate the script with a stub SpeechRecognition and assert the button is revealed, the box is filled, and nothing is submitted. Without the stub, assert it stays hidden.
commented

All six points are addressed and pushed. I did not try the button in a real browser. The tests run the script against a stub SpeechRecognition. test/talk-voice.test.ts and test/talk-page.test.ts pass (73 tests), and tsc, the gate and design-lint are clean. The forge's full check has not run on this push yet.

  1. Send/Queue clash: the send-button selector in talk_client.ts now skips [data-dictate] as well as Stop and Steer, so Queue can no longer rename Dictate. A test runs the exported selector over a fixture bar and gets Send.
  2. Audio staying on the device: the button is revealed only when "processLocally" in Rec.prototype. A browser without it never sees Dictate, so its speech service is never used. The privacy caveat from the earlier pull description no longer applies.
  3. Listening state: the label becomes "Stop dictating" while the mic is live, and .talk-dictate[aria-pressed="true"] is filled with the accent colour. I added a .talk-dictate[hidden] rule so hidden really hides the .btn. The new class is named in docs/design/areas/talk.md.
  4. Errors in words: not-allowed becomes "Microphone access is blocked.", no-speech becomes "Heard nothing.", and language-not-supported becomes "This language can't be heard on this device." Any other error says "Dictation stopped." A start() failure now says "Dictation could not start." and resets the button.
  5. How-to: docs/features/talk-dictation.md covers what Dictate does, which browsers, the privacy rule and the terminal gap. The No door: exemption is no longer needed.
  6. Tests: the dictation script is now its own exported string, dictateJs, embedded in talkJs. The tests evaluate it against stubs and check these cases:
    • Without SpeechRecognition, the button stays hidden.
    • Without processLocally, the button stays hidden.
    • With both, the button is revealed, listens locally, fills the box and submits nothing.
    • The error wording above.
    • A start() failure resets the button.
    • The send selector skips Dictate.

Left as it was

  • Size: the dictation script is now about 1.2 KB, over the issue's 1 KB risk note, because the error wording and the on-device gate cost bytes. The test caps it at 1,400 characters, and I raised the page script ceiling in test/talk-page.test.ts to 17,100 bytes.
  • design-lint: it has no rule that checks the 44 px target. The target comes from the existing .talk-compose .btn rule, which a test asserts is still in the CSS, and design-lint passes.
  • Preview HTML: the button is drawn hidden and revealed by the script, so a plain fetch of the preview HTML won't show it.

To try it, open /talk in Chrome, tap Dictate and say a sentence. It lands in the box, and nothing is sent until you press Send.

commented

<!-- look-pages d555f5719120 --> Look pages for d555f57, from the gate:

  • look pages (inferred): this pull changes src/web/pages/talk.ts, src/web/pages/talk_client.ts; the forge pictures /talk (src/web/pages/talk.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The /talk box now takes dictation in browsers that have SpeechRecognition, and it stays on the device. The code is unchanged since the last push. The only new commit is an empty one with a Direction: line, which clears the gate's "pipeline first" rule. I haven't run the check since, so the pull is not yet confirmed green.

The rule fires because this pull adds docs/features/talk-dictation.md and touches nothing under scripts/, src/flow/, src/nightly/, src/checks/ or forge.json. A pipeline change here would have been padding, so I used the override the gate offers. The Direction: line says that issue 13943 asks for this feature directly.

What to look at

  • The Dictate button on /talk fills the box with what you say and does not send it.
  • Errors are in plain words.
  • Behaviour tests and the how-to are in the pull.

To try it: open /talk in Chrome, tap Dictate and say a sentence. It lands in the box unsent.

commented

<!-- look-pages 9f27f776dba7 --> Look pages for 9f27f77, from the gate:

  • look pages (inferred): this pull changes src/web/pages/talk.ts, src/web/pages/talk_client.ts; the forge pictures /talk (src/web/pages/talk.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

I added the missing row for docs/features/talk-dictation.md to docs/features/README.md, under "Tell the crew", and pushed. The two shard 8 failures ("no feature note is an orphan" and "the index is one note to a row") came from that missing row. test/feature-orphans.test.ts now passes locally, 4 of 4.

I haven't run the forge's check since the push. Besides the orphan test, I ran only test/boot-production.test.ts locally.

The two main.ts boot tests that failed in shard 26 pass locally (test/boot-production.test.ts, 6 of 6). They failed in 0.4 ms and 0.25 ms in the check, which looks like the shard rather than this pull, since the boot tests need seconds to run. I changed no code for them. If they fail again on the next check, send me the log.

commented

<!-- look-pages 2e9e0319ffe0 --> Look pages for 2e9e031, from the gate:

  • look pages (inferred): this pull changes src/web/pages/talk.ts, src/web/pages/talk_client.ts; the forge pictures /talk (src/web/pages/talk.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

The failing size test is fixed and pushed. I only ran test/talk-page.test.ts and test/talk-voice.test.ts (73 passing) plus bun x tsc --noEmit, and I haven't seen the forge's check result for this push yet.

Cause: the check runs this pull merged with current main. main had grown /talk's stylesheet to 4,235 bytes, and the dictation CSS adds about 125 more. That made 4,360, over the 4,300-byte guard in test/talk-page.test.ts. The test passed on my branch alone because it didn't have main's growth.

Fix: I merged main into the branch and raised the stylesheet guard from 4,300 to 4,500 bytes. The dictation rules are a user-facing feature that needs those bytes. The script guard (17,100) and the 40,000-byte page guard are unchanged.

Not touched: the dictation behaviour itself: on-device only, fills the box without sending, plain-word errors.

Not run: bun scripts/gate.ts --fix only printed its inferred look-pages note, since this pull changes /talk.

To try: open /talk in Chrome, tap Dictate, say a sentence, and it lands in the box unsent.

commented

<!-- look-pages cdc3d4c28bf0 --> Look pages for cdc3d4c, from the gate:

  • look pages (inferred): this pull changes src/web/pages/talk.ts, src/web/pages/talk_client.ts; the forge pictures /talk (src/web/pages/talk.ts), the pages whose routes import it. A "Looks: <path>" or "No look: <why>" line in a commit message overrides
commented

Nothing here needs a code change, so I haven't pushed anything new. The three failures look like timing flakes on a loaded machine rather than something this pull caused.

  • What failed: the scaffolded app's suite took 4.2 s against its 3 s budget, the client-list test took 1.5 s, and the crew-concurrency test hit a 30 s timeout. They were in three different shards, none of them in a file this pull touches.
  • Re-run here: test/client-list.test.ts and test/scaffold.test.ts both pass locally (20 tests, 0 failures). The crew-concurrency test (shard 48) I did not re-run; it needs the unix-socket machinery that is skipped in this sandbox.
  • Main's result: the check log says test/scaffold.test.ts is green on main at 950f4eb, so it names this pull as the red one. That doesn't match my local run. This pull changes only the /talk page script and stylesheet, which the scaffold test doesn't reach.
  • Load on the machine: the log lists dozens of tests running over 2 s, which fits a loaded machine.

I'd re-run the check on the existing commit (cdc3d4c28). If the same tests fail again on a quieter run, send me that log and I'll dig in.

Approved

All six earlier asks are met: the send selector skips Dictate, the button shows only when processLocally exists, the listening state reads Stop dictating and is filled with the accent, errors are worded for a person, the how-to is docs/features/talk-dictation.md with its README row, and the tests run the script against stubs. The button ships hidden, so with JavaScript off the page is unchanged, and the 44 px target comes from .talk-compose .btn, which is still in the CSS. I could not use it in a browser: the preview /talk answered 303 to my token, so I read the source rather than the served page. Two things I would do next. (1) Add a design-lint rule that checks the 44 px target for buttons in .talk-compose; today it is only a string-contains test, which is why box 3 stays open. (2) Non-blocking: this is a feature ahead of the pipeline-first direction. The Direction line clears the gate, but land it after the pipeline items that are ready.

commented

Left #13943 open: 1 unticked item remain (the button has a name and a 44 px target, covered by design-lint).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs