Skip to content

Research · November 8, 2026

Voice and computer use, with the person in control

We compared open tools for voice and for browser and computer control, measured how much of the app voice can reach, and wrote down why a spoken yes must never count as an approval.

Voice and computer control have the same risk in common: the assistant acts for you, so the question is who decides. This note combines a study of open tools, a small speech measurement and a coverage count, and it records one rule we are firm about.

Tools we compared

The voice study compared open dictation and speech projects against one rule: only permissive licences (MIT, Apache, BSD, ISC) in what ships, copyleft only as a separate program the person installs, and nothing non-commercial. Its verdict on the project it examined first was to learn from it and not to embed it. The reasons were concrete: it is written for one operating system, its default engine has no Turkish, and the project it was forked from had no licence file when checked.

The agent study applied the same licence rule to browser and computer control. It listed Playwright MCP, Browser Use and Cua as candidates, with their licences read from the repositories on the research date. Licence and fit were the criteria, not popularity.

The licence rule is not a legal opinion. It is the rule we applied, and the file shows each licence it read.

One small measurement

The voice study ran a local speech engine on a Mac. On a six-second Turkish clip the smaller model took 0.57 seconds with no word errors, and on an English clip 0.43 seconds with a 6 percent error rate, where the only mistake was writing "three" as a digit. The study warns that the clips were synthetic voices, so real voices will do worse. We treat these numbers as a starting check that the engine runs, not as a quality claim.

How much of the app voice reaches

A generated report from scripts/conductor/coverage-report.mjs counts what voice can reach. It reads the real registry and nothing in it is typed by hand. For the page registry, 34 of 71 route heads are reachable by voice. For settings, 159 of 197 rows are. Most of the settings rows are reached by opening the row and explaining why a tap does the change, not by changing it silently: only a few rows are changed by voice with an undo or a read-back.

The report lists the gaps one by one, and its own verdict is "gaps listed". It does not say complete, and neither do we.

The rule we are firm about

The recommended design in the voice study says speech is never an approval. The assistant may propose an outward action. A card shows what will happen and where, and the decision is a physical click or Enter on that card. The reason given is plain: a recording, a video call, a television or another person in the room can say "approve". Voice is not authentication.

The same attitude applies to computer use. Agents can search the web and use the computer through browser and system tools, and each step that reaches outside waits for your approval. Push-to-talk works by holding the shortcut while speaking and letting go to send. See the voice control entry, the computer use entry and the phone approval entry.

What is not decided

The study lists what it did not verify: speech accuracy with real Turkish voices for any engine, latency of other projects, the exact keyword-spotting model licences, and how to block network access for the engine on Windows and Linux. A wake word was left out of the first version because the available pretrained models are non-commercial or need a key, and an always-listening microphone needs its own privacy design.

Where this comes from

  • docs/research/voice/VOICE_RESEARCH.md, accessed 2026-10-30; the speech numbers come from its measured-baseline section.
  • docs/research/agentic/AGENTIC_OSS_2026-10-30.md, licences read on 2026-10-30.
  • docs/research/measurements/s5-r16-coverage.md, generated 2026-11-08.

Sources