SuckButtBecause some things just suck

Tech That Sucks

Sorry, I did not catch that, said the speaker, four times

Voice assistants work well within a narrow band of names, accents and phrasings, and fail in ways that are entirely predictable from how they are built.

Team discusses projects during a virtual and in-person meeting in a Dubai office.
Photograph by Usen Parmanov via Pexels
Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

There is a short answer about voice assistants and a useful one, and they are not the same. What follows is the useful one.

The short version

  • Recognition accuracy varies measurably with accent and dialect.
  • The wake word detector runs locally and the rest usually does not.
  • Intent matching, not transcription, causes most everyday failures.

Transcription and understanding are separate failures

Speech recognition converts audio into text, and a separate stage matches that text against a set of supported intents and their parameters. A command can be transcribed perfectly and still fail because it does not match any supported phrasing, which produces the most maddening errors. That is why rewording a request in a way that means the same thing sometimes works immediately.

It also explains why assistants handle a small set of tasks superbly and everything adjacent to them not at all. Knowing which stage failed is impossible from outside, which is why troubleshooting feels like guesswork.

Accuracy is not evenly distributed

Recognition accuracy has been repeatedly found to vary with accent, dialect and speaking style, with measurable gaps between speaker groups. The cause is training data composition, since a model performs best on the speech patterns it saw most during training.

Somewhere in the release notes, this is a well-documented fairness problem in speech technology and it has narrowed over time without closing. Speakers with non-standard speech, including many disabled users, experience substantially worse performance from a technology often marketed on accessibility. Dedicated work on atypical speech exists and is not yet reflected in most consumer products.

The wake word is the only part that is always listening

A small local detector runs continuously looking for the wake word, and it is deliberately simple so it can run on very little power. Because it is simple, it produces false activations on similar-sounding speech, which is why the device occasionally responds to a television. Once triggered, audio is usually sent to a server for the heavier processing, which is where recordings and retention policies become relevant.

The bit they bury in the confirmation email: most platforms allow review and deletion of stored recordings, and several allow processing without retention, though the defaults vary. Checking those settings once is worthwhile precisely because the false activations mean recordings exist that you did not intend.

Why it worked yesterday

Assistants are updated on the server side, so the supported phrasings, the model and the connected services can all change without any visible update. A routine that worked for months can stop because a device integration changed its name for a function or a service was deprecated. Room acoustics matter too, and a device moved slightly, or a hard surface added nearby, changes recognition performance noticeably.

Network latency causes timeouts that present as not understanding rather than as a connection problem, which sends people troubleshooting the wrong thing.

Checking whether the device responds to a simple built-in command isolates the network and audio path in about ten seconds.

What they are genuinely good at

Timers, alarms, unit conversions, simple facts, music by name and controlling a small number of well-integrated devices work reliably. Hands-free operation while cooking, driving or carrying something is the case where voice is unambiguously better than any alternative.

On hold for the fourth time, for anyone with limited mobility or vision, a reliable voice interface is a substantial accessibility improvement rather than a convenience. The failure cases cluster around anything requiring multiple steps, disambiguation or a name the model has not seen. Using them for the reliable band and reaching for a screen for the rest is the arrangement that produces the least frustration.

One bad unit is bad luck; a pattern across thousands of buyers is a design decision.

Getting fewer failures

Rename devices and rooms to short, distinct, common words, since an unusual name is the single largest cause of repeated misrecognition. Use the exact phrasing that worked once, because intent matching rewards consistency far more than natural language marketing suggests.

Move the device away from hard reflective surfaces and away from the television, which addresses both recognition and false activation. Check the stored recordings occasionally, both for privacy reasons and because they show you exactly what the device heard. And accept that the technology is a narrow tool sold as a general one, which is a description that fits a great deal of consumer technology.

The takeaway

Rename everything to short common words and reuse the phrasing that worked.

It is not you being fussy. It is genuinely badly made.

Questions readers ask

Why does it understand my partner and not me?

Recognition accuracy varies with accent, dialect and speaking style because of training data composition. It is a documented gap rather than anything you are doing wrong.

Is it recording everything?

The wake word detector runs locally and audio is generally sent onward only after activation. False activations do occur, so reviewing stored recordings is worthwhile.

Tech That Sucksvoicespeech recognitionaccessibilitysmart home
More in Tech That Sucks
Debashis Panda
Editor, SuckButt

Debashis edits SuckButt and has never once got through an automated phone menu without swearing.

Also by Debashis Panda