For more crisp and insightful business and economic news, subscribe to The Daily Upside newsletter. It's completely free and we guarantee you'll learn something new every day.
Google may want to get rid of hotwords once and for all.
The company wants to patent a system for "hot-word free adaptation" of automated assistant functions. Rather than relying on a wake-up phrase such as "Hey Google," the company's system basically attempts to pick up on social cues to determine whether or not a user is addressing it, relying on lots of user activity monitoring to do so.
Here's how it works: First, this system periodically collects a whole bunch of sensor data, including: a user's gaze direction; distance from the device; mouth movement; voice activity; facial, speaker and presence recognition; body pose relative to the device; and gestures. If no user activity is detected, the system may stop collecting data and go dormant to save on computational resources.
Using this data, the system comes up with "occurrence and/or confidence metric(s)" that determine whether or not a user intends to interact with the device. These metrics will then control the system's functionality.
For example: If you look at the device and speak a command, such as "turn on the lights," the device will complete that request. But if you're having a conversation and happen to glance toward the device, it may say something along the lines of "Looks like you're talking to the Assistant, look away if you don't want to." In another example, gestures can indicate certain actions, such as a thumbs up to increase volume, or a "stop" gesture to stop an alarm or timer.
The metrics of occurrence and confidence are determined using neural network-based models, so they improve over time as they correctly (or incorrectly) determine that a user is, in fact, talking to the device.
The company argues in the filing that hotwords like "OK Google" are "unnatural prescribed phrases that are awkward to speak," and getting rid of them will cut down friction.
Google has long been working on ways to make its voice assistant technology better at listening. The company has sought plenty of patents related to speech recognition, including tech to enable natural conversations between users and their speakers, a system to correct what a device mishears, and a way to permit (or restrict) device access depending on who's talking.
Patents aside, Google also introduced a feature called Quick Phrases to certain devices, including Pixel smartphones and some voice assistant devices, which allows users to assign specific phrases to certain actions. For example, a user can command its device to "stop" or "snooze" when an alarm is going off, or can say "accept" or "decline" on an incoming call, without having to yell "Hey Google" beforehand. This filing, however, suggests expanding these capabilities far beyond a few fixed phrases.
Google's patent also brings up a quandary that comes up with almost all new consumer tech devices this day in age: How much privacy are users willing to give up for the promise of convenience?
In lieu of hotwords, Google's proposed system collects and uses a whole lot of user data, monitoring its owner's environment nearly constantly whenever they're in the room. The filing itself brings up one benefit of hotwords: "to preserve user privacy ... a user must often explicitly invoke an automated assistant before the automated assistant will fully process a spoken utterance."
Whether or not users will trust a system like this depends on consumer confidence in the devices and the tech companies themselves. And according to NPR's Smart Audio Report from last year, 47% of survey respondents were uncomfortable with the fact that their smart speakers were "always listening," and 52% were concerned that bad actors could gain access to these devices. Around 58% surveyed trusted the company that makes their smart speaker.
Another consideration: Google is not the only company looking to cut down hotwords. Apple recently announced plans to get rid of the "Hey" in "Hey Siri," and Amazon allows users to control their speakers with either "Hey Alexa" or just "Alexa." Getting to that point, though, was likely not an easy engineering feat, so whether or not Google's hotword-less technology will actually work is to be determined.
Have any comments, tips or suggestions? Drop us a line! Email at [email protected] or shoot us a DM on Twitter @patentdrop. If you want to get Patent Drop in your inbox, click here to subscribe.