For more crisp and insightful business and economic news, subscribe to The Daily Upside newsletter. It's completely free and we guarantee you'll learn something new every day.
Apple has promised lots of new features in its upcoming Vision Pro mixed-reality headset. According to a recent patent, the headset may make it easier to have conversations with real-life people.
The company is seeking to patent a digital assistant for "providing real-time social intelligence." Apple's system can determine whether a user is engaged in conversation by monitoring user activity and accommodating for "real-time social engagements of the user with other human(s)," according to the filing. Basically, it makes a user focus on the person in front of them instead of playing AR Fruit Ninja.
First, using a machine learning model, this system determines whether a user is in an "engaged state," meaning that they're talking to another IRL human being, based on at least one of a few factors: detecting another person in the near-field of vision of the user, determining that a user is looking at that person, and detecting that said person is talking to the user (or that the user is talking to said person).
In response to detecting this real-life conversation, the headset stops providing most, if not all, output to the user for the duration of that interaction. To determine if an interaction meets the conditions for pausing output, the system comes up with an "engagement score" based on the above factors. For example, if someone passes by and says "hello" while you're playing a game in the headset, but you don't react too much, the system may not pause your game.
Apple said this system provides a user a "hands-free" and "touchless" way to interact with other people socially while they are using a headset, as well as halts any potential interruptions while they're in conversation. Apple said this system "makes the process of managing a user's interactions with the digital assistant and other humans simple and efficient by reducing a user's time in enabling and disabling communications."
Apple's tech applies close-up user monitoring to cease any content that may be playing on a headset to allow for a conversation, accomplishing the same outcome as just taking the headset off completely. What's the point of this? It's simple: Apple doesn't want you to take the headset off, so they're alleviating the isolation that a user might feel when wearing one.
"They want the headset to be part of you, like a hat, like sunglasses, like something that's just part of your accessories," said Jake Maymar, VP of Innovation at The Glimpse Group. "If it just interrupts you, you're immediately going to notice it. It becomes an annoyance rather than something that's helping you manage your day to day."
It seems improbable that anyone wouldn't be distracted by a conversation with another human while they wear a massive headset with eyes projected on the front. But that sentiment may change as extended reality embeds itself further into the cultural zeitgeist, said Maymar. And as these headsets get smaller and give way to smart glasses, rather than bulky face-worn iPads, he said, it may become less distracting.
A patent like this is pretty on-brand for Apple, Maymar noted. The company is well-known for creating sleek experiences that are deceptively simple to use when considering how much engineering goes into them. But to achieve this level of seamlessness, the company's devices are collecting scores of user data at a near-constant pitch.
But unlike other Big Tech firms, Apple has managed to gain users' trust to the point where they're willing to hand over even the most sensitive of personal data. The company hasn't had a major public data breach, Maymar said, and provides enough value that users are willing to eschew data privacy concerns that they might have with other tech companies.
To put it simply, Maymar said, "Apple sells trust. Security is in their DNA." That trust allows the company to push the boundaries on how close it can get to its users. So far, at least, it hasn't overstepped.
One other thing to consider: While this patent only outlines how this kind of tracking could be used to accommodate for person-to-person interactions, what isn't said is how this monitoring could benefit the company in other ways, Maymar noted, like advertising or in-experience user engagement tracking.
