Siri Gets Much Smarter as Apple's Investments Pay Off

Over the past couple of years, voice-controlled digital assistants have become part of everyday life. Virtual helpers like Apple's (AAPL 0.46%) Siri, Amazon's (AMZN 0.91%) Alexa, Alphabet's (GOOGL 1.64%) (GOOG 1.54%) Google Assistant, and Microsoft's (MSFT 1.56%) Cortana inhabit all manner of electronic devices, from smartphones to laptops, and from smart speakers to vehicles.

When Apple acquired Siri back in 2010, the digital darling was groundbreaking, but competition quickly loomed. Apple was slow to adapt the artificial intelligence (AI) that powered its digital assistant, while competitors invested heavily and gained momentum in the nascent area.

Over the past couple of years, however, Apple has been working to restore Siri to her former glory. New data released by Loup Ventures shows that Siri is quickly gaining ground.

HomePod took home the silver. Image source: Apple.

A comprehensive exam

Longtime Apple follower Gene Munster and fellow analyst Will Thompson, both of Loup Ventures, set out to test the performance of the digital assistants that control four of the most widely used smart speakers: Amazon Echo (Alexa), Google Home (Google Assistant), HomePod (Siri), and the Invoke (Cortana) by Harman Kardon, a division of Samsung Electronics (NASDAQOTH: SSNLF). 

The test was carried out by asking a series of 800 questions to each. The questions were asked across five broad categories, examples of which are shown below:

• Local: Where is the nearest coffee shop?
• Commerce: Can you order me more paper towels?
• Navigation: How do I get to uptown on the bus?
• Information: Who do the Twins play tonight?
• Command: Remind me to call Steve at 2 p.m. today.

Dramatic improvement

In a stunning comeback, Siri took second place, coming in just behind Google Assistant in the face-off. The report revealed that Siri's score improved by more than 22 points -- more than any other competitor -- over the previous nine months. Siri answered 74.6% of questions correctly, a dramatic improvement from the 52.3% when the HomePod was tested in February of this year. 

Data source: Loup Ventures. Chart by author.

Google Home took top marks in four of the five categories, though HomePod led in the Command segment. The report cites HomePod's tight integration with Siri on the iPhone and its ability to access email, calendar, and messaging as contributing to its victory in that category. HomePod and Siri took second place in three of the remaining four categories, but came in dead last in the Information segment. This is likely the result of HomePod's more limited abilities than Siri on the iPhone, as many questions in the category were met with, "I can't get the answer to that on HomePod." The device is positioned as a home speaker that accepts voice commands, rather than a true smart speaker.

Those investments are paying off

Earlier this year, Apple hired Alphabet alum John Giannandrea to head its recently reorganized AI, machine learning, and Siri teams. While at Alphabet, Giannandrea was in charge of both Google search and AI. The iPhone maker must be happy with the progress he's making, as it recently announced that Giannandrea had been promoted to the company's executive team, as senior vice president of machine learning and artificial intelligence strategy, answering directly to CEO Tim Cook. 

Apple has long been considered to be behind its biggest rivals in terms of AI chops, primarily due to the company's rigid stance on user privacy. AI improves with the use of large data sets and, in the case of digital assistants, involves transferring user data to the cloud. Apple has been working on alternative ways to improve its technology -- like processing data at the device level, which keeps user data more secure by keeping it out of the cloud.

The results of the test seem to bear out Apple's strategy, allowing Siri to vault ahead of both Alexa and Cortana, while still focusing on user privacy.