Which Is Smarter: Google Home or Amazon Echo?
These so-called "voice-activated knowledge devices" use cloud-based AI to provide answers. Here's how they did with a set of 5,000 questions.
For decades, sci-fi movies have promised us intelligent computers that respond to natural-language commands and can even hold a conversation. With Google Home and Amazon Echo, which answers to the name of Alexa, some say the future has officially arrived.
These so-called "voice-activated knowledge devices" use cloud-based AI to serve up answers to your every question. But are they equally smart?
Digital marketing firm Stone Temple Consulting thought it was time for a performance test, posing 5,000 questions to both Amazon Echo and Google Home to determine how many each device could answer correctly.
"It's very important to note that our test is NOT a test of all the capabilities of both devices," the company stressed. "Our test focused on the ability to provide direct answers to specific user questions. In short, we're only testing the ability of each device to provide factual answers to questions."
In terms of accurate knowledge, the contest showed that Google Home was the clear winner.
Stone Temple first counted the number of questions that each device attempted to answer. This is a strong indicator of the device's ability to recognize natural-language queries. In that head-to-head showdown, Google owned the category. Google Home attempted to answer 68 percent of the 5,000 total questions, while Amazon Echo hazarded a guess for only 21 percent.
Both Google Home and Amazon Echo get their brain power from the cloud and must have a wifi internet connection to work. Google Home is powered Google Assistant, the latest iteration of Google's consumer-focused AI. As the test results show, Google's language-processing technology is a few steps ahead of Amazon Web Services, the cloud-based AI behind the Echo.
Language-recognition aside, which device was better at retrieving the correct answer?
This category was basically a tie, with Google Home delivering a correct answer 90 percent of the time and Amazon Echo getting it right 87 percent of the time.
It's important to note, however, that those results were only for the questions each device attempted to answer. Google Home correctly answered 90 percent of the 3,400 questions it tried to answer for a total of 3,060 right answers. Amazon Echo correctly answered 87 percent of the 1,050 questions it attempted to answer, for a much lower total of 914 right answers.
Then there's how often an answer from one of the devices was determined to be incorrect. Amazon Echo came out ahead (or behind, as it were) on that score.
In the write-up of its findings, Stone Temple mentions that the majority of Google Home's correct answers were gleaned from third-party websites ("According to Wikipedia...") rather than Google's own knowledge base called the Knowledge Graph. Again, Google's search supremacy gives it an edge over Amazon, at least for now.
Stone Temple was quick to note that "knowledge" is only one of the services offered by these voice-activated devices - others include music streaming, traffic reports, and smart home control - and that Amazon Echo might still be the right device for many consumers.
It's also worth noting that the test did not gauge a relative grasp of specialized knowledge. For example, Amazon recently teamed with the American Heart Association to program Alexa to quickly provide potentially life-saving instructions for CPR and as well as a list of signs of cardiac arrest and stroke.
That could be the start of a growing healthcare-driven orientation that could give Amazon an edge in answering the questions that matter most when time is critical.