The test is simple: Put a machine in one room, a human interrogator in another, and have them talk to each other through a text-based conversation. If the interrogator can identify the machine as nonhuman, the device fails; otherwise, it passes.
The simple and intuitive test has become hugely influential in the philosophy of artificial intelligence. But from the beginning, researchers found flaws in the test. For one, the game focuses on deception and is overly focused on conversation as the metric of intelligence.
For instance, in the 1970s, an early language-processing program called ELIZA gave Turing test judges a run for their money by imitating a psychiatrist's trick of reflecting questions back to the questioner. And in 2014, researchers fooled a human interrogator using a "chatbot" named Eugene Goostman that was designed to pose as a 13-year-old Ukrainian boy.
Right to remain silent
Warwick was organizing Turing tests for the 60th anniversary of Turing's death when he and his colleague Huma Shah, also a computer scientist at Coventry University, noticed something curious: Occasionally, some of the AI chatbots broke and remained silent, confusing the interrogators.
RELATED: See the Dreams of an Artificial Brain: Photos
"When they did so, the judge, on every occasion, was not able to say it was a machine," Warwick told Live Science.
By the rules of the test, if the judge can't definitively identify the machine, then the machine passes the test. By this measure then, a silent bot or even a rock could pass the Turing test, Warwick said.
On the flip side, many humans get unfairly tarred as AI, Warwick said.
"Very often, humans do get classified as being a machine, because some humans say silly things," Warwick said. In that scenario, if the machine competitor simply stayed silent, it would win by default, he added.
The findings point to the need for an alternative to the Turing test, said Hector Levesque, an emeritus computer science professor at the University of Toronto in Canada, who was not involved with the new research.
"Most people recognize that, really, it's a test to see if you can fool an interrogator," Levesque told Live Science. "It's not too surprising that there are different ways of fooling interrogators that don't have much to do with AI or intelligence."
Levesque has developed an alternative test, which he dubbed the Winograd schema (named after computer science researcher Terry Winograd, who first came up with some of the questions involved in the test).
RELATED: 'Ex Machina': Science vs. Fiction
The Winograd schema asks AI a series of questions that have clearly correct answers. For example, it might ask, "The trophy would not fit in the brown suitcase because it was too big (small). What was too big (small)?"
These queries are a far cry from the rich discussions of Shakespearean sonnets that Turing envisioned taking place between AI and humans.
"They're mundane and certainly nowhere near as flashy as having a real conversation with somebody," Levesque said.
Yet, answering correctly requires an understanding of language, spatial reasoning, and context to figure out that trophies fit in suitcases.
And still other proposed alternatives to the Turing Test have focused on different aspects of human intelligence, such as creativity.
The Lovelace test to measure creativity requires a robot to create a piece of artistic work in a particular genre that meets the constraints given by a human judge. But even in this domain, robots are gaining on mere mortals: Earlier this year, researchers created a "new Rembrandt" painting in the style of the Dutch master, using artificial intelligence and robot painters.
Original article on Live Science.
Copyright 2016 LiveScience, a Purch company. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.