Is the AI trying to fool the humans? Can it lie with intent to deceive?

Question: What exactly constitutes "human" by your definition. The entire purpose of a Turing test is to determine if an artificial intelligence is self-aware, or at the very least its responses appear such that it is indistinguishable from a human. If an A.I. can pass a Turing test its thought process might be close enough to human to pass a Voigt-Kampff or perform complex scene analysis. Or at the very least modifying a Turing test to exclude the A.I. results in a test that humans might fail. Humans cannot generate truly random series of numbers either.

"a human can generate truly random numbers due to the complexity of the brain" I'd strongly challenge that. Humans are awful at generating truly random numbers. Ask thousands people to give you three random digits and you likely won't even get three identical ones because "it's not random" according to people. Go and talk to somebody who plays lottery. Ask them what they think of a combination like 1, 2, 3, 4, 5, 6 and see them scoff as this is apparently impossible to get. Even though it has the exact same chance as their 31, 11, 23, 7, 12, 5.

You may wish to research Benford’s Law for more information about ways human beings suck at randomness, just to back up @VLAZ point.

Present it with a logical paradox?

When has the Turing test ever been not sufficient? We're already 20 years, or 40%, past the projected date and there is still no sign of the Turing conjecture ever becoming true.

@user207421 an AI already passed the Turing test 5 years ago. Besides, the Turing test has never been sufficient John Searle already has shown that "appears to understand" doesn't mean it does. The Turing test is not magic, it's just an AI conversing with people and then people try to determine if their conversation partner was an AI or not. Even real people have failed the Turing test. I can't really see any output would be conclusive either way.

@colmde these androids will naturally have "paradox-absorbing crumple zones" built into their heads.

This question is more about androids than "just" AI.

We touch an almost philosophical question here: if an AI can pass these tests, couldn't it be considered human in a way? Or wee need a more precise definition of what is human..

@Kaddath - not necessarily - as others have pointed out (esp VLAZ), an AI could be able to figure out the correct responses without knowing what they mean. or else just produce natural sounding responses, e.g. a machine could translate "How's it going?" into "RUN SELF-DIAGNOSTIC /QUICK" and then in response, translate "0 ERRORS FOUND" into "I'm not too bad, keeping well..."

@colmde John Searle's Chinese Room thought experiment just means we can't know. It's entirely possible that the AI is just very successfully emulating how other humans behave enough to pass for "normal". However, aren't real humans also doing the same to a large extent? If an AI can really be sufficiently convincing, then is there really a difference whether it truly understands or not? Putting the burden on the AI to prove being human and doubting it even then suggests there is some intrinsic value to "human-ness" that's outside of how one appears. Yet, we've not quantified that.

Seems to me you're not looking for the Turing test, you are looking for a captcha.

@user2352714 You don't even need to modify the Turing test. Humans don't pass it reliably.

Your "random number generation" thing might work, in reverse - if the AI doesn't know it's being tested, its random numbers are likely to be uniformly distributed, whereas human-generated ones won't be anything resembling random.

Adding to the random number test issue, there's some speculation that truly random number generating methods are impossible even in real world randomness.

What about testing for something an AI can never have, but a human will always have, such as blood or saliva?

This next question is if an AI is indistinguishable from a human, is there any meaningful difference between it and a human? Not something we can answer here ofc.

@VLAZ Note that the Chinese Room is far from having 'shown' anything. It merely tries to say something, but has issues relating to how it arbitrarily treats two systems differently without even defining the difference (similar to the issues with the p-zombie argument).

They can tell it a joke. I've watched 1960s science fiction television shows extensively, and the one thing that a robot cannot do is laugh at jokes. The only possible catch here is that the robot will explode and catch the space station on fire, something you may not wish to happen in your story.

@vicky_molokh I absolutely agree. The Chinese Room was just a retort to the "When has the Turing test ever been not sufficient?". And the answer is "it's never been sufficient". At least not by itself. An AI succeeding the Turing test shows nothing more about the AI than it passed the Turing test. It doesn't mean it's self-aware, nor that it can do independent decision making, nor that it's "a human", nor that the AI technology has in any way passed some significant hurdle. A Turing test is passable by machines and even then it's hardly sufficient to determine anything.

Unplug the supercomputers? or disconnect the supercomputers from the communications link?

@VLAZ I still have not heard of any credible claim that the Turing test has been passed yet. I just looked into the instance that you cited from a few years ago, and the claim is highly suspect. I read a little bit of the transcripts of where it was identified as a human, and it was obvious it was not human. Things like "AI: Calm down. Judge: I am very calm. AI: Don't even bother me with the fact you are very calm." are a dead giveaway, the program could not carry on a simple exchange. Other articles likewise refute the claim.

One of the problems with performing Turing tests is that the real humans frequently try to act dumb or what they perceive as robot-like. This leads to judges figuring out not only if this is an AI trying to pass as human, but also if this is a human being an idiot and trying to ruin the Turing test. There have been other "Turing test passed!" claims which were failures for this reason, and based on the transcripts for Eugene I'd say that was the culprit yet again. That AI's chat does not resemble >33% of human conversations.

There are plenty of ways to generate practically non-deterministic (i.e. "cryptographically secure") random numbers, especially if you have access to purpose-designed hardware. Any test based on random numbers is probably more likely to succeed based on the AI's numbers being too random, rather than not random enough.

What @Matthew said... With the observation that a random number generator is built into every x86 processor made in the last quarter of a century. An AI will have zero difficulty in generating truly random random numbers.

@Matthew, to tell the difference between a pseudo-random sequence and a truly random sequence requires a VERY long sequence and either computer-assisted analysis or a heck of a long time with pencil and paper.

@WGroleau, all the more reason why you won't catch an AI because their random numbers aren't.

Humans actually have a harder time generating random numbers than AI. Humans are biased toward certain "random" numbers based on cultural influences and and a natural attraction to patterns. For example, a person who has to come up with a 3 digit number, that begins with the number 1 will typically go with "123" or "147", or some other number that creates a symmetric pattern on the number pad.

@Loduwijk to pass the turing test, the AI has to succeed in convincing 30% of the judges. It doesn't mean "anyone ever will forever be convinced they are talking to a real human". So, it's not exactly a high bar to begin with. The chatbot that passed, managed to convince 33% of the judges - barely the minimum amount. Once again proving passing that the Turing test isn't actually a notable feat. It's sort of "Oh...OK" rather than "OMG, WE MADE IT!".

@VLAZ My point is the test can be bogus, and some past claims of having passed were bogus. Example: "Judge: What is your favorite color? AI: Blue. Human: Please rephrase your sentence. I do not understand "What is your favorite color?"" Or worse yet "Human: My favorite color is null" or "My favorite color is java.lang.NullPointerException". After a different "Turing test passed!" headline, it came out later that the judge had ruled "Neither one seems like a human even though one is. This test is bogus but you're forcing me to choose so that one. shrug" That's what I suspect happened here too

@Loduwijk from now on, my favorite color is official null.

@JoeBloggs I'm not quite sure of the relevance of Benford's Law here. It's about the distribution of digits in many sets of real world data, not necessarily about the distribution of digits in human made "random" numbers. Random numbers are usually requested to be a fixed size, which limits the applicability of Benford's Law. And even if requesting something to which it is applicable, violating it just suggests that the resulting number isn't real data. It says nothing about whether it was created via a true random process, a shoddy deterministic generator, or a shoddy human imagination.

@8bittree: you may wish to follow my link and go down to the section on applications. The essence is that if you ask a human being to generate a sequence of random numbers the first digit of the numbers in the sequence they produce will not follow Benford’s law, because human beings suck at randomness and tend towards uniform distributions for the first digit because it seems more random (when in actual fact it isn’t).

@JoeBloggs I did. And I reiterate: violating it just suggests that the resulting number isn't real data. It says nothing about whether it was created via a true random process, a shoddy deterministic generator, or a shoddy human imagination. For a specific example, the numbers from rolling a fair 10-sided die are uniformly distributed; they do not follow Benford's Law.

Please define "can pass the Turing test". Can it give correct answers to non-trivial questions? What external sources of knowledge it uses if it cannot produce an answer from its internal memory? Can it pass a reasoning test? Is it good at critical thinking? Would it detect a hoax / disinformation / impossible fact?

What was the AI designed for. That will inform a lot what its weaknesses are.

> Is it good at critical thinking? Would it detect a hoax / disinformation / impossible fact? If so, then it's probably a robot. :p

@8bittree: Ah, I see your point now. You agree, however, that it is relevant to detecting shoddy human imagination (for example when asked to pick a random number and the human in question defaults to the numbers 1-10, which clearly does not conform to Benfords law)

Remember that the Turing Test is not now and never was a test for humanity. Rather, it was a thought experiment intended to address the question "is there a minimum standard at we might plausibly consider an entity to be as intelligent as a human?"

As presently written the question is of the form, "How can you prove something when the only available test yields false positives?"

How do I know whether this question was asked by a person?

If the current standard isn't good enough... I must turn the question around. What does it mean to be "human?" This is the subject of volumes of science fiction, including The Positronic Man (which became the movie Bicentennial Man). I highly recommend that book/movie to anyone asking these questions. The questions are so terribly slippery.

Are you allowed to poke it and see if blood comes out?

@craq that would be John Searle's approach.

@AaronF I think the Picard conjecture disproves that approach...

@JoeBloggs I don't agree. It's relevant for detecting ignorance of Benford's Law, and that's about it. Note that someone who is aware of the Law, could easily pick a bunch of numbers that fit the distribution. Doesn't mean they were randomly generated. And many, perhaps most, sources of random numbers are designed to provide a uniform distribution. If I asked someone for a set of random numbers and they followed Benford's Law, I'd actually suspect something very strange was going on!

@8bittree: aside from all those distributions that have e hidden somewhere in them. Which is (as you rightly pointed out) most real world data (hence the whole ‘using benfords law to detect when humans suck at producing random numbers’ thing). Not only that but Benford’s law has analogies in almost all distributions. I’d wager humans trying to mimic the fair uniform dice example you gave would fall foul of it by not producing enough numbers starting with 1, instead unconsciously favouring a uniform distribution of starting digits when the actual starting digit distribution is very different.

@8bittree: We can take this to a chat and continue the riveting discussion of randomness vs distribution matching there if you’d like?

If an AI can convincingly act like a human, test physical traits. A human can report, in person, for a blood draw in sickbay while an AI cannot.

If an AI can pass a Turing test, it's not a valid Turing test, right? I mean, isn't that the point?

Is you AI aware of its environment? Could you remotely flick the lights on and off without it knowing? That would be a good way of checking if the video feed is live or synthesized. AI could assume lights are always on, turn them off and see if the video feed reflects that.

@hszmv: COuld you back up that claim please? I would love to get to know, how throwing a dice sin't random.

@Zaibis: It's been a while since I read it, so I'll have to dig through online stuff. I had difficulty understanding the principles myself, just the conclusion that the Universe might not be truly random is a possible thing. Good news is it's sufficiently random that yes, you rolled that Nat 1 in your D&D game the other night. Shut up and let the DM give you your epic fail.

Ask it a bizarre question, such as “What can we do to make your professional life better?” Or better - pull the fire alarm, and say you can meet outside to discuss it.

Very few people seem to have any idea of what the Turing Test (aka Imitation Game) is. It isn't some test that proves you're intelligent or you're an AI or you're a human. It's just a thought experiment about people being able to tell the difference between an AI and a sentient being (or, in the original imitation game, a man and a woman).