Tonight on Jeopardy: IBM’s Watson beats humans at their own game

A big part of what makes us human is the way we communicate with each other. We speak words, big deal, machines can do that. But what sets us apart as humans are the flaws, nuances, slang, shorthand and ambiguity we often use when we speak. The famous game show Jeopardy asks questions that are not designed for a computer to answer. They are spoken in natural English language, a form of communication that is not what computers were designed to understand… until now. In 1997 IBM’s Deep Blue defeated the world’s number one chess player. Fast forward 14 years, in a joint effort with the game show Jeopardy, IBM has created a supercomputer that is a cloud comprised of roughly 100 IBM POWER7 servers . This 10-rack mini data center was represented by a friendly avatar known as Watson who competes against human contestants on Jeopardy.

Until now, in order extract an answer to a complex problem from a computer one needed to be very well versed in a language that computers understand. You needed super-human-geek-brain-power to understand complex algorithms and formulas, machine codes and the binary state of being in order to ask the mech. Watson has changed all that. Now anybody can simply speak (or type) a question, in natural, human language and Watson will answer you in natural-sounding, simple to understand speech. Now doctor House can fire all his staff, work with Watson and do a differential diagnosis in just minutes, save the patient’s life and have time to a real relationship with Cuddy. Next! But I digress, back to Jeopardy…

This week IBM’s Watson beat the pants off the top two former Jeopardy champions, Ken Jennings and Brad Rutter. Watson rang in first and had the correct response what seemed like 90% of the time. There were a few instances where the humans managed to ring in first but those were only in times where Watson was unsure. With each answer Jeopardy displayed Watson’s top three responses (in his mind) along with how sure he was of each response. Whenever Watson was more than 50% sure of one response he would ring in and give the correct question. There was a cute moment when Watson told host Alex Trebek “I’ll take a guess” and then responded correctly with only 32% certainty of his answer. Although Watson got the Final Jeopardy response wrong (based on flawed programming no doubt) he was wise enough to wager very little as he already had an unbeatable lead over the humans. The final scores: Watson $35,734, Brad Rutter $10,400 and Ken Jennings $4,800. (Above image: all three contestants just before the Final Jeopardy round)

I can’t help but think however that Watson might have an unfair advantage. While the other contestants must listen to the host read the clues Watson gets them electronically texted to him, plus he does not have to perform the time-consuming, mechanical action of clicking the a button to ring in. Could any human really click a button faster than a machine can electronically ring in? I wonder if they took any measures to level that playing ground. Nevertheless, Watson’s ability to frame and verbalize a correct response from natural, often ambiguous human wording and compete in the natural flow of a human game show is more than impressive. It’s downright amazing.

I will be tuning in to tonight’s Jeopardy final of man versus machine to determine the million dollar winner. Anyone else? Your thoughts and comments are welcome below.

Check out these videos about Watson:
[youtube]cU-AhmQ363I[/youtube][youtube]dQmuETLeQcg[/youtube]

[Dani]

What most interests me is what Watson's database is like. I think it's half the battle figuring out what question is being asked. But it's the other half of the battle coming up with a correct answer to an infinite number of questions on an infinite number of topics.

[whipaway]

Watson does have to click the mechanical button to ring in just like any other contestant. A special actuator was constructed so that Watson can activate the mechanical button. This actuator was shown in the Feb 14th airing of Jeopardy.

[WASDted]

@whipaway, I did notice that but they never went into how he actually rings in. The clue are sent to Watson via a text file at the same time they are revealed to the human contestants. A computer can certainly read text much faster than we can. Add to that the humans have to process what they hear or read then their brain has to send a command to the muscles in their fingers and hand to press the button. It seems like Watson just sends an instant pulse to an electronic switch... much faster. But again, the idea wasn't to prove if machines can think and react faster than we can, we already know they can. The idea was to see if it can play a game or respond in a situation where natural human language is used to communicate and it did an impressive job. Even the whole bit about the confidence level and buzz level, Watson knowing what he knows and what he doesn't know (ie when not to ring in) was very impressive to me. Isn't that a level of 'self awareness'?

[Dani]

Impressive, yes. But I wouldn't go so far as to call it any type of self-awareness.

[WASDted]

Well, of course I don't mean true self-awareness... it was programmed to be 'aware' of when it knows and when it doesn't know, and then respond or refrain.