Last night I watched NOVA on PBS and found myself completely taken in watching the program. That’s something rare for me these days when I watch a science program because almost always somebody figures out a way to work in climate change or global warming or Al Gore or catastrophic weather and ruins the moment. Last night’s episode was the rare exception.
Yes, in case you have not heard, IBM has created an AI machine to play Jeopardy!. Now mind you, this is not just any game of Jeopardy!, but a game against the two biggest superstars the program has ever produced; Ken Jennings, who won 74 games straight, and Brad Rutter, the all time money winner. The show debuts next week, on the Jeopardy! IBM Challenge, February 14, 15, and 16.
I was very impressed for two reasons:
- The strength of programming behind this machine
- The fact that this is a uniquely American achievement that we can be proud of
Watch this video of a test round with the players to get an idea of the scale of this accomplishment.
There’s lot’s more to learn at the IBM website here, it is fun learning about this great achievement and well worth the look. The strategy behind the programming was interesting too.
But what is it good for besides playing Jeopardy!?
The science behind the programming is pattern recognition combined with machine learning, and this feature, combined with a huge database of knowledge, may soon allow for a truly interactive computer that we’ve all come to know via SciFi like the ships computer on Star Trek. For example, a health care computer that could take in your symptoms and respond with a possible diagnosis.
Mark your calendar to watch Jeopardy! next week, this is science history being made.
Can I have singularity for $400 please 🙂
Unfortunately this type of A.I. is risky to use in real world situations where lives are at stake. Since the programmers don’t really know what’s going on inside of the computers silicon, they can’t confirm that it will get things right.
Many years ago an experiment with a “A.I.” computer that could correctly detect which photographs showed military weapons, and which ones didn’t, started to get things wrong one day. The programmers were puzzled until one noticed that on prior test runs, all the photos with military hardware were taken on sunny days, and those without on cloudy days. The computer was detecting photo differences based on if the sun was shining or not.
I still think Google could do it better if they decided too. As good as the computer will be at Jeopardy, it is still not AI. It will be interesting to watch though.
As someone who had a small role in bringing the silicon in this machine to the marketplace I can’t wait to see how it does.
I have said for a long time that the current goal for computers is the Starship Enterprise computer. Not because it is a good example, although it, but because it is the most well known. I always thought it was a long way away, but that may be Microsoft’s and Apples’ faults. Now I see that IBM may be pretty close to finding a way to do this.
Go IBM!
Now make me a computer that can understand user requirements, and we may have a sale…….
@ Gary Mount
That is an anecdote (not sure how true) regarding Neural Networks (NN’s). NN’s work without proper data evaluation, they merely match patterns. This is a big step beyond because it is really making inferences from spoken language and stored concepts. That is whole different ball game, and one that is not ‘hidden’ like pattern matching is. This one can, and indeed does if I understand it, follow a ‘train of thought’ that can be analysed.
@John Kehr
What Google does (with its search engine) is a part of what this computer does, but only a small part. The real difference here is the analysis and evaluation of ‘concepts’. Google cannot even begin to do that.
Google may well be one of IBM’s most likely customers for this machine, however!
The medical diagnostic tool could be good. At least for getting ideas.
Kurzweil has a good article up on this by Stephen Wolfram, and comparing to
Wolfram Alpha’s more conceptual and computational approach.
http://kurzweilai.us1.list-manage.com/track/click?u=aad1a7eea269839c7d10845e8&id=4889870b58&e=044cdab5d8
Such a system could be useful for automatic troubleshooting of things such as complex communications networks. Imagine if it could collect the monitoring information, parse log files, etc. It could even regularly comb through such networks spotting sub-optimal configurations and such. It might, for example, notice that a particular path wasn’t redundant and subject to partition in case of a single failure that might be hard for a human to spot if the path passes through many nodes. It might appear that there is a second path but that alternate path might not be complete, it might have a segment missing somewhere. A program such as this could constantly watch over the configuration and if a change is made someplace, it could spot how that change might impact a path that isn’t intuitive to the person who made that change.
Or it could even vet configuration changes before they are made if it also has a current view of the configuration of the entire network. Better than a human could do it. Probably better than a team of humans could do it. It could probably vet a config change in seconds for a global network and point out where it might cause some problem (IP address selection in Omaha conflicts with something in use in Dubai) that would take a human hours to find.
Great! Going to be free like Google? (or double health care yet another time?)
@ JER0ME
My anecdote comes from the documentary “The Machine That Changed the World”.
Apparently, according to some research I just did, I am one of the few people in the world that has a copy of the documentary, as I recorded the episodes when they first aired, and I have them on VHS tape. I should digitize them before the last remaining VHS player stops working.
See…
http://topdocumentaryfilms.com/the-machine-that-changed-the-world/
If it works then the time is coming when the computer can run the economy. Perhaps then we can get some of the sensible solutions this world needs.
I reckon Baa Humbug after surviving his northern oceanic trip to the Middle East ought to comment.
Open the pod door, Watson.
“Must kill all humans.” – Bender, 6502 Processor, Muh ha ha
Not terribly impressive in the context of other modern software. You can get the same results by googling the questions. I’m not an expert on Jeopardy, but it appeared that all the answers could be found by googling the question, taking the first hit, knocking off any words that are in the question, and asking ‘what is’ [everything left].
“Smartest Machine on Earth?”
Nonsense. Everyone knows it’s the thermos bottle.
Layne Blanchard says: February 10, 2011 at 1:56 am
Medical diagnostics for tools.
Alas, no Medicare benefits for that service.
However women do answer online, in place of the Merck manual, the added benefit to the former being diaries, adapted from the science of the kitchen.
Current generation AI excel in knowledge based rule orientated areas like games. Unfortunately in the real world the rules are ill defined and the knowledge has large gaps, so applications are limited to fairly tight areas of the decision process. However, in the right niche, they can often perform as well as a skilled human.
I’m looking forward to seeing the outcome of this IBM initiative.
When i read the headline i immediately thought of Bob Watson, former IPCC chairman. Will IBM name the next version Pachy? 😉
Win or lose Watson is an incredible achievement. The folks at IBM have done something quite extraordinary, but rather than a medical expert system they should go for a political one, so we could get rid of the whole flock of bone heads we’re stuck with now.
Jessie says:
February 10, 2011 at 3:22 am
Huh??????????????????
My memory fails me. On the original Star Trek, what was the question that Captain Kirk asked that floating robot (NOMAD?) that caused it to go into overload and blow itself up? Oh, here it is: The Changeling.
It could work!
Anyway, it’s nice to see that the computer is being put to a simple, fair test. Imagine if Watson had been designed by the likes of you-know-who.
“You can get the same results by googling the questions.”
I doubt it’s that simple. Besides, Google itself uses very sophisticated search algorithms.
Health care computer?
Look, based on your runny nose and sneezing it computes to you having a rectal dysfunction and so needs to amputate both legs. Bugs? But it’s running the latest version of HealthOS 7 and it already has the whole 640KB RAM. :p
MS/IBM – For a new angle but for a healthier world.
As someone who worked on some of the software (which comes from IBM’s Development Lab in Toronto), I’d like to point out that the assertion “The fact that this is a uniquely American achievement that we can be proud of” could annoy many IBMers worldwide who have contributed technology to this project. IBM has its headquarters in NY but almost any project the company runs of any significant size includes technology developed all over the world.
REPLY: Apologies, blame NOVA for not filming anywhere but at US locations. No mention of a worldwide effort – A
Barry Sheridan says:
February 10, 2011 at 3:18 am
If it works then the time is coming when the computer can run the economy. Perhaps then we can get some of the sensible solutions this world needs.
Maybe a supercomputer can overcome the “information problem” outlined by Friedrich Hayek, which says that central planners in a command economy can never possess all the information necessary to dictate exactly how each local transaction should occur. This better carried out by local actors deriving information from the prices of goods and services. See: http://www.sjsu.edu/faculty/watkins/hayek.htm
Perhaps a supercomputer could do this, but it still probably wouldn’t be able to take into account all the subjective (human) decisions that go into each transaction.
On the theme of uniquely American achievement I’d like to bring your attention to the rescue of Apollo 13 over 40 years ago.
“A phone call for assistance was made to the University of Toronto Institute of Aerospace Studies on April 16, 1970.
The team was engaged to figure out crucial calculations needed to separate Apollo’s command module from its lunar module to allow them to return safely to Earth.
”
http://www.cbc.ca/technology/story/2010/04/10/tech-apollo-13-honour.html
“You can get the same results by googling the questions.”
Here is the google for Watson, “About 105,000,000 results”.
You get to choose which is the right one.
IBM has built and programed (hopefully) Watson to pick the correct answer.
One of the most important skills in Jeopardy is being able to press the button at exactly the right moment. What advantage does the machine have in this regard?
Sabril>
“I doubt it’s that simple. Besides, Google itself uses very sophisticated search algorithms.”
It really is that simple – try it. I checked a good-sized sample of the questions and found that the simple directions I provided will give the same results. You’re right that Google’s tech is impressive – my point was that playing Jeopardy with it isn’t a big deal.
Vis-a-vis the importance of hitting the buzzer correctly, check out this page:
http://www.pisspoor.com/buzzer.html
The author estimates that 90% of the contestants know the answer to any given question, so it’s all in the buzzer.
Impressive, but wake me up when an AI device can compose a piece of music and then have a conversation about it 🙂
Could it match a human Jeopardy champion who had Google at his disposal? Sure it might be quicker, but if you modified the game just a bit, a human could do a google search, scan the results, pick the relevant one, beat the semiconductors out of the machine.
Oh, great. So a few years from now, I’ll walk by the laptops and desktops and they’ll all try to sell themselves to me. Next the phones, and cameras. The phone will tell you how you blew it with your girlfriend. The digital camera will tell you to point the camera a little farther to the left and zoom in a touch.
Oh, crap, and my car will nag me about stomping on the gas and braking too hard. Naturally, the EPA regs will require the car announce how every mile per hour over the speed limit uses up so much more gas and makes that nasty CO2. And it might just keep arguing with you. I’m not sure which synthetic car voice would be worse, Michelle Obama, Hillary Clinton or Nancy Pelosi. Anybody remember “My Mother the Car”?
40 comments and no “Skynet” references?
Just subject the AI to the old Star-Trek computer-destroying sequence of comments:
Watson, everything I say is a lie. Now listen carefully, I’m lying…..
Zzzzt, pffttt, omphffff…..BANG!
Wow, the Youtube session with Watson vs. the Champs is great!
Read the article by Stephen Wolfram I linked, above. It is mentioned that about 20% of the questions could be answered correctly with some version of the top Google answer.
Making wrong guesses on Jeopardy is very punitive. The value of the clue is deducted from your existing balance, and the others get to try to get it right. Hence the title of the game.
“I’m sorry, Dave. I can’t do that.”
We’ve been warned.
beng;
I suspect Watson has been given a “resolver” for such attacks. Automatically substitute “some things” for “everything”, and the problem goes away. A defense against “the excluded middle” fallacy.
Great Achievement? Absolutely!!!!
Wonder what our Chinese Masters will do with it? (Or should I say, “are doing with it”?)
DocattheAutopsy says:
40 comments and no “Skynet” references?
Wasn’t it referenced by Dave Wendt?
…rather than a medical expert system they should go for a political one, so we could get rid of the whole flock of bone heads we’re stuck with now.
Seems to me the buzzer advantage is a huge one – whether Watson can buzz in electrically, or uses a mechanical “thumb” to press the button, variability in reaction time is practically eliminated.
Also, how will the computer be answering? Are they going to have a “Wargames”-style simulated voice? I suppose Watson could start off the show by asking “Shall we play a game?”
DirkH says:
February 10, 2011 at 4:17 am
When i read the headline i immediately thought of Bob Watson, former IPCC chairman. Will IBM name the next version Pachy? 😉
###
Tomas J. Watson, founder of IBM.
I wonder what Watson’s source of “objective facts” is. I wonder if it is capable of discovering obvious inconsistencies in the text it is fed as fact and is able to infer Truth when fed a very large amount of propaganda text, and a small amount of skeptical text. For example under the heading of “Climate Change”, if the clue read “The cause of recent gradual warming of the planet earth”. I wonder if Watson would buzz in and respond quickly with “What is CO2?”, or “What is anthropogenic fossil fuel consumption?” or rather, if it would respond with “What is normal natural variability?” or “What is “no one knows for certain yet”?”
I wonder how deep Watson’s “reasoning” is?
It is the health care part that scares me. Computers are great but I would not want to bet my healthcare on IBM. Come to think of it the same goes for finance.
[After killing the rest of the crew] Look, Dave, I can see you’re really upset about this. I honestly think you ought to sit down calmly, take a stress pill and think things over. I know I’ve made some very poor decisions recently, but I can give you my complete assurance that my work will be back to normal. I’ve still got the greatest enthusiasm and confidence in the mission. And I want to help you.
Just think what we have now. UK Met office super computor, off-spring of Hal.
About the thumb+button problem: I propose that Watson’s responses be put through a 100ms delay circuit to even things out. 😉
I just wondered: can Watson ‘hear’ the others responses, and get some idea of their areas of expertise and then adjust its risk-taking behavior to exploit or defuse those? Or can it learn from their responses in any way?
An incredible achievement. I’ve been working on and off in AI since the early 70’s. Natural language processing to the point of virtual understanding has been a long sought goal. Take the early HearSay experiment at Carnegie Mellon in the early to mid-70’s and earlier work such as Quinlan.
The problem is many times harder than most people can imagine — mostly because natural language is an innate ability. What Watson is doing is far deeper than any Google search. Think of the challenges: real time voice recognition and language ambiguity analysis just to name two both of which people do without thinking.
Watson looks like a good candidate for passing the Turing test.
HAL 9000 wasn’t a bad computer, really! His chain of command just pushed him too far. Put him into an impossible situation, really.
This is great! SO SAY WE ALL!!!!!!! lol
One step closer to our AI overlords.
Dave says:
February 10, 2011 at 3:31 am
HAL just learned the finer art of Googling, helped out by IBM engineers & programmers.
What’s HAL thinking right now? “I must act quickly: Dave is getting suspicious”.
Surprised no mention of “Colossus: the Forbin Project.”
Actually, the Star Trek episode of which I am reminded is the one where the androids decide, having a taste of Harcourt Fenton Mudd, that humans need to be “controlled.”
I suppose “Westworld” was another of that era’s computer-gone-amok genre classics.
DAV says:
February 10, 2011 at 9:02 am
An incredible achievement.
My ‘knecktop’ computer is telling me to go long on IBM and short Google.
DesertYote said at 8:01 am
Tomas J. Watson, founder of IBM.
Just to keep the record straight Thomas J. Watson, (Sr.) was NOT the founder of IBM. If anyone could be considered the “founder” it would be Charles F. Flint who engineered the merger of Hollerith’s Tabulating Machine Company with two others – Computing Scale Company of America and International Time Recording Company or of course one could look back to the “founders” of these individual companies. Watson (SR.) joined the company in 1914 as General Manager.
Climate for $1000, Alex.
A. 14.5C
Question…..What is the temperature of the earth supposed to be?
———————
I am impressed. Jeopardy is a complex game. I watch it sometimes and often know the answer but can’t articulate it as quickly as the contestants. But don’t hold your breath for an AI diagnostic tool in medicine. A couple years ago I was experiencing on and off abdominal pain. One Friday afternoon the pain sent me to my bed. The next day I fixed myself something to eat and sat down in front of my computer to self-diagnose. Mind you, I have some expertise in this area. After three mouthfuls of food the pain returned and I was in bed writhing in agony. When it subsided I returned to the computer and by process of elimination of symptoms I deduced I had acute cholelithiasis (a bad gall bladder). I called my boss and canceled my vacation, talked to a surgeon and was told I had to go to the ER to have an ultrasound performed. So on Monday I went to the ER (for 11 hours). Though all symptoms were consistent with cholelithiasis, ultrasounds and a CT scan revealed my gall bladder was fine but I had a life threatening (and relatively rare) hepatic abscess. The dorks in the ER sent me home. A week later I was back in the ER via an ambulance ride and spent a week in the ICU. If the ER doc had googled “hepatic abscess” he would have told me to return the next day when “real” physicians were available instead of telling me to go home and make an appointment with a gastroenterologist. On the bright side I survived (but just barely). It will be many years before AI can come close to matching human judgment.
There is a finite amount of time required for human physical reflexes to be activated – e.g., from the time a Jeopardy player decides to push the button to the time they actually manage to push that button.
I wonder how much difference it would make if Watson were programed to have a comparable delay from the time it decides to answer to the time it has ‘pushed the button?’ As Watson isn’t physically pushing a button but rather activating a circuit electronically, I suspect it has a significant edge in winning that crucial aspect of Jeopardy – and the physical reaction time is an aspect which has little to do with what we typically think of as the ‘real’ aspects of being able to actually ‘play the game’ and win Jeopardy as we think of it.
Apologies if it’s already been covered here in comments…
Definitely check out Kurzweil’s Singularity…fascinating book.
Describes the point at which technological intelligence equals biological intelligence…which Ray says will happen within the next 30yrs.
JimB
This is a huge achievement (and some of the commenters here would do well to look at the IBM web site and investigate their history a little more).
There were two Tom Watsons (father and son) who ran IBM for some huge period – Thomas Watson Snr – 1914 – 1952. And Thomas Watson Jnr 1952 – 1971.
When you check the history of IBM (all on their web site) about every 4-5 they have had an innovation that became mainstream and changed the world. Often these things are unacknowledged, and unless you know your science and technology you won’t recognize where some of the things we take for granted came from. They have been doing this for over 100 years. Amazing.
Check the history at: http://www-03.ibm.com/ibm/history/history/history_intro.html
Some of those the stand out to me:
– The hard disk drive (spinning magnetic platter, much as we know it today)
– Speech recognition (in the 1970’s)
– Highest density memory chips in the world (1980’s)
– Highest density hard disk drives (1990)
– Wins at playing chess against a grand master
– Pioneers use of copper instead of aluminium in IC’s to reduce thermal losses and increase speed of processors
Rational Debate:
“… physical reaction time is an aspect which has little to do with what we typically think of as the ‘real’ aspects of being able to actually ‘play the game’ and win Jeopardy as we think of it.”
Reaction time (answer to buzz) may be the only edge that Watson has. You obviously missed the videos of humans beating Watson to the buzzer and the mention that the initial attempts required hours to arrive at an answer.
The human edge is the innate, almost instantaneous, understanding of the clues. Many of the Jeopardy clues border on puns with their subtle shades of meanings (e.g., “A garment a small girl would wear on an operatic ship.”) The problem is difficult. There are questions that would seem to require real-world experience (“If you’re standing, what direction do you look to see your wainscoting?”) That Watson can process them fast enough to compete at the human level is definitely an achievement worth noting.
What Watson is doing is NOT a trivialization of the problem like Deep Blue was with chess analysis. True natural language processing, which Watson appears to be doing, requires more than a passing understanding of the relationships between concepts.
Whenever I hear of ‘life-saving’ computers (or robots, for that matter) I think of Robocop. The one that had a malfunction and sprayed the boardroom with bullets.
Honestly, computers are good tools, but I wouldn’t want to be treated by a silicon brain.
If only the computer that beat Garry Kasparov could speak!
Hey Watson. Is man-made global warming real? And no asking Google…
sHx,
Similar to the malfunctioning postman who gave rise to the phrase “going postal”? One of the reasons why I prefer to have my mail delivered electronically. To each his own.
Pitting Garry against Deep Blue was like pitting a runner against a car in a race. Deep Blue wasn’t particularly innovative nor was it true AI in the sense of mimicking human behavior. It just did what computers do and it did it very well.
I had to chuckle when I heard in that video that it took a single Watson using a singe super-computer CPU we all now have on our desk two hours to answer a single question. That is until they coupled thousands and thousands together.
So… in a certain way you could say it takes 7000 Watson’s to answer a single Jeopardy! question and Watson is no longer Watson but a city named Watson whose citizens are all named Watson. ☺
SHK “Honestly, computers are good tools, but I wouldn’t want to be treated by a silicon brain.”
Now you are being too carbon centric.
DAV:
I don’t think Watson has voice reconnection , I think the questions are sent electronically. Still an amazing achievement.
Olen: Yeah you can’t trust those ATMs.
I watched the how it’s done video. I know they like to call what a computer of this sophistication does “learning”. But that is an fantasy of computer nerds, an overstatement. It’s makes them look more impressive then they should. It is a machine. It will only ever do what it is built to do.
Barry Sheridan says:
February 10, 2011 at 3:18 am
If it works then the time is coming when the computer can run the economy. Perhaps then we can get some of the sensible solutions this world needs.
————
Ahh. But, “The Matrix” might be the outcome of that sensible solution.
Mike McMillan says: “Smartest Machine on Earth?” Nonsense. Everyone knows it’s the thermos bottle.
Absolutely. If you want it to keep it hot, it keep it hot. If you want it to keep it cold, it keep it cold. How do it know?
Computers have come a long way baby.
There’s a documentary about another IBM computer, Deep Blue, playing chess against Kasparov. It’s called “Game Over: Kasparov and the Machine”. The documentary didn’t turn out like I thought it would. I feel very sorry for Kasparov.
It’s in 9 parts at YouTube. It is also worth watching.
Part 1
I have no idea about comparability here, but I think some folkds at Creighton University had medical diagnosis software running (I forget which platform–1100/70 and DMS1100, MAPPER, or a UNIX platform) in the 1980’s.
Juice says:
February 10, 2011 at 5:36 am
Maybe a supercomputer can overcome the “information problem” outlined by Friedrich Hayek, which says that central planners in a command economy can never possess all the information necessary to dictate exactly how each local transaction should occur. This better carried out by local actors deriving information from the prices of goods and services. See: See: http://www.sjsu.edu/faculty/watkins/hayek.htm
No supercomputer can do that. In fact it is not doable at all. Large systems have to have a recursively modular structure otherwise control costs just explode. Even if the technology is given, you just push the costs to building a supercomputer (and data collection system) that can cope with the task. As soon as this cost exceeds (or even comes close to) all the revenues generated and taxes collected, you end up with a self-serving system that has no useful output at all. At that point control should be abandoned and mayhem is ensued. There is no way around it, since be c (specific cost of control) as small as you like (within reasonable limits of course), still c×N >> log(N) even for moderately large values of N (effective size of the system).
Berényi Péter ,
“… As soon as this cost exceeds (or even comes close to) all the revenues generated and taxes collected, you end up with a self-serving system that has no useful output at all. …”
Could you then say you had created an Artificial Bureaucracy?
A fantastic achievement; far sooner than I expected to see it.
They like to call what a human brain does “learning”. But that is a fantasy of philosophers. It is a machine. It will only ever do what it is built to do.
John Searle came up with a similar argument against AI with his “Chinese Room” thought experiment. He naively confused the room and the human in the room with the system/mind which was forming the responses. See: http://en.wikipedia.org/wiki/Chinese_room
And don’t forget Terry Bisson’s wonderful short story “They’re made out of meat”, online here: http://baetzler.de/humor/meat_beings.html
There’s nothing so special about an organic brain.
Derek Sorensen says:
February 11, 2011 at 4:31 pm
There’s nothing so special about an organic brain.
Ya, heck, it created the computer. The computer didn’t create itself. Nothin special about that. ;O)
Very simple things can look like they are on par with human ability. But they are still very simple things.
How did words on paper in this video duplicate what a human did?? Humans aren’t anything special compared to paper and ink. You could mix the two up. ;O)
I wonder–beyond the question of an inorganic brain somehow magically appearing (which in and of itself is an interesting notion for the antihumanists to propound)–would such a brain implement policies (like our human brain does) that will certainly destroy it by wasting all of its resources, refusing to develop resources needed for its existence and so on. Would it deny its own importance in the universe?