The recent set of Jeopardy! matches between Ken Jennings, Brad Rutter and Watson, IBM’s question-answering machine, has been entertaining. Sure, there’s a lot of self-promotion going on, but that’s par for the course with the show. As a trivia nut and computer scientist, this challenge appealed to me on multiple levels. As much as I would have loved to see our carbon-based brethren dominate, I’m not surprised by Watson’s victory. The format of the show presents some challenges. Watson bombed some questions in spectacular fashion, but for straight-up knowledge questions, the machine was dominant.
If you want to read more about the match, there’s lots of commentary, so I’ll leave it at that. However, there is a particular article that I wanted to point out. In his answer to the final question, Jennings riffed on a classic Simpson’s line in welcoming our new computer overlords. Ben Zimmer takes exception to the comment in his piece about the match for The Atlantic.
Elsewhere, Ferrucci has been more circumspect about Watson’s level of “understanding.” In an interview with IBM’s own magazine ForwardView, he said, “For a computer, there is no connection from words to human experience and human cognition. The words are just symbols to the computer. How does it know what they really mean?” In other words, for all of the impressive NLP programming that has gone into Watson, the computer is unable to penetrate the semantics of language, or comprehend how meanings of words are shot through with allusions to human culture and the experience of daily life.
We still have a long way to go before we have computers with true natural language processing.
Baker’s undoubtedly right about that, but we’re still dealing with the limited task of question-answering, not anything even vaguely approaching full-fledged comprehension of natural language, with all of its “nuance, slang, and metaphor.” If Watson had chuckled at that “computer overlords” jab, then I’d be a little worried.
After the Toronto answer to the question about U.S. cities, I remember thinking that Watson must be joking. I actually thought the machines were mocking us on national television. It concerned me. Time to figure out where I put that red pill.