Stanford’s Christopher Manning and a Columbia colleague discuss talking, thinking machines
If you’ve asked your phone for a dinner recommendation, or Skyped with a French speaker and had your computer translate, you’ve witnessed the magic of natural language processing, or NLP.
NLP underlies the software that gave us Apple’s smart assistant, Siri, and Skype’s real-time translator. NLP has made science fiction robots such as HAL, C-3PO and Samantha seem suddenly, and often scarily, plausible.
In a review paper published recently in the journal Science, computer scientists Christopher Manning and Julia Hirschberg discuss the past, present and future of NLP.
Manning, a professor of computer science at Stanford University, focuses on the theory and algorithms behind NLP and co-teaches a free NLP course online.
Hirschberg, a research professor at Columbia University’s School of Engineering and Data Science Institute, focuses on training computers to detect lies.
Q: What is NLP?
Hirschberg: It’s using computational techniques – algorithms and statistical analysis – to learn, understand and produce human language content.
Q: What makes teaching language to a computer so difficult?
Manning: A language is a large, complex and changing social creation whose interpretation depends greatly on context. While standard rules apply, exceptions and new expressions are constantly evolving. A recent one is the “because NOUN” construction: “I was late because YouTube.”
Q: Why was the shift to a Big Data approach so effective?
Manning: It became clear that computers could pick up language much faster if trained on large bodies of text. With enough examples, complex patterns come into focus. This is somewhat how children learn language.
Q: What was the first commercial app to use NLP?
Manning: If you count spell check, Ralph Gorin developed his Spell program at Stanford in the early 1970s. Commercial packages appeared a decade later. As early as 1981, an automated translation system called METEO translated Canadian weather forecasts between French and English.
Q: How did the Web speed innovation?
Hirschberg: It made state-of-the-art information retrieval technology available to anyone. By incorporating information extraction techniques into web querying, we can get much more accurate answers than 10 to 15 years ago. The Web also opened up linguistic data to anyone.
Q: How does computer translation work?
Manning: If we collect lots of parallel text – in our target language and translated from our native language – we can use statistics to infer which words typically follow other words, and how words are translated in context, to produce new translations. The trick lies in projecting from statistical counts to make accurate predictions given the large vocabularies of most languages.
Q: Computers still have a hard time choosing the right word. You give the example of “bordel” - French for “messy” or “brothel” depending on the context. What other pitfalls are there?
Manning: Selecting the correct word is certainly a problem. The English word for “ask” often translates in French as “demander.” But if you wish to ask a question, the right word is “poser.” That’s only the beginning. There’s putting words and phrases in correct order, using natural sounding word combinations and connecting clauses intelligibly. Syntax is another stumbling block that can leave a sentence jumbled.
Understanding the context and intended meaning of a phrase is the key to a good translation, especially given minimal information. In a Chinese web chat you might see the response 不做, which translates literally as “not do” but, depending on the context, means “Don’t do it!”
Q: How will translation improve?
Manning: We are among several groups developing neural machine translation techniques. Under a neural network-based approach, algorithms process information in layers to gain deeper understanding. This method, similar to how the human brain works, is much better at inferring context and recognizing similar meanings.
Q: NLP is now used to mine social media to infer the public mood. What are the commercial applications?
Hirschberg: Companies are using this information to assign prices to products and predict the rise and fall of stocks. Ad agencies and political consultants can check the crowd’s pulse without extensive polling. Health officials can spot everything from food poisoning outbreaks to epidemics from symptoms described on Facebook and Twitter.
Q: Computers are learning to detect basic emotions when we speak and write. What are the applications?
Hirschberg: Some companies have started to predict changing national moods based on emotions detected across social media. A recent example is the international debate over Greece’s debt. In medicine, emotion detection may be used to diagnose conditions such as Parkinson’s disease, autism and depression, and assess patient attitudes about treatment options.
Q: Computers are also learning to detect lies. How is that helpful?
Hirschberg: Government agencies have been especially interested in deception-detection technology since 9/11. Currently, researchers look for cues in what is said, how it’s said, in gestures and facial expressions, odor and brain activity, as well as biometric data traditionally used by polygraphs. Together, these features make it easier to tell when someone is lying.
Deceptive behavior also varies by culture, so research has expanded to include how people of different nationalities deceive and detect deception. This work can help in training humans to spot additional deception cues and make better decisions.
Q: What’s entrainment, and how can it make computers (and humans) better conversationalists?
Hirschberg: Entrainment is when humans begin to unconsciously behave like their conversational partners, matching their words, accent, pitch, speaking rate, loudness, gestures and expressions. Research shows that people who entrain more are rated as smarter, more likable and more engaging than those who don’t entrain.
Q: Will computers eventually replace translators, journalists and others who work with words?
Manning: Computers already produce news reports on company earnings and sports results, but writing is an art. We value the original ideas that only a human author can provide. Algorithms are fine for making music recommendations, but music services increasingly emphasize human editorial judgment.
Read Hirschberg and Manning’s article: Advances in natural language processing. Science, July 17, 2015.
Kim Martineau writes for the Data Science Institute at Columbia University.
