2017-10-28 00:00:00
10 min read
“Hey Siri, is there a chance of rain tonight?”
Siri: “It doesn’t look like it is going to rain tonight.”
“Alexa, can you wake me up at 5 am tomorrow?”
Alexa: “Sure. Alarm set for 5:00 am.”
“Hey Siri, what are the popular hangout spots in this city?”
Siri: “Here are the top cool places to hang out in Amsterdam!”
*presents a list of hangout spots sorted in order of popularity*
With smart assistants like Siri, Cortana, Alexa, and Google Assistant, such conversations have become very common these days. These intelligent personal assistants are adept at understanding the context of the query raised by us and presenting results in the form of spoken language, sometimes also providing useful links, directions on maps, etc. Such systems are based on Natural Language Processing (NLP)- a combination of computer science, artificial intelligence, and computational linguistics – aimed to help humans and machines communicate in natural language, just like a human-to-human conversation.
In the year 1950, Alan Turing, a well-known mathematician and computer scientist proposed the well-known Turing Test. In his paper Computing Machinery and Intelligence, he stated that, if a machine can have a conversation with a person and trick him into believing that he is actually speaking to a human, then such a machine is artificially intelligent. This was the foundation of NLP technology. An effective NLP system can comprehend the question and its meaning, dissect it, determine appropriate action, and respond in a language the user will understand.
NLP = NLU + NLG
As stated above, Natural Language Understanding (NLU) and Natural Language Generation (NLG) are two subsets of Natural Language Processing (NLP). Although all three of them deal with natural language, each one plays different roles at different points.
Natural Language Processing (NLP), Natural Language Understanding (NLU), and Natural Language Generation (NLG): What is the difference between them?
The terms Processing, Understanding, and Generation are self-explanatory here.
- NLU understands the human language and converts it into data
- NLP identifies and processes the most significant data and structures it into text, numbers, or computer language
- NLG uses the structured data and generates meaningful narratives out of it
Let us go through each one of them separately to understand the differences and co-relation better.
What is Natural Language Understanding (NLU)?
NLU uses AI to comprehend information received in the form of text or speech. It attempts to understand the meaning behind the written text. After having the speech recognition software convert speech into text, NLU software deciphers the meaning of words, even if it has common human errors or mispronunciations.
It is quite possible that the same text has various meanings, or different words have the same meaning, or that the meaning changes with the context. However, the understanding of natural language is based on the following three techniques:
- Syntax: understands the grammar of the text
- Semantics: understands the actual meaning of the text
- Pragmatics: understands what the text is trying to communicate
Knowing the rules and structure of the language and understanding the text without ambiguity are some of the challenges faced by NLU systems.
Applications of Natural Language Understanding (NLU)
The main use of NLU is to read, understand, process, and create speech & chat-enabled business bots that can interact with users just like a real human would, without any supervision. Popular applications include sentiment detection and profanity filtering among others. It can be applied to gather news, categorize and archive text, and analyze content. Google acquired API.ai provides tools for speech recognition and NLU. It mainly scrapes through unstructured language information and converts it into data, preferably structured data, which can be processed and analyzed for the desired results.
What is Natural Language Generation (NLG)?
NLG does the exact opposite of NLU. Given the data, it analyzes it and generates narratives in conversational language. It converts data into easy-to-understand natural language. It goes way beyond template-based systems, having been configured with the domain knowledge and experience of a human expert to produce well-researched, accurate output within seconds. This technology is the key behind Turing’s vision of tricking humans into believing that a computer is conversing with them or reasoning and writing just like humans.
Applications of Natural Language Generation (NLG)
The NLG market is growing due to the rising use of chatbots, the evolution of messaging from manual to automation, and the growing use of technology involving language or speech. NLG bridges the gap between organizations and analysts by offering contextual understanding through storytelling for data and steers companies towards superior decision-making. It enables non-data experts to take advantage of the free flow of vast data and make informed decisions that were previously mostly dependent on experience and intuition.
Narratives can be generated for people across all hierarchical levels in an organization, in multiple languages. NLG can be of great utility in Finance, Human Resources, Legal, Marketing and Sales, and Operations. Industries such as Telecom and IT, Media and Entertainment, Manufacturing, Healthcare and Life Sciences, Government and Defence can benefit from this technology to a great extent. Some of the most common applications of NLG are written analysis for BI dashboards, automated report writing, content creation (Robo journalism), data analysis, personalized customer communications, etc.
Despite its business applications, NLP has not been widely accepted currently due to the following common challenges.
- There is information overload and businesses are not fully aware of which data to be considered important from the vast data sets.
- Businesses lack the tools and techniques to extract relevant and useful information from unstructured or semi-structured data.
- NLP models require high-end machines in order to process, store and maintain data from heterogeneous data sources.
What does the future look like?
Businesses will invest more in NLP, NLU, and NLG techniques to build systems that would be unsupervised, effortless, and be able to interact successfully in a human-like manner. For this, there is ongoing research in the areas of syntax, semantics, and pragmatics of natural language.
Moving into the future, these AI technologies are constantly leveraging techniques and perspectives from a range of fields such as cognitive science, computer science, linguistics, language, data science, and more to unlock a whole new level of interaction between humans and machines. Organizations are using BI and augmented analytics tools to make better sense of their enterprise data and improve their decision-making processes.
In a nutshell…
- NLU reads and interprets language, while NLG writes and generates language.
- NLU explains the meaning behind the written text or speech in natural language, while NLG generates the natural language using machines.
- NLU draws facts from the natural language using various tools and technologies such as parsers, POS taggers, etc, while on the other hand, NLG uses the insights generated from parsers, POS tags, etc. to generate the natural language.
NLP is increasingly becoming an important area of interest, and major tech giants like Google, Apple, and IBM are investing heavily to make their systems more human-like. According to a study by Tractica, the global NLP market is expected to reach $22.3 billion by 2025. These systems are already trending and it is only a matter of time before they redefine the way we interact with technology on a daily basis.
About Phrazor
Phrazor empowers business users to effortlessly access their data and derive insights in language via no-code querying
You might also find these interesting
