Natural language processing (NLP) is a fast-growing type of artificial intelligence (AI). You've probably benefitted from NLP if you've asked your smartphone for directions to a restaurant or run a spell-check on a word processing program. Read on to learn more about NLP, its history and development, and how it's being used.
What is natural language processing?
NLP is an increasingly common branch of AI, found in everything from smartphones to home kitchens, and involves the ability of computers to understand spoken language and text. Although most uses are fairly pedestrian -- Alexa, what is AI? -- NLP is becoming more sophisticated.
For example, tools like Generative Pre-trained Transformer 3 (GPT-3), developed by OpenAI, use a neural network machine learning model that can not only code but also write articles and answer questions, frequently in a manner virtually indistinguishable from a human response.
Machine Learning
Development of natural language processing
We generally think of AI and its branches, like NLP, as relatively recent phenomena. But it actually dates back to 1950, when famed mathematician Alan Turing developed criteria for determining whether a computer was capable of AI. One criterion for the test involved deciding whether the computer could interpret and generate natural language.
For the next 50 years, linguists developed NLP using painstaking trial-and-error rules. In the 1990s, however, computers became much faster and more capable of doing calculations in seconds, even those that previously took hours or days.
The result was a shift to statistical methods to develop NLP capabilities. The current approach to NLP uses both linguistic and statistical methods to interpret and respond to instructions.
NLP involves two basic steps: data preprocessing and algorithm development. Data preprocessing generally uses at least one of four steps to allow the machine to work with text data:
- Tokenization: The text is broken down into digestible parts.
- Stop word removal: Common words are removed from the text, providing more information from unique words.
- Lemmatization: Words are reduced to their root forms.
- Part-of-speech tagging: Words are broken down into parts of speech (i.e., nouns, verbs, and so forth).
Once data preprocessing has been completed, an algorithm takes over. Two main types are used in NLP:
- Rules-based: This approach uses linguistic techniques that allow the computer to process the information. It's similar to the approach used in the very early days of AI.
- Machine learning-based: Statistical systems are used to teach the computer to process the information, often using a combination of machine learning and neural networks.
Semi-Supervised Machine Learning
Natural language processing innovation
Natural language processing innovation
Already, innovators are making progress on NLP-based tools that could eventually take the place of people. Consider the underappreciated chief of staff or the administrative assistant who schedules and takes notes during meetings and generally keeps the organizational trains running on time.
The start-up Xembly is using an automated, NLP-powered platform to handle many office jobs that often get lost in the shuffle. Its conversational AI agent, Xena, can listen to meetings, take notes, schedule meetings through Slack or email, remind people of action items, and even understand who is talking to whom when there is more than one speaker.
Although Xena may never be able to clear out the refrigerator in your office building or ensure everyone actually signs a birthday card, the agent is likely a harbinger of bigger things to come in the NLP world. Smart companies are already considering how to utilize NLP and other AI tools to make their workplaces more efficient and profitable. And smart investors will pay attention to these tools and how they're used as they continue to develop.
