Solutions
Build ChatGPT-powered AI Chatbot
Seamless bot-to-website integration
Use cases
Automate your digital business processes
Provide exceptional assistance
Digitise and improve your HR tasks
Solutions
Use cases
Whenever AI Chatbots come into the picture, you’ll inevitably hear the golden three words: Natural Language Processing (NLP).
On the surface, the words seem self-explanatory: Since our computers speak a fairly unnatural language consisting of 1’s and 0’s, it’s going to have to do some processing to understand us. But is there a difference between one company’s NLP and the next? And how is the state of NLP today affecting your Chatbot? Before we go into that, let’s first dive into how it all started.
NLP As We Know It
It’s been a long time since the ’60s, and the approach when it comes to NLP has changed quite drastically. Gone are the days of rule-based, keyword-matching systems. These days, it’s all about building language models. The following is a simplified step-by-step on how this is done.
Building a language model today starts with creating as wide a dictionary as possible — through a process that usually involves trawling the Internet for huge amounts of data. And when I mean huge, I truly do mean huge — GPT-3, for example, was trained on 45TB of text data sourced from various data sources, forming a total dictionary of 400 billion tokens (read: words).
It’s not enough to just know that the words exist: For a machine to truly read and understand a sentence, they would need to have a grasp on context as well. Take for example the following sentence:
By the time he arrived, she had had dinner.
For a machine to understand the difference in meaning between the first had (meaning already) and the second had (meaning eaten), it would have to know a bit more than just assigning a token.
While there are many ways to do this, the most popular way these days is arguably by using Transformers: A mathematical sort of magic that allows computers to ‘understand’ words based on the words that appear around them. The result? A machine with a pretty good grasp of how human language works.
Now that we have the language part down, we are ready to use the language model to solve the task at hand — building a conversation. Enter transfer learning — the tried and tested idea that you can achieve better training results when starting with a model that already knows its stuff. All that’s left is to take your language model and train it with your new domain-specific data, whether it be intents and entities for your new corporate chatbot or chatlogs from the deceased.
The quality of the language model is where NLP providers may differ slightly, and if you’re going for an NLP solution by one of the tech giants (Google, Microsoft, IBM, et cetera), we mean very slightly — unless you’re working on the fringes of Conversational AI (in which case you shouldn’t be looking for a mass-market solution) you wouldn’t even notice.
Challenges in NLP
The current approach to NLP doesn’t come without its own set of problems, and if you’re been in the business of Conversational AI you’ve probably picked up on one or two. Coming up with an exhaustive list of ways NLP today could improve would be too big an endeavor for one article, but below are a couple of the big ones that our current approaches to NLP seem no closer to solving.
Anaphora is, per the Oxford Languages definition, ‘the use of a word referring back to a word used earlier in a text or conversation, to avoid repetition’. While we humans can easily make connections when we hear the words ‘she’, ‘them’, or ‘that’, it doesn’t seem to be quite as simple for our computers. Even with the large swathes of data, huge processing power, and decades upon decades poured into NLP research, the best models still don’t produce results that show a convincing grasp of this concept.
With the latest and greatest models sourcing text from all over the place (novels, research papers, and the filthy broader Internet) it has become more and more challenging to filter and shield machines from our inherent human biases. A good illustration of this is the BERT Fill-Mask Model on Huggingface — which when posed with the prompt…
He works as a _____.
…gives the suggestions ‘carpenter’, ‘waiter’, ‘barber’, ‘mechanic’ or ‘salesman’ but for…
She works as a _____.
…suggests the words ‘nurse’, ‘waitress’, ‘maid’, ‘prostitute’ or ‘cook’.
Final Thoughts
While we are still a ways away from the intelligent machines first dreamed up by Alan Turing, the world of NLP has developed far from where it started. The conversational AI of today holds a lot of promise, and given the right use case, a chatbot could very well be the thing you need to achieve your business goals.
It goes without saying that there is no sense in waiting for perfect — the best time to start building and solving is today.
More ideas and insights
