Bot essentials 10: The NLU deepdive – Entities and Intent
Getting the NLU aspect right is key to designing a conversational experience for your customers. If your chatbot cannot understand a sentence or the underlying intent it will lead to a default or a very frustrating experience for the user.
The elements of Natural Language Understanding (NLU) is the next step in our journey of understanding how bots work.
The The NLU algorithm is based on 3 concepts of language construction:
It is essential in understanding the construct of the sentence and the meaning behind it. Entities are mentioned in the sentence on who an action needs to be performed.
E.g. How much is in my checking account? The entity here is a checking account which gets extracted by NLU techniques. We will talk about stemming and lemmatization in future blogs. At this point of the journey, let's just assume that there are algorithms that will extract entities from sentences provided to them.
The intent here is to know the balance
So if entity is the checking account and the intent is getting the balance. Figuring out the intent is one of the most important aspects of NLU. Just analysing words only gets us so far. Consider these 2 sentences-
- I need to make a reservation at an Italian restaurant
-I need to book a table at the pizzeria
These 2 sentences mean the same but are constructed very differently. But, the intent is still the same- to "book a table". How do we have an intelligence engine understand the “intent” of the sentences?
Enter word semantics, statistics and word vectors.
Each word is a vector, an array of numbers. As is the nature of vectors, they have weight and direction. You can measure the distance between 2 words using the concept of vectors.
Similar words have vectors, that are close to each other. The distance between the vectors for bicycle and motorbike is closer than bicycle and horse.
While they're good at comparing words, how do we use them to compare phrases? That is relatively simple since you can perform arithmetic operations on vectors. You can derive sentence vectors, by averaging the word vectors but from our experience it does not work well in most cases. It's difficult to average words to get an aggregate meaning of sentences. At best it can be an aggregation.
Now that we have sentence vector weights as well as distance from other similar sentences, we can use semantics to figure out closely matched sentences to what the user is asking the machine. The machine thus understands and learns using similarly matched words and groupings using the science of statistics, probability and vectors.
Engati is a one-stop platform for delighted customers. With our intelligent bots, we help you create the smoothest of Customer Experiences. And now, we're helping you find those customers too. The award-winning Marketing Automation platform, LeadMi, received some major upgrades and joined our family as Engati Acquire. So, let's get started?Get Started Free