<script type="application/ld+json">
 "@context": "https://schema.org",
 "@type": "FAQPage",
 "mainEntity": {
   "@type": "Question",
   "name": "What are the most relevant message metrics which are believed to be relevant?",
   "acceptedAnswer": {
     "@type": "Answer",
     "text": "1. Conversation-starter Messages.
2. Bot Messages.
3. In Messages.
4. Miss Messages.
5. Total Conversations.
6. New Conversations."

Drive to Reimagine

A deeper understanding of bot metrics [Conversational analytics part 2]

Engati Team
Oct 14
3-4 mins

Table of contents

Key takeawaysCollaboration platforms are essential to the new way of workingEmployees prefer engati over emailEmployees play a growing part in software purchasing decisionsThe future of work is collaborativeMethodology

In this part of our blog, lets see how chatbot metrics, which you choose for your chatbot, influence conversational analytics, and set you up for chatbot success. If you are feeling lost about the relevance of metrics and conversational analytics for chatbots, you can find Part 1 of this blog here. Reading our blog on crafting conversational UI will weed out any other doubts that might have about the relevance of this topic.

Previously, we have discussed about the importance of measuring the performance/efficiency of your AI chatbot. We went further in detail about how choosing the right metrics is important to measure your chatbot’s effectiveness. We also discussed the broad classifications of metrics that we thought were important for measuring your chatbot’s performance.

  • User metrics
  • Message metrics
  • Bot metrics

Having discussed how user metrics can inform some of your analytical decisions, let us now discuss the importance and relevance of message and other key chatbot metrics.

A walkthrough on message metrics

While user metrics inform you about the overall trend in your user base, they are not enough to craft individual’s interactions with your chatbot. Message metrics will help you gather details that will help you in crafting individual experiences, speed up customer service, and increase engagement. Here are the ones we believe are most relevant.

Chatbot metrics influence conversational analytics - Engati
Photo by Georgia de Lotz on Unsplash

Conversation-starter Messages

This is the number of messages where the conversation is initiated by the chatbot. This is a crucial metric for measuring how organic your platform is. Compelling your chatbot users to engage with the bot by sending out messages is something one can do only in the initial stages. As time passes, organisations should focus on lowering this metric’s numbers. If your chatbot’s main focus is on digital marketing or customer relation management, it will serve you well to have the engaged users that interact with your chatbot directly.

Bot Messages

Bot messages are the number of messages sent by the bot during a conversation with active users. The more the number of messages sent by the chatbot in a conversation, the longer it is. Longer conversations are better in terms of metrics, however, care should be taken that the chatbot’s replies are all standard. We don’t want it stuck in a loop repeating the same message over and over, this generally happens when bots cannot comprehend the users’ input.

In Messages

This metric records the number of incoming messages from users in a conversation. This will inform you how engaged your users feel in the conversation, if it’s significantly low, you probably should rethink using a chatbot. In such cases you are better off using a social media page or account to reach out to your user base.

Miss Messages

This metric captures the number of messages that your chatbot could not process. This metric may be hard to calculate, you need to record all the times a message is misinterpreted by the chatbot. This becomes a key metric when you start engaging users from a different region than you currently serve due to the idiomatic use of language.

Total Conversations

This metric records all the successful conversations during a day. That means the conversation was started, relevant information was exchanged and the conversation reached a proper end. This will also point to the number of times your chatbot engaged users thoroughly.

New Conversations

This metric captures the total number of conversations started by both new and returning users. While new users are inexperienced with the platform, returning users might be interested in a different topic, a new problem or just about another order. User metrics and message metrics can act as key metrics for metric construction. This will in turn will facilitate the creation of other traditional digital marketing metrics.

Understaning Bot Metrics

Monitoring AI chatbot performance metrics is key when understanding conversational analytics. Only when you can measure how your bot responds in different situations, can you draw informed insights about its performance. Here are some metrics to consider.

chatbot metrics and analytics
Photo by Mark Finn on Unsplash

Retention Rate

This metric records the number of users that return to interact with your chatbot within a specific time frame. Since you need to engage your users for a longer time to generate insights about their preferences, this metric plays a very important role. You can achieve higher retention rates by making conversations more engaging or using promotional campaigns. If this can be achieved organically, it is the best. A high-quality chatbot that caters to the needs of customers can ensure higher retention rates.

Goal Completion Rate (GRC)

This metric is helpful when tracking the percentage of successful engagement through a chatbot. Every user is unique and will want to access different services and information on your website. In any given scenario, when the bot completes a task assigned to it the GRC increases. For example, if the bot can be responsible for delivering updates about an order, every time it completes this task, the GRC will improve. This metric basically allows you to record every time your bot receives an input and responds with relevant information.

Goal completion time and mode

This metric will record each time you accomplish a goal and the modes that you choose. This helps in informing your decisions about which are the best modes of interaction with your bot and fastest. This metric helps when you are trying to figure out new layouts. Minimising the effort in goal completion can improve your customer experience.

Fall Back Rate (FBR)

The FBR metric records the number of times your bot has failed or reached a near failure situation. Generally, we prefer lower FBR, in case of higher values for FBR consider changing the source of data. For more on chatbot automation, please visit us at Engati

Haven’t built a bot yet? We can fix that right away, click here to begin your bot building journey!


Engati Team

At the forefront for digital customer experience, Engati helps you reimagine the customer journey through engagement-first solutions, spanning automation and live chat.

Andy is the Co-Founder and CIO of SwissCognitive - The Global AI Hub. He’s also the President of the Swiss IT Leadership Forum.

Andy is a digital enterprise leader and is transforming business strategies keeping the best interests of shareholders, customers, and employees in mind.

Follow him for your daily dose of AI news and thoughts on using AI to improve your business.

Catch our interview with Andy on AI in daily life

Continue Reading