Chatbot Analytics — How To Measure The Success Of Your Chatbot

Your chatbot is built. You already have the first users. But do you know how successful your chatbot is? The implementation and launch of a chatbot are only the beginning. Once you have mastered this step, it’s about identifying metrics that will help you establish relevant chatbot analytics. That way, you can continuously measure the success of your bot and derive further improvement areas.

To help you get started, this article explains how to use metrics to measure chatbot success and which metrics are most relevant to Conversational AI analytics.

Download our free e-book to learn everything you need to know about chatbots for your business.

What is the difference between key figures and KPIs?

Before we get to the most important key figures for chatbot analytics, let’s first look at two terms that are important for a successful analysis, but are often mixed up: Key figures and KPIs.

Key figures are values that can be measured directly. For chatbots, this can be the number of conversations or the duration of the conversation. A KPI (Key Performance Indicator) goes one step further. The KPI defines a corresponding target value for the key figure and displays the deviation from this target value.

Let’s assume that you want to measure the key figure “number of new users”. You specify that you want to attract 500 new users by the end of the month. You got 200 users already. So you still have to get 300 new users. A KPI helps you to link the key figures with your individual goals and to measure the achievement of your goals.

 

3 key figures for your chatbot analytics

As you can see, selecting the right key figures is only the first step. You have to define your own individual target values. This is the only way to measure the success of your chatbot.

To help you get started, we have compiled three key figures that are relevant for Conversational AI analytics and offer you a good entry point to measuring your chatbot success.

 

1. Customer satisfaction

Customer satisfaction is a critical factor when it comes to measuring the success of chatbots in customer service. After all, customer satisfaction is what counts in the end. The measurement of customer satisfaction in a chatbot conversation is similar to the measurement of a conversation with a human employee.

It’s usually done via a short questionnaire after the conversation and it can cover questions, such as:

• Was the user able to understand the bot without any problems?
• Could the bot answer even very specific questions?
• Was it possible to talk to a human employee? And was the user forwarded to him if the bot did not understand a question?

Another possibility is to use chatbot ratings. After the conversation, you can ask the user to rate the conversation. This is what we use for some of our chatbots. Of course, the more detailed the feedback about why a conversation was good or bad, the better, but star ratings are well suited for an initial picture of your bot’s perception.

 

Chatbots, voice assistants and AI, stay informed with the free Onlim newsletter.

 

2. Completion rates

In addition to customer satisfaction, chatbots are primarily intended to help relieve customer service and save costs. Therefore, the completion rate is another key figure for measuring the success of chatbots. Although it should always be possible for a user to be forwarded to a human employee, ideally a bot should be able to handle customer queries on its own.

In addition to the completion rate, it also makes sense to capture the questions that are forwarded and derive further improvement potential from them.

 

3. New vs. returning users

The number of new users is a good indicator to measure the number of users for your bot. However, if you want to know more about user satisfaction, it is advisable to measure the number of returning users as well. As in an online shop, the number of recurring users is a good indicator of how satisfied the users are.

Set the number of new users and the number of returning users in relation to each other. This gives you an overview of whether the number of returning users also increases with the growth of your user base.

We hope this was useful to get you started with measuring the success of your chatbot and if you have any questions around chatbots and Conversational AI, we are happy to help.