Introduction
Customer experience (CX) can be defined as the perception and impressions a customer has of their interactions with your business. CX is the result of every touchpoint a customer has with a company, from navigating the website, to talking to customer service and receiving the product/service they bought.
As companies across the world embed conversational AI technologies in their processes, the CX delivered to their customers becomes transformed as well. When this happens, questions like “are my customers more satisfied with our conversational bot or my agents?” or “What KPIs should I measure?” tend to arise.
This article gives an overview of how CX should be measured in the context of a Conversational AI solution.
Conversational AI and CX
Generally, CX can be divided into three components: Success (were customers able to accomplish what they wanted to do?), Effort (how easy or difficult was to accomplish their goals?) and Emotion (how did the customer feel during the interaction?).
Since Conversational AI is such a versatile technology that integrates with all business communication processes, companies need to consider how adopting it will reshape each of the three components of CX rather than merely affecting them. This means evaluating each dialogue and analyzing if using Natural Language is the optimal experience for a customer:
- Can buttons or forms be used in a chat to collect information instead of asking questions? (Effort),
- Is this use case best supported by a conversational Bot? (Success),
- Should there be any handover to a human agent if the customer is upset?(Emotion)
This is the first step to take before measuring any KPI, mainly because it will indicate companies where and how they should approach CX measurements. In addition, it will also condition the scope of a project and will also be intrinsically related to the value the project is delivering.
Measuring CX
Conversational AI Projects usually have their own set of KPIs generally associated to UX and performance even though some of them can also be used to get an understanding of user experience (like session duration)
KPIs used to measure CX however, do not differ with those used by companies since customers are the same and want the same thing, regardless on how they contact a company. A common practice is to measure the same KPIs across different channels to compare their performance. For example, if a company sends a small survey after a chat with an agent to measure Customer Satisfaction, it would make perfect sense to measure the same indicator after a dialogue has been ended with a conversational bot.
Keep in mind that if the same indicator is going to be compared across channels, the criteria by which its measured should be equal: If an NPS survey is launched in a call with an agent after the user has indicated they have achieved what they wanted, then the NPS survey for the bot should be triggered at a point in the conversation flow where the bot gives the same answer/functionality as the agent would have.
The table below summarizes the most common KPIs used to assess CX, most of which can be measured in Teneo by using Metadata and by analyzing the conversation logs using Teneo Query Language.
| KPI | CX Component Impacted | KPI Description |
|---|---|---|
| Customer Satisfaction Score (CSAT) | Success, Emotion, Effort | Measures the short-term happiness and experience of your customers |
| Net Promoter Score (NPS) | Success, Emotion, Effort | NPS measures advocacy, i.e how likely they will recommend you. It is used for benchmarking with other companies and also an indicator of potential growth |
| Customer Effort Score | Effort | Usually the question: “How easy was it to deal with our company today?” Is used to assess how much customers are struggling to get their needs met |
| Time to Resolution | Effort | Time to resolution is an average of the amount of time taken to resolve a customer issue. It is highly relevant when measuring savings and its usually measured within Average Handling Time (duration of a conversation) |
| User Feedback | Success, Emotion, Effort | Collecting feedback from open questions, yes/no questions or doing is a great source of insights and one of the richest insight sources for understanding CX |
| Knowledge Coverage | Success, Emotion, Effort | % of user inputs that are properly covered in the knowledge base or were understood correctly. It is an indicator that whatever the user is trying to achieve is covered (See what can be done when a user is out of the initial bots’ scope. |
Conclusions
Measuring CX is important since it’s usually one of the main drivers in implementing CAI projects and it’s a great way to quantify its impact and evaluate its ROI. Since each company’s CX is unique, there is no “one size fits all” KPI for all companies. However, as long as customers’ success, effort and emotion are taken into account, companies will have an accurate indicator of the perception and impressions a customer has of their interactions with your business.
Regardless of what indicator is used, it is very important that the experience delivered by a conversational bot is better than (or at least as good as) the one delivered through the existing channels as it is the only way to generate traction and adoption within customers. To achieve this and truly reap the benefits of Conversational AI, it is fundamental to think of how this technology will transform a company’s customer experience in every single interaction of a conversation.
What about you? How do you measure CX in your conversational AI projects? Do you use any of the KPI mentioned above? Let us know by completing a quick poll below
- Customer Satisfaction
- Customer Effort Score
- Net Promoter Score
- Time to Resolution
- User Feedback
- Knowledge Coverage
- None
- Other (If you chose this, let us know in the comments below which KPI you are using)
0 voters