Most companies adopt Conversational AI (CAI) technologies to fulfill their visions for enhanced CX, seamless interaction and single contact resolution. This vision can be easily achieved when a project revolves around one conversational bot but how is this accomplished when the number of bots involved in the same conversation with the same end user increases?
This is where the need of orchestration arises. In some companies, orchestration is set as a milestone on their CAI roadmaps. However, handling a dialogue between two different bots and the same user is not as simple as transferring a call between agents. Embarking on an orchestration journey opens the door to a new degree of complexity in terms of conversation design, integrations and intent recognition that should be carefully addressed.
This article aims to give readers some considerations on when to adopt an orchestration strategy within their CAI journeys.
What is Orchestration?
Bot orchestration involves developing strategies, architecture, and integrations to automate and coordinate interactions between one user and more than one bot in the same conversation.
When is Orchestration necessary?
“How many employees check their holiday balance (HR Bot) and report an issue with their computer (IT Bot) in the same conversation?”. Most cases in which Orchestration is necessary tend to rise from a technical or organizational requirement rather than a desired design.
There is a tendency to think that a company should have different Subject Matter Expert bots to handle specific knowledge domains but having a variety of intents handled by the same bot and ML model might perform better than having separate ones. Even if there are technical limitations to a model you can connect different models to the same bot.
There are cases however, in which Orchestration is unavoidable. One of the most common is when business units chose their own technologies for developing their conversational bots or inherit bots as a byproduct of organizational restructuration or mergers with other companies. This can potentially create a multi bot ecosystem in need of coordination.
Another case occurs when companies do not centralize their CAI Development in one area but have different teams within a department developing different Subject Matter Expert bots that ultimately serve the same end users. Though this can be solved by choosing a platform with collaboration features like Teneo, it is often necessary to “make the bots talk between each other” .
As it can be seen, in most cases the need for orchestration ends up being a result of a lack of a long-term, cross department and holistic CAI strategy. So, companies that are entertaining this idea should first stop and consider if orchestration is essential or if this can be avoided by redesigning and optimizing their bots.
Challenges in Adopting an Orchestration Strategy
Though an occurring reality, Conversational Bot orchestration is far from what everyone imagines and sees in movies where humans interact with machines that then control other machines in a seamless automatic and instant way.
A successfully orchestrated conversation requires the consideration of several factors:
-
Conversation Design: Does the conversation need to move back and forth between different bots? At what point is the conversation passed from one bot to another?
-
Intent Recognition: How are very similar intents going to be handled? (Who’s handling the conversation if the user states “I have a problem” ? Which bot is going to handle inputs common to both domains? What business or linguistic criteria is used for the knowledge domains?
-
Model Optimization: Managing Machine learning models from different bots imply maintaining and optimizing each one, but also making sure they work well together. This implies efforts and resources that should be taken into account. A clear definition is required to understand how the management of out of scope inputs and adding new use cases will be handled.
-
Expansion to new languages: Even though Teneo has native features for rapidly scaling languages, there is still an investment of time and resources that needs to be made to replicate the orchestration logic for each new language. An interesting approach to handle this would be using the multilingual localization capabilities from CLU.
-
Integration between technologies: If bots are developed with different frameworks and technologies, how easy and scalable is it to integrate this?
-
Quality Assurance: The QA process becomes more complex as new tasks are added to make sure the orchestration logic works as designed.
Once again, most of these factors are boxes to check when defining a CAI Strategy
Orchestration Approaches
There are two main ways of handling this logic: either using an Orchestration Bot or creating an Orchestration Layer.
Orchestration Bot
This approach consists of a a fully functional main bot which has a direct connection to further child bots in order to consult those for their best answers whenever required. The orchestration bot can have its own knowledge base to answer questions without help of the connected child bots. It can also establish the rules when child bots are consulted and chose which possible answer will be displayed back to the user in the end. The criterion of the answer selection is normally based on the confidence score of the machine learning models of all participating bots. This alternative can be easily developed with Teneo.
Orchestration Layer
This approach involves having a routing logic layer above all the bots which takes care of sending out the user input to all available bots in order to be able to take a decision which bot should answer the user’s request. This decision is taken based on the confidence score of the connected machine learning models and the returned confidence scores. Microsoft offers several services which can help you to set this up in a nice way, such as Orchestrator and Orchestration Workflow.
Conclusion
Companies should avoid starting their projects with the goal of having an orchestration bot with many Subject Matter Expert bots, but rather know that it is a complex endeavor that can be achieved if the need arises. Most importantly, companies should focus first on developing a long-term, cross-department CAI strategy to determine if an orchestration approach is truly necessary.
Have you been faced with the need to orchestrate? Is Orchestration in your plans? Let us know in the comments below
#teneoTuesday