What you need to know about Conversational AI performance guarantees

Share this 

Share on facebook
Share on twitter
Share on linkedin

For starters, Conversational AI performance guarantees are not about uptime or other traditional SLA metrics. You’ve implemented a Conversational AI to satisfy customers, and that is what should be guaranteed.  You want to measure the output of the AI and how well it performed at its intended task. In the virtual assistant world, this translates to how often it understands and satisfies your customers correctly. As AI continues to advance, these core metrics will continue to impress.

For NLU accuracy, the amount of time your system will correctly understand the customer, you should expect at least 80%. Now, don’t get caught by the sneaky tactics of some vendors who will only offer a performance number for specific “use cases” – this number should apply to absolutely every customer interaction with your brand about anything at all. The next metric is a percentage of resolved customer issues before a transfer to a live agent is required.  This is case dependent but should fall between 25 and 35%. Again, don’t get stuck on specific “use cases”, your customers will come with all of their questions and your virtual assistant should perform well across them all.

As cliche as it may sound, AI is a journey and not a destination. It needs to continuously learn to improve and get better. One analogy we like to use is comparing an AI to a child. Children need a lot of attention, as well as great teachers to help them reach their full potential. If left to themselves, they’ll devolve into something like Lord of the Flies. The key takeaway here is that they do not learn on their own, but rely on great teachers to help them along the way.

Many will argue that self-training AI is on the horizon, but having worked in this space for the last 5 years, we are still many years away from truly automated self-learning in the enterprise market.  Black boxes (automated training or deep learning) scare people in this industry, they need 100% control over every message that a customer will receive. So, we’re going to be using supervised learning with people behind the AI training for the foreseeable future.

Any competent vendor can source and install technology. But teaching is something new to the tech industry. It requires a new type of team, a new set of analytics, and a solid set of training tools. Just like any teaching environment, someone needs to set the curriculum and expectations, and continuously teach, monitor, and optimize to constantly improve.

We take pride in being great teachers of the AIs under our care, and have equipped our teams with the best tools, technologies, and AI data library to confidently remain dedicated to our goal of continuous improvement. This assures we can guarantee performance levels for our clients, and provide the white glove service our customers expect.


0 replies on “What you need to know about Conversational AI performance guarantees”

Let’s supercharge your virtual agent together

With outstanding customer experience being the most important differentiator, make sure your virtual agent is the one leaving the competition in the dust.

Wysdom is a conversational AI optimization platform that enables higher performing, lower cost conversational virtual agents, continually learning and delivering high-quality customer experiences.

© Copyright Wysdom 2021. All rights reserved.

Artiom Kreimer

VP, Product & Analytics

Artiom has spent 10 years in software and mobile engineering, specializing in quality assurance and customer service. He has worked in testing and QA at both startups and in enterprises such as Clickfree, TELUS, and Freescale Semiconductor.

Michel Benitah

VP, Optimization & Delivery

Michel has 20 years of experience in leading the successful delivery of Conversational AI and Natural Language Customer Care solutions to some of the largest financial, telco, healthcare, utilities, and retail enterprises throughout North America. 


Prior to joining Wysdom, Michel spent 20 years at Nuance Communications, holding senior management and leadership positions within the enterprise division, most recently as director of the Toronto office and professional services team.

Frederic Lam

SVP, Sales

Fred brings in 25 years of international experience in sales and business development across North America, the Caribbean, Asia-Pacific, Europe, and the Middle-East.


Prior to Wysdom.AI, he held sales leadership positions at Oracle, Redknee, and Movius/Glenayre, successfully growing revenues in both large and small organizations. Fred has also been involved in the start-up community in the earlier stages of his career as an Investment Manager with SP Capital and was an alternate director on a few investee companies.

Karen Chan

Chief Engineering Officer, Co-Founder

With 20 years of experience in software and mobile, Karen has held senior technical roles at 5 startups, including Wysdom.AI, Clickfree, Mobile Diagnostix (HP), Teamatic, and Virtualthere.

Karthik Balakrishnan

Chief Technology Officer

Karthik has over a decade of hands-on, proven global expertise in emerging technologies and implementing intricate platforms and solutions for telecommunications and enterprise during his time at Amdocs, with senior positions in their India, Cyprus, America, and Canada offices.

Nitin Singhal

Chief Operating Officer

Nitin has over 20 years of success in global executions of business technology, driving operational efficiency and digital scalability for some of the world’s largest enterprise clients. 


Nitin spent 16 years at Redknee holding executive positions in Research and Development, Customer Operations, Partner Alliances, and most recently as COO.

Jeff Brunet​

President, Co-Founder

Jeff has more than 20 years of experience in the startup world, founding and growing 4 software companies: AracNet, Mobile Diagnostix (HP), ClickFree, and Wysdom.AI. 


His in-depth understanding of software development and the challenges in making new technologies successful in the startup world prove invaluable as he serves on the boards of XMG, SurfEasy (Opera), Locationary (Apple), Groupie, and as an advisor to Pushlife (Google), LogMeIn (IPO) and HP. 


Jeff holds 23 issued patents in the wireless and consumer electronics spaces and is the lead inventor on 30+ pending patents.

Ian Collins​

CEO, Co-Founder

Ian has founded and grown 6 technology companies over the past 20 years, primarily in the enterprise software space including Wyrex, Mobile Diagnostix (HP), Clickfree, and most recently Wysdom.AI. 


Ian invests, mentors, and sits on the boards of several startups in the Toronto area.