Top 3 Pitfalls to Avoid when Deploying a New Chatbot

Share this 

Share on facebook
Share on twitter
Share on linkedin

The majority of B2C enterprises are now working on deploying Conversational AI products, for both internal and external facing use cases.  With the relative immaturity of the industry, companies are still being caught off guard with the amount of work that can be required to deliver a successful virtual assistant.

To speed up sales, many vendors are selling on a message of “this isn’t any work at all” which is obviously not true. This is the snake oil of Conversational AI and is hurting the whole industry.  By going into a project without the proper expectations, companies are being caught off guard with the amount of effort required to actually deliver a great customer experience.

Here are the top 3 pitfalls to avoid when starting a Conversational AI project.

1. Forgetting to gather system feedback or “close the loop”

One of the ways AI in the enterprise is different from all of the enterprise software delivered in the last 20-30 years is that it requires a feedback mechanism.  Whatever function you are using, AI will start off at some rate of success and will improve over time. This is very different from the software world we’ve all become accustomed to.  In the traditional software world, we gathered requirements, built the best product in a certain amount of time, tested and released it into a production environment. Now, in the Conversational AI world, it is very different.  

In AI, you create a deep learning model that will make a prediction to support your business.  For example, a model that determines if you should make a car loan to an applicant would predict the probability of them paying you back..  In the first iterations of the model created to make this prediction, the performance will be weak. You will then optimize the model, feeding more training data and adding pre and post processes to improve accuracy.  The key to AI is that this process will never end. You will reach a point where the predictions are accurate enough to release your AI to a production world, but that isn’t the end of optimization.

To enable this AI “training” in the enterprise, you need a closed loop system.  You need to find clues that may not be explicit in order to tell you if the model succeeded.  If you make tens of thousands of predictions with your model, and have the ability to know the accuracy of these predictions, you can use that feedback to drive continuous improvement.

If not considered prior to your system deployment, this will add considerable cost that could have easily been avoided.  This isn’t needed in traditional software deployments, so is overlooked many times.

2. Not analyzing the data from the AI feedback loop

At this stage of a deployment, where you have a mechanism to automatically take “clues” from your system and use that to determine if the model succeeded, you can also burn a lot of cycles.  In many cases, this feedback loop data isn’t clear and you may need to build additional models to cluster and analyze it. You’ll need to wrap the core model with other operational data to determine how to improve.

Let’s go back to the above example in reference to making loans to buy cars – in this particular scenario, you may be presented with data showing many default fields, and then add all of the other customer profile information that is available.  There will be thousands of opportunities to improve the core prediction model, but where should you start? What you need is a clustering system that will identify prevalent failure patterns that you should focus on eliminating from your prediction model.  We call these “trainable events”.

Developing a system that automates as much as possible without creating operation overhead will take time and another type of resource.  If you don’t have this planned from the beginning, you will be caught off guard and spend more time and money than necessary.

3. Failing to tune & test the models

You’ve finally reached the part of AI where you actually retrain your models.  You’ve closed the loop and built practical analytics to lead you to the most effective changes you can make.  Now, you want to influence as many model changes as possible without moving the performance backwards. This is one of the more difficult parts of using deep learning. In some cases, when you make changes to a model or introduce new training data, the system will actually perform worse than before!

To solve this problem, you’ll need tools that let you efficiently make changes and then run automated testing cycles.  This will allow you to make many changes and determine what impact they will have on performance. This must be done quickly and frequently. Using the right tools will enable an efficient training and testing cycle.

Anyone who has worked in enterprise software knows that delivering a quality system can consume a lot of QA resources.  To maintain the speed of system improvement and avoid wasting a pile of money , you’ll need to have planned and built these systems long before they are needed. This is the third pitfall to avoid if you want to keep your Conversational AI system learning and satisfying customers.

This all leads to what we call “AI Operations”. At Wysdom, our tools and experienced AI supervision team can efficiently manage an AI application.  This is rarely mastered by a new team within an organization that has expertise in other areas. 

To find out more about common AI pitfalls and how to prevent them, download our Whitepaper: 5 Reasons Conversational AI Fails and One Way to Guarantee Success

We use cookies to ensure that we give you the best experience on our website. By clicking “I Accept” or if you continue to this site, we will assume that you consent to the use of cookies unless you have disabled them.

Data Scientist

As a Data Scientist, your day-to-day will involve writing queries, building dashboards, and preparing analytical reports about product performance for our clients and the Wysdom.AI team.  You will work with SQL, Tableau, and Python and ML Frameworks/libraries (among other tools) to create stunning visuals showing how Wysdom.AI is making their customers’ experience even better.

Implementation Engineer

As an Implementation Engineer you will be responsible for the solution integration of an enterprise grade conversational AI experience from a technical perspective. You will work closely with the lead solutions architect and be the technical face of the implementation team and lead the customer through the entire implementation cycle. You will work with one of the most diverse teams of linguists, data scientists, and innovators to deliver the best AI enabled customer experience.

Solution Architect

As a Solutions Architect within the Client Services team, you act as trusted advisor, responsible for the technical requirements and end to end solutions integration of Wysdom cognitive services within the client’s environment.  You will work with one of the most diverse teams of linguists, data scientists, and innovators to deliver the best AI enabled customer experience.

Cognitive Data Specialist

As a Cognitive Data Specialist, you will be responsible for the performance of the AI and quality of the corpus and will focus primarily on the VA training.  You will work with the client as required to ensure corpus is performing in an optimal manner.

Conversational Experience Designer

As a Conversational Experience Designer, you will be responsible for the designs of the overall customer experience, including the end-to-end dialog flows & journeys of the solution ensuring  design leverages  UxD best practices for optimal customer experience.

Conversational AI Specialist

As a Conversational AI Optimization Specialist, your responsibility will be to help drive the success of our solution for our clients. This involves building conversation flows, performing AI training, and partnering with clients to enhance their deployments.

Conversational AI Lead

As a Conversational AI Lead, you will be responsible for leading all Conversational AI program activities.  You will work with all team members to ensure deliverables are completed on time, with high quality and exceeds client expectations and goals.

Program Director

Responsible for the overall success for the client, including the end-to-end delivery and optimization of the solution, you will manage the sales process from pillar to post, including technical and commercial proposals, pipeline management, sales forecasting, and contractual documentation.

David Trotter, Wysdom

David Trotter

SVP, Sales & Marketing

David has 30 years of global sales leadership experience as a collaborative leader who believes in a strong team concept within sales and marketing organizations. David has spent many years working with growth companies and enjoys being face to face with customers and partners to create solutions that have a lasting effect on the customer’s business environment. 

Prior to joining Wysdom, David was the Vice President, Sales at Scalepad, and previously spent 11 years as Vice President for Latin America and Asia Pacific for Absolute Software. He also held senior sales management positions at GE Capital and Clevest.  

Michel Benitah

VP, Optimization & Delivery

Michel has 20 years of experience in leading the successful delivery of Conversational AI and Natural Language Customer Care solutions to some of the largest financial, telco, healthcare, utilities, and retail enterprises throughout North America. 


Prior to joining Wysdom, Michel spent 20 years at Nuance Communications, holding senior management and leadership positions within the enterprise division, most recently as director of the Toronto office and professional services team.

Frederic Lam

SVP, Sales

Fred brings in 25 years of international experience in sales and business development across North America, the Caribbean, Asia-Pacific, Europe, and the Middle-East.


Prior to Wysdom.AI, he held sales leadership positions at Oracle, Redknee, and Movius/Glenayre, successfully growing revenues in both large and small organizations. Fred has also been involved in the start-up community in the earlier stages of his career as an Investment Manager with SP Capital and was an alternate director on a few investee companies.

Karen Chan

Chief Engineering Officer, Co-Founder

With 20 years of experience in software and mobile, Karen has held senior technical roles at 5 startups, including Wysdom.AI, Clickfree, Mobile Diagnostix (HP), Teamatic, and Virtualthere.

Karthik Balakrishnan

Chief Technology Officer

Karthik has over a decade of hands-on, proven global expertise in emerging technologies and implementing intricate platforms and solutions for telecommunications and enterprise during his time at Amdocs, with senior positions in their India, Cyprus, America, and Canada offices.

Nitin Singhal

Chief Operating Officer

Nitin has over 20 years of success in global executions of business technology, driving operational efficiency and digital scalability for some of the world’s largest enterprise clients. 


Nitin spent 16 years at Redknee holding executive positions in Research and Development, Customer Operations, Partner Alliances, and most recently as COO.

Jeff Brunet​

President, Co-Founder

Jeff has more than 20 years of experience in the startup world, founding and growing 4 software companies: AracNet, Mobile Diagnostix (HP), ClickFree, and Wysdom.AI. 


His in-depth understanding of software development and the challenges in making new technologies successful in the startup world prove invaluable as he serves on the boards of XMG, SurfEasy (Opera), Locationary (Apple), Groupie, and as an advisor to Pushlife (Google), LogMeIn (IPO) and HP. 


Jeff holds 23 issued patents in the wireless and consumer electronics spaces and is the lead inventor on 30+ pending patents.

Ian Collins​

CEO, Co-Founder

Ian has founded and grown 6 technology companies over the past 20 years, primarily in the enterprise software space including Wyrex, Mobile Diagnostix (HP), Clickfree, and most recently Wysdom.AI. 


Ian invests, mentors, and sits on the boards of several startups in the Toronto area.