3 Painful Mistakes To Avoid When Buying AI Solutions

85% of global executives believe that AI can become their competitive advantage. So, the rush to AI adoption is understandable. Unfortunately, implementing AI from scratch takes time, and success comes with experience in building and deploying solutions. 

To speed things up, “buying” instead of building from scratch seems like a sensible way to get started; You don’t have to hire a team of data scientists, spend on additional infrastructure, or have support staff on call to troubleshoot model problems. Plus, off-the-shelf solutions are often rigorously tested.

But buying AI solutions is tricky. If you don’t pursue it strategically, you could be buying your way into trouble. Here are three mistakes I see companies make (which you should avoid) when considering third-party AI solutions. 

1: Not testing vendor performance 

Performance, what performance, you may be thinking? This is the accuracy or error rate that vendors publish to sell their AI solutions. For example, a vendor may claim a 3% word error rate in automatic speech recognition. Often, you’ll find that vendors tout shiny performance numbers. By itself, this is not a problem, it gives you an idea of how usable the solution is. The problem really starts when you take these numbers at face value.

The data that vendors use to test their solution, can look very different from your company data. Let’s take a sentiment classifier (sentiment classifiers predict sentiment polarity such as positive/negative on some given text). Say a third-party API boasts 98% accuracy on a user reviews dataset, but your plan is not to use it on user reviews. You want to use it for forum comments. Do you think that the 98% accuracy will still hold? It’s unlikely. 

Action Tip: To avoid buying into a solution that works on paper, but not on your data, test, test, test—on your own data before making a purchase decision. Use additional metrics that make sense for the problem, and do a side-by-side performance comparison of different vendor solutions on YOUR data.

2: Not thinking about long-term costs and risks

One significant advantage of buying a solution vs. custom development is cost. It’s often cheaper to buy an off-the-shelf solution than to custom build, especially in-house. It’s also more convenient as you don’t have to put together a team for development. But is this truly more cost-effective?

If you skipped the testing step above, your AI solution could be error-prone even without you knowing it. And with that comes the risks. Are you losing customers because of bad predictions? Are customers constantly complaining due to problems introduced by your AI solution? This not only costs customers, but also your reputation. 

Also, the pricing of third-party solutions can change with time, and some solutions need specialized infrastructure, which can be hard to maintain. Navigating through such issues would require additional investments. While you thought you got a good deal, costs may be quietly adding up.

Action Tip: Think through the potential risks of the third-party AI solution in consideration and have a fallback plan. The costs and performance of the AI solution may be justified today. But long-term costs due to lower than desired accuracy, additional infrastructure requirements, and pricing changes may require that you switch paths. Having a plan B can help you act swiftly.

3: Not considering business systems compatibility

When evaluating third-party AI solutions, many companies go with solutions that use the latest techniques or with the highest published accuracy. But this may not always be the right move for your organization. Third-party AI solutions come in all shapes and forms. Some are available as a cloud service. Some require code integration with your existing software systems. 

Without compatibility between the consumer application and the AI solution, you’d have to jump through many hoops to get the consumer application and the AI solution talking. If you find that integration is impossible or just too expensive after you’ve purchased the solution, it’s too late. You risk wasting the subscription or license altogether.

Action Tip: When testing different vendor solutions, try to attempt light integration with a handful of shortlisted solutions. This will help guide your purchase decision to solutions that not just perform well but are also easy to integrate.   

Final Word

Buying off-the-shelf AI solutions can be a convenient option to hit the ground running with AI. But doing it strategically is essential. High vendor published accuracy doesn’t automatically mean high accuracy on your company data. Or a cheap off-the-shelf AI solution does not mean it’s truly more cost-effective. This is why it’s best to dig deeper as you’re shortlisting solutions.

Use the action tips from above as a starting point to guide your purchase decisions. Establishing performance benchmarks, investigating integration compatibility, and analyzing costs and risks are all within your control. Plus, the groundwork that you lay today, will serve you in the long term, such as when you’re looking to revert to an in-house solution or want to consider other third-party options.