A Buyer’s Guide to AI and Machine Learning Posted on : May 22 - 2020

B2B software sales and marketing teams love hearing the term “artificial intelligence” (AI). AI has a smoke and mirrors effect. It sounds impressive. But, when we say “AI is doing this,” our buyers often know so little about AI that they don’t ask the hard questions.

In industries like the DevTools space, it is crucial that buyers understand both what products do and what their limitations are to ensure that these products meet their needs. If the purpose of AI is to make good decisions for humans, to accept that “AI is doing this” is to accept that we don’t really know how the product works or if it is making good decisions for us.

When we’re in the buyer role, we often don’t hold ourselves responsible for understanding AI and machine learning (ML) products because these technologies are intimidating. They’re incredibly complex.

This article addresses the limitations of AI and ML, so software buyers can ask the right questions to understand what they are buying.

The Test Oracle Problem

One limitation of some AI or ML products is that for certain applications of the technology, there is no source of absolute truth to compare against the accuracy of the output. For example, neither humans nor machines know how to produce the perfect set of end-to-end tests for any given application. This is the test oracle problem: there is no objective standard of truth. No one wants to introduce this kind of uncertainty into their sales process. Yet, our buyers deserve well-informed answers about our products.

As a buyer, you need to understand the intended advantage of your seller’s AI product before making a purchase decision. Is it meant to make a decision that is more accurate—against an objective standard—than a human? Is it meant to make a faster decision with less cost? Or introduce an alternative methodology that uses new data in a new way? Answers to these questions influence how you will use the product and what value it provides.

AI Versus ML

Though AI is commonly accepted as “any machine that uses math to make decisions,” true AI is self-taught. AI has a neural net that mimics neurons in a human brain which allows it to teach, update and evolve itself. Because of this, true AI is difficult to build and is often experimental rather than commercial.

More often, what’s being described when we say AI is actually ML. ML is human-taught: Machines learn through human feedback using a probabilistic decision-making process that improves via ongoing correction. Machines take in data, run algorithms against it and output a decision — or series of assertions — based on probabilities. Humans correct the machine by telling it whether it was accurate in its assessment, and the machine updates. As it receives accuracy feedback, machines learn to make better decisions. And because ML is based on probabilities, it will sometimes make the wrong decisions. View More