How To Evaluate AI Software Posted on : Jul 10 - 2021

Buying off-the-shelf AI (Artificial Intelligence) software is a good first step for those companies that are new to the technology. There should be little need to make investments in technical infrastructure or to hire expensive data sciences. There will also be the benefit of getting a solution that has been tested by other customers. For the most part, there should be confidence in the accuracy levels as the algorithms will probably be implemented properly.

But there is a nagging issue: there are many AI applications on the market and it is extremely difficult to determine which is the best option. After all, it seems that most tech vendors are extolling their AI capabilities as a way to stand out from the crowd.

Then what are some factors to consider when evaluating a new solution? Let’s take a look at the following:

Data Connectors: AI is useless without data. It is the fuel for the insights.

But when it comes to a new AI solution, it can be tough to find the right sources of data, wrangle it and integrate it. Thus, when evaluating an application, you need to make sure that there are ways to handle this process.

“The most complex task in an AI solution is not to implement the machine learning algorithm anymore—this is usually available as a set of functions in every tool—but to collect the data,” said Rosaria Silipo, a Ph.D. and a principal data scientist at KNIME. “That is, to connect to a variety of data sources, on premise, on the web, or on the cloud, and extract the data of interest.”

Flexibility: AI does not have general scope. Instead, it is focused on particular use cases. This is known a “weak AI.”

This is why it is important to see if the application is built to handle your particular vertical or situation.

“Take search, for instance, where AI can be used to re-rank results and improve relevance,” said Ciro Greco, who is the Vice President of Artificial Intelligence at  Coveo. “When applied to ecommerce, search is searching on semi-structured records, such as products with little text available and we can count on reasonable amounts of behavioral data produced by users who browse the website. A strategy based on user behavior can be very effective, because we can count on having enough data to learn from.” View More