AI Bias a Real Concern in Business, Survey Says Posted on : Dec 02 - 2019

The potential benefits of AI for businesses are huge. But those benefits could be eroded by hidden biases that damage brand reputation and customer trust, according to a survey of US and UK businesses commissioned by DataRobot.

According to DataRobot‘s “The State of AI Bias in 2019,” which was released last week, 42% of organizations surveyed reported being “very to extremely” concerned about AI bias occurring in their organizations.

This number jibes with another finding from the DataRobot survey: that 38% of the organizations surveyed reported they use “black box” machine learning systems that offer no insight into how it makes decisions.

The juxtaposition of AI bias concerns and black box systems is enough to warrant serious questions about the direction compaines should take with their machine learning, according to John Giannandrea, Apple’s senior vice president of machine learning and AI strategy.

“If someone is trying to sell you a black box system… and you don’t know how it works or what data was used to train it, then I wouldn’t trust it,” DataRobot quotes Giannandrea as saying in its report.

The survey indicates that organizations are aware of the potential pitfalls and are actively working to mitigate it. DataRobot found that 64% of survey respondents say they’re “very to extremely” confident in their ability to identify AI bias.

The survey found some interesting differences among US and UK companies. While UK companies were more concerned about “technical bias” in the ML algorithms than their US counterparts (47% compared to 34%, respectively), the survey suggests that US companies were more concerned about “emergent bias” than UK firms (35% versus 22%, respectively). View More