NOT ACCOUNTING FOR BIAS IN AI IS RECKLESS Posted on : May 16 - 2019

I’ll never forget my “aha” moment with bias in AI. I was working at IBM as the product owner for Watson Visual Recognition. We knew that the API wasn’t the best in class at returning “accurate” tags for images, and we needed to improve it.

I was nervous about the possibility of bias creeping into our models. Bias in Machine Learning (ML) models is the exact sort of problem the ML community has seen time and again, from poor facial recognition of diverse individuals to an AI beauty pageant gone awry and countless other instances. We looked long and hard at the data labels we used for our project and, at first blush, everything seemed fine.

Just prior to launch, a researcher on our team brought something to my attention. One of the image classifications that had trained our model was called “loser.” And a lot of those images depicted people with disabilities.

I was horrified. We started wondering, “what else have we overlooked?” Who knows what seemingly innocuous label might train our model to exhibit inherent or latent bias? We gathered everyone we could — from engineers to data scientists to marketers — to comb through the tens of thousands of labels and millions of associated images and pull out everything we found objectionable according to IBM’s code of conduct. We pulled out more than a handful of other classes that didn’t reflect our values.

My “aha” moment helped avert a crisis. But I also realize that we had some advantages in doing so. We had a diverse team (different ages, races, ethnicities, geographies, experience, etc.) and a shared understanding of what was and wasn’t objectionable. We also had the time, support, and the resources to look for objectionable labels and fix them.

Not everyone who is building an ML-enabled product has the resources of the IBM team. For teams without the advantages we had, and even for organizations that do, the prospect of unwanted bias looms. Here are a few best practices for teams of any size as they embark upon their ML journey. Hopefully they help avoid unintended negative consequences like those we almost experienced. View More