Your AI skills are worth less than you think Posted on : Feb 16 - 2019

We are in the middle of an AI boom. Machine Learning experts command extraordinary salaries, investors are happy to open their hearts and checkbooks when meeting AI startups. And rightly so: this is one of those transformational technologies that occur once per generation. The tech is here to stay, and it will change our lives.

That doesn’t mean that making your AI startup succeed is easy. I think there are some important pitfalls ahead of anyone trying to build their business around AI.

The value of your AI skills is declining

In 2015 I was still at Google and started playing with DistBelief (which they would later rename to TensorFlow). It sucked. It was painfully awkward to write, the main abstractions didn’t quite match what you expected. The idea of making it work outside of Google’s build system was a pipe dream.

In late 2016 I was working on a proof of concept to detect breast cancer in histopathological images. I wanted to use transfer learning: take Inception, Google’s best image classification architecture at the time, and retrain it on my cancer data. I would use the weights from a pretrained Inception as provided by Google, just changing the top layers to match what I was doing. After a long time of trial and error in TensorFlow, I finally figured out how to manipulate the different layers, and got it mostly working. It took a lot of perseverance and reading TensorFlow’s sources. At least I didn’t have to worry too much about dependencies, as the TensorFlow people mercifully prepared a Docker image.

In early 2018 the task from above wasn’t suitable for an intern’s first project, due to lack of complexity. Thanks to Keras (a framework on top of TensorFlow) you could do it in just a few lines of Python code, and it required no deep understanding of what you were doing. What was still a bit of a pain was hyperparameter tuning. If you have a Deep Learning model, you can manipulate multiple knobs like the number and size of layers, etc. How to get to the optimal configuration is not trivial, and some intuitive algorithms (like grid search) don’t perform well. You ended up running a lot of experiments, and it felt more like an art than a science.

As I am writing these words (beginning of 2019), Google and Amazon offer services for automatic model tuning (Cloud AutoML, SageMaker), Microsoft is planning to do so. I predict that manual tuning is going the way of dodo, and good riddance.

I hope that you see the pattern here. What was hard becomes easy, you can achieve more while understanding less. Great engineering feats of the past start sounding rather lame, and we shouldn’t expect our present feats to fare better in the future. This is a good thing and a sign of amazing progress. We owe this progress to companies like Google, who are investing heavily in the tools, and then giving them away for free. The reason why they do that is twofold. View More