When AI Meet Experimentation
 

At Uber we have built experimentation platform to enable conducting experiments at scale. Most experiments are performed under A/ B testing setting. I will discuss standard technique used and potentials pitfalls to avoid during hypothesis testing. Because of the nature of a global market place, sometimes experiments are conducted in a non A/B fashion using Synthetic Control technique. Applications of artificial intelligence (AI) is used in practically every industry domain. A relatively new area of application of AI is hypothesis testing through experimentation. I will discuss how Uber leverages AI to build technique for conducting non A/B experiments.

While doing experiments, guarding against potential regressions on business metrics is just as important as conducting the experiments. I will discuss deep learning based techniques that are used to detect regressions in business metrics during the experiments and the feature roll out.