In this talk, I build a real-world machine learning pipeline using TensorFlow Extended (TFX), KubeFlow, Airflow, and MLflow.

 

Described in a 2017 paper from Google, TFX is used internally by thousands of Google data scientists and engineers across every major product line within Google.

 

KubeFlow is a modern, end-to-end pipeline orchestration framework that embraces the latest AI best practices including hyper-parameter tuning, distributed model training, and

model tracking.

 

Airflow is the most-widely used pipeline orchestration framework in machine learning and data engineering.

 

MLflow is a lightweight experiment-tracking system recently open-sourced by Databricks, the creators of Apache Spark. MLflow supports Python, Java/Scala, and R - and offers native support for TensorFlow, Keras, and Scikit-Learn.

 

Attendees will come away with a better understanding of the tools needed for a modern, iterative ML/AI pipeline.

Chris Fregly is a Developer Advocate for AI and Machine Learning at Amazon Web Services (AWS) based in San Francisco, California. He is co-author of the O'Reilly Book, "Data Science on AWS."

 

Chris is also the Founder of many global meetups focused on Apache Spark, TensorFlow, and KubeFlow. He regularly speaks at AI and Machine Learning conferences across the world including O’Reilly AI & Strata, Open Data Science Conference (ODSC), and GPU Technology Conference (GTC).

 

Previously, Chris was Founder at PipelineAI where he worked with many AI-first startups and enterprises to continuously deploy ML/AI Pipelines using Apache Spark ML, Kubernetes, TensorFlow, Kubeflow, Amazon EKS, and Amazon SageMaker.