AutoML 2.0: Is The Data Scientist Obsolete? Posted on : Apr 07 - 2020

It's an AutoML World

The world of AutoML has been proliferating over the past few years - and with a recession looming, the notion of automating the development of AI and Machine Learning is bound to become even more appealing. New platforms are available with increased capabilities and more automation. The advent of AI-powered Feature Engineering - which allows users to discover and create features for data science processing automatically - is enabling a whole new approach to data science that, seemingly, threatens the role of the data scientist. Should data scientists be concerned about these developments? What is the role of the data scientist in an automated process? How do organizations evolve because of this new-found automation?

AutoML 2.0, More Automation for Data Science

First-generation AutoML platforms have focused on automating the machine learning part of the data science process. In a traditional data science workflow, however, the longest and most challenging part is the highly manual step known as feature engineering. Feature engineering involves connecting data sources and building a flat "feature table" with a rich, diverse set of "features" that is evaluated against multiple Machine Learning algorithms. The challenge of feature engineering is that it requires an elevated level of domain expertise to “ideate” new features and is very iterative as features are evaluated and rejected or chosen. New platforms, however, have recently emerged that provide additional capabilities and automation aimed at solving this challenge. Platforms with "Automated Feature Engineering" capabilities now allow for the automated creation of feature-tables from relational data sources as well as flat files. This ability to "auto-generate" features in the data science process is a game-changing capability. Suddenly, the "citizen" data scientists - Business Intelligence (BI) analysts, data engineers, and other technically savvy members of the organization with deep domain knowledge - can become valuable contributors to an organization's development of ML and AI models. Through Automated Feature Engineering, BI teams can suddenly develop sophisticated predictive analytics algorithms in days, significantly accelerating their productivity with minimal help from data scientists.

Automating Data Science: Democratization

One of the chief benefits of AutoML 2.0 platforms is true data science democratization. When data science automation can accelerate and automate the process of discovering and creating features, it allows for a more diverse and abundant group of users to contribute to the data science process. Automation of feature creation allows the "citizen" data scientist to create incredibly useful, highly optimized use-cases. Because citizen data scientists typically have a high degree of "domain expertise," they can focus on use cases that are of high value to the organization with minimal if any assistance from the data science team. The added benefit of enabling citizen data scientists is that it allows the business to expand their use of data science without having to worry about hiring armies of data scientists. The ability to empower new data science contributors is especially significant given the difficulty organizations in the US have had in hiring data scientists, as examined in a 2018 LinkedIn study. With economic uncertainty facing the global community, enabling a new class of AI/ML developers with minimal investments becomes a game-changing value proposition to maintain or increase competitive advantages. View More