10 predictions for data science and AI in 2020 Posted on : Dec 11 - 2019

As we come to the end of 2019, we reflect on a year whose start already saw 100 machine learning papers published a day and its end looks to see a record-breaking funding year for AI.

But the path getting real value from data science and AI can be a long and difficult journey.

To paraphrase Eric Beinhocker from the Institute for New Economic Thinking, there are physical technologies that evolve at the pace of science, and social technologies that evolve at the pace at which humans can change — much slower.

Applied to the domain of data science and AI, the most sophisticated deep learning algorithms or the most robust and scalable real-time streaming data pipelines (‘physical technology’) mean little if decisions are not effectively made, organizational processes actively hinder data science and AI, and AI applications are not adopted due to lack of trust (‘social technology’).

With that in mind, my predictions for 2020 attempt to balance both aspects, with an emphasis on real value for companies, and not just ‘cool things’ for data science teams.

1. Data science and AI roles continue the trend towards specialization.

There is a practical split between ‘engineering-heavy’ data science roles focused on large production systems and the infrastructure and platforms that underpin them (‘Data/ML/AI Engineers’), and ‘science-heavy’ data science role that focus on investigative work and decision support (‘Data Scientists/Business Analytics Professionals/Analytics Consultants’).

The contrasting skill sets, different mental models, and established department structures make this a compelling pattern. The former has a natural affinity with IT and gains prominence as more models move into production. It has also shown to be a viable career transition from software engineering (such as here, here and here). Conversely, the immediacy of decision support and the need to continuously navigate uncertainty require data scientists working in a consulting capacity to be embedded in the business rather than managed via projects.

We continue to quietly move away from the idea of the unicorn because just because someone can do something, does not mean he or she should. For all the value of the multi-talented performer, they are not a comparative advantage when it comes to building and scaling large data science teams.

2. Executive understanding of data science and AI becomes more important.

The realization is dawning that the bottleneck to data science value may not be the technical aspects of data science or AI (gasp!), but the maturity of the actual consumers of data science.

While some technology companies and large corporations have a head start, there is a growing awareness that in-house training programs are often the best way to develop internal maturity. This is due to their ability to customize the content, start from where an organization is at and align training with identifiable company business problems and internal data sets. View More