Digital transformation and IoT related use cases are forcing analytics to move from a batch descriptive mode to actionable predictive and prescriptive mode. Additionally technology advances with accelerated compute, availability of lots of data and open sourcing of deep learning frameworks, is leading to convergence of  Data Analytics and  Cognitive analytics ( AI / deep learning ) use cases. Data analytics is increasingly being used for ETL and data transformation as part of AI and deep learning pipelines. Enterprise customers are asking for multi-cloud or hybrid data lakes spanning on-premise and multiple public clouds, to provide best of breed infrastructure and prevent vendor lock-in, while data as a strategic asset is been stored on-premise.

 

Scale and performance needs are forcing users to look beyond HDFS

MapReduce and HDFS has served the batch descriptive use cases well, however with market transitioning, Spark and Presto in-memory frameworks are becoming the norm. On the storage side there is a need to provide more performance, scale and durability, multi-cloud workflows and better economics for managing the data deluge.