How Microsoft Azure Databricks can help companies speed big data and AI adoption Posted on : Mar 23 - 2018

The analytics platform based on Apache Spark and optimized for Microsoft Azure will help businesses better integrate and scale machine learning projects.

• Azure Databricks, the Apache Spark-based analytics platform optimized for Azure, is now generally available from Microsoft.
• Azure Databricks aims to help businesses speed up and simplify the process of building big data and AI solutions.

On Thursday, Microsoft announced the general availability of Azure Databricks, the Apache Spark-based analytics platform optimized for Azure, aiming to help businesses speed up and simplify the process of building big data and artificial intelligence (AI) solutions.

Apache Spark is an open source standard for advanced analytics, machine learning, AI, and big data, that has seen rapid enterprise adoption in recent years, according to a Thursday Microsoft blog post. It was created by the founders of Databricks. With Azure Databricks, Microsoft aims to help businesses work with data in a faster, easier, and more collaborative way.

The new offering could help continue Microsoft Azure's rapid growth in the enterprise, especially as companies look to integrate AI into products and processes. While Amazon Web Services (AWS) remains in the cloud lead, adopted by 64% of firms in 2018, Azure is quickly catching up, and is now used by 45% of firms, according to RightScale. The lead of AWS among large enterprises is also shrinking, RightScale found: While 68% of enterprises use AWS, that number represents only 15% year-over-year (YoY) growth in adoption. Comparatively, Azure experienced 35% YoY growth, to reach 58% of enterprises.

Azure Databricks was developed in part to improve user productivity in developing big data applications and analytics platforms, the post noted. The interactive notebooks used with Azure Databricks allow data science teams to work in popular languages such as R, Python, Scala, and SQL, and bring in all of their data sets to create machine learning models, rather than only a sample. The solution's native integration with Azure makes it easier to build end-to-end products, the post noted.

Using Azure Databricks, the company renewables.AI was able to increase the productivity of its data science team by more than 50%, according to the post. "Instead of one data scientist writing AI code and being the only person who understands it, everybody uses Azure Databricks to share code and develop together," Andy Cross, director of renewables.AI, said in the post.

With Azure Databricks, Microsoft also wants to help customers scale globally. The solution provides a fully managed, cloud-native service that automatically scales based on an organization's needs, according to the post, and simplifies the process of building data pipelines and deploying machine learning models at scale. View More