Speaker "Kapil Surlaker" Details Back
-
Name
Kapil Surlaker
-
Company
Linkedin
-
Designation
Director
Topic
Building a real-time, self-service data analytics ecosystem at LinkedIn.
Abstract
LinkedIn has a rich ecosystem of data-driven products like People you may know, Who viewed my Profile, recommendation products as well as business facing insights. Building a data product end-to-end requires a lot of technologies to come together and work seamlessly and requires innovations far beyond traditional data warehousing. A major focus at LinkedIn has been to improve the agility of the engineers and data scientists in creating these data products end to end. To that end, we have developed a number of systems in the analytics data ecosystem. These include a platform to manage ingestion of variety of data sources at scale, a platform to do joins and complex calculations at extremely large scale, a platform for extremely fast OLAP serving including real-time drilldowns and a platform to enable data lineage analysis and data discovery. These are all the pieces required to have an effective self-service offline data ecosystem. In this talk, we will go into the details of some of these systems and show how they provide a self-service real-time analytics ecosystem.