Attunity Relishes Role as Connective Tissue for Big Data Posted on : Oct 19 - 2017

In the big data world, there are a few patterns that repeat themselves over and over again. One of those is the need to move data from one place to another. For data integration tool maker Attunity, that pattern has provided ample opportunities for its changed data capture (CDC) technology.

CDC emerged just after the turn of the century, at least five years before Yahoo started filling the first Hadoop cluster. Back in those days, enterprises built data warehouses and stocked them with the latest data from core business applications running atop IBM DB2, Microsoft SQL Server, Sybase (now owned by SAP), and Oracle‘s eponymous database.

At first, bulk extracts were the norm. But as CDC technology filtered into the marketplace, enterprises realized that automatically detecting changes recorded by the database’s log, and then replicating the changed parts – not the whole table –constituted a better way.

Over the years, the data ecosystem has evolved in dramatic fashion. While relational data warehouses are still widely used, enterprises have rushed to embrace Apache Hadoop-style storage and processing, whereby raw data is loaded into a distributed file system and processed at run-time. This “schema on read” approach has flipped the script, so to speak, on the highly structured nature of the old data warehousing world. But it didn’t eliminate the need to move data in carefully orchestrated time slices.

Apache Sqoop may be widely used to load bulk data into Hadoop, but CDC technology has retained a foothold when it comes to loading data from relational databases, which remain the go-to platforms for operational systems in the Fortune 500. Getting data out of those databases and into analytic systems is a booming business at the moment.

 “We’re best known for real-time movement. CDC is the heart of what we do,” says Attunity’s vice president of product management and marketing Dan Potter. “We make our money by unlocking the data from those enterprise systems.” View More