Data Modeling In The Age Of NoSQL And Big Data Posted on : Dec 29 - 2015

Hadoop Hbase. MongoDB. Cassandra. Couchbase. Neo4J. Riak.

Those are just a few of the sprawling community of NoSQL databases, a category that originally sprang up in response to the internal needs of companies such as Google, Amazon, Facebook, LinkedIn, Yahoo and more – needs for better scalability, lower latency, greater flexibility, and a better price/performance ratio in an age of Big Data and Cloud computing.

They come in many forms, from key-value stores to wide-column stores to data grids and document, graph, and object databases. And as a group – however still informally defined – NoSQL (considered by most to mean “not only SQL”) is growing fast. The worldwide NoSQL market is expected to reach $3.4 billion by 2018, growing at a CAGR of 21 percent between last year and 2018, according to Market Research Media.

“There are all kinds of applications being built now with NoSQL systems,” says Rick van der Lans, an independent analyst, consultant, speaker and author who specializes in data warehousing, Business Intelligence (BI), database technology, and data virtualization. His book, Introduction to SQL, was the first book about SQL databases available in English.

Web sites and BI provide two examples where NoSQL databases are being adopted, “both worlds are Big Data worlds,” he says. With web sites like Amazon, for example, “they are analyzing what you as a customer do on the spot. You are looking for a book and they are hosting recommendations, so they are analyzing live the enormous web logs of highly complex and multi-structured data created in the background as users move around their site.”

Indeed, one of advantages that NoSQL brings to the table for Big Data is that it allows storage of schema-less data, which makes it well-suited to Big Data environments where the data doesn’t have a particular structure – it may be unstructured, like text, and it may be open to your coming up with many different structures for the same data:

“That’s why some call the data multi-structured, meaning that you can look at the same data from different angles,” says van der Lans, perhaps from the point of view of the customer today and from the supplier angle tomorrow. “It’s as if you are using different filters when looking at the same object.”

Instead of coming up with the structure for modeling the data in advance, as is the case with relational databases, “NoSQL systems let us store the data as it comes in,” he says, in nested and hierarchical structures, in records in tables that can have different structures, and to which values can be added for which no column has been defined yet. “When we access the data, when we query it, then we determine the structure we want to use. That means it’s more flexible.” View More