Power Laws and the Never-Ending Big Data Boom Posted on : Jun 21 - 2018

We’re about a decade into the big data boom, which may seem like a lifetime for those who have worked through it. But in many ways, we’re just getting started with what big data and AI technology is capable of giving us, Forrester principal analyst Brian Hopkins said at Hortonworks DataWorks Summit yesterday.

Forrester’s respected analyst gave a sneak peek into his vision for the future of business and technology during his keynote address before an estimated 2,100 people yesterday at the DataWorks Summit, which continues today and tomorrow at the San Jose McEnery Convention Center.

Hopkins had some practical advice for data practitioners. For instance, he encouraged customers to reconsider their big data architectures. Instead of building batch-oriented data lakes and relying on DataOps techniques to run it, he told the audience they were probably better served by investing in big data fabrics, real-time edge processing, and “NoOps” serverless cloud implementations.

Metcalfe’s Law was proven with Facebook data

But it was Hopkins’ views on where technology is headed and how businesses can prepare for it that really resonated.

Hopkins observed that several computing “laws” are currently working themselves out, even if we don’t notice them on a day-to-day basis. The first of these, Moore’s Law, is probably the best known. And indeed, we’re told on a regular basis that Moore’s Law is petering out as processors reach physical limits.

But other laws are picking up where Intel founder’s Gordon Moore’s leaves off. That includes Metcalfe’s Law, which states that the effect of a telecommunications network is proportional to the square of connected users. Metcalfe’s Law was first postulated in 1980 by Robert Metcalfe, who founded 3COM (now part of HPE) and co-invented Ethernet. But it wasn’t until 2013, when Metcalfe had access to Facebook social networking data, that he was able to prove it, Hopkins says.

Another law working itself out is Bezos Law, which says the cost of cloud computing power decreases by 50% every three years. Bezos Law, of course, was named after Amazon founder Jeff Bezos, who is now the richest man in the world. “We’re just getting started with that,” Hopkins says about Bezos Law. “We really don’t know. We haven’t measured that enough to know if it’s true.” View More