Master Data Management

Peter Evans (Global) - Big Data: The Hype and Impact on BI

What is the ‘Big Data’ hype?  Well, in the European market, the answer must not be very much – given the very few replies to a recent survey on Quroa.  While the market is definitely ready for the introduction of Big Data, there seems to be not as much push within the European zone to actually build and start companies that are focusing directly on the technology as there seems to be in the U.S. and Asian markets.  The story is different, however, if you talk to either IT or the Business groups of a number of well-known major European corporations that are pushing for ‘Big Data’ integration tools and social networking technologies to be included in their overall spectrum of market analysis.   This move is driven by a perceived need from the marketing and sales departments to try and get ahead of the curve with adoption in this fast-moving area.

Part of the adoption problem for start-ups may be that an agreed definition of ‘Big Data’ does not actually exist yet, and with the emerging nature of the topic, it may be quite a while before it does. One of the reasons for this is that it’s very difficult to define what is ‘big’ in the fast moving world of social networking, where unstructured data is already out scaling the Exabyte.  NoSQL implementations like Cassandra and Dynamo can out scale the terabyte - and on to the petabyte size - by utilizing horizontal scaling and multiple nodes and, in particular, the cost differences between SQL and NoSQL implementations are significant.  However, each type of NoSQL system uses its own proprietary code for its connections and this is a major stumbling block for both start-ups and business wishing to take advantage of scalability and analysis opportunities. In the European sector, the early adopters of the technology seem to be in the ad optimization, gis and language processing sectors.

The problem with the hype is that the buzz of ‘Big Data’ is driving CEOs to question their CIOs without fully understanding what ‘Big Data’ can do for their company. This, in turn, is driving IT to look at all the technologies and employ consultants and experts to experiment with Big Data as a Business Intelligence system, which generates more interest from competitors who do not want to be left behind.

Reasons for this may include the following:

•    When dealing with large data volumes, several ‘Big Data’ technologies can be  more cost effective than existing proprietary solutions
•    The volume of data that organizations gather has grown massively in recent years
•    Some technologies are better suited for large-scale data mining and Business Intelligence than the traditional relational databases that are widely used today.

The impact of the big data hype on the Business Intelligence landscape can be identified by the number of BI and Data Discovery companies, globally, that are working hard to add connectors for Hadoop, NoSQL (Cassandra, Mongo DB etc.) and the other various ‘Big Data’ technologies to their applications.  However, while these companies have the connectors in place, very few of even the leading edge commercial companies are actually utilizing ‘Big Data’ at the moment.  It is not actually the volume of data or even the velocity of the transactions that makes the data ‘Big.’  I like to think that it is only when you can actually use all of the data to add value to your company does it become big, and only then does it become important to overall decision-making within the company, which is what Business Intelligence is all about.

It is interesting to note, in these overall discussions about ‘Big Data,’ that the European Union (EU) has started to get very interested  both in law-making to promote privacy of the data, and also in utilizing the data to enable decision-making and promotion of  government objectives through social networking. 

By Peter Evans, a recognized expert in business intelligence and data warehousing affiliated with Quest Software. Peter's bio, blog, Twitter.  


« Klaus Holse Andersen (Europe) - How Cloud is Engineering a New Breed of CIOs


Franklin N. Nnebe (Africa) - Africa's Changing Internet Usage Patterns »


Do you think your smartphone is making you a workaholic?