Master Data Management

Paul Gustafson (US) - Big Data: Your Own Data is Not Enough Part 2

In our previous post about the impact of big data on business, we talked about how the U.S. government's initiative,, along with other linked data examples like the Linked Clinical Trials project, are building communities of innovation where people can discuss data and leverage related data sources.

Some of our most difficult problems with linking data will be solved by gleaning insights from an even wider set of federated data sources. The science of climate change requires both NOAA and climate data. Understanding the effect of climate on people requires combining climate and health data. Better energy management depends on data about location, terrain, weather and time combined with sensor data and enterprise data (e.g., ERP systems, security systems).

Financial services and consumer products companies will reap the rewards of combining enterprise data with social network data. The idea is to match the "why" inherent in social data with the "results" inherent in well-understood traditional data like ERP and CRM systems, sales spreadsheets and financial reports. It is also important to check the ongoing pulse of customer sentiment so that negative feedback can be dealt with head on, illustrated by the Procter & Gamble case of parents complaining about a new form of Pampers diapers. Even political movements benefit from mining social networks like Twitter for just-in-time data, as the uprisings in North Africa and the Mid East showed.

No One Person Owns the Data

These new data connections are forging new connections among organizations who need to collaborate on the data. This means no single organization owns all the data. Thus the idea of Data as a Service starts to make sense because the data needs to be available to multiple parties. (from is a pioneer, initially open to developers in 2010 and offering a full developer preview in 2011. Data as a Service feeds a new era of experimentation and participation as data is made readily available as a service, not hoarded in a proprietary, walled-in database. It is also the next logical step in the evolution of cloud computing.

Hand in hand with Data as a Service is Analytics as a Service, a natural in the progression of business analytics that supports both scalability and mobility. Healthcare is fertile ground as electronic health records come on line and are federated with other health data; Massive Health is one company combining data analytics (as a service) and health data with mobile devices to help people understand immediately how their chronic condition has changed. Opera Solutions and Microsoft's Project Daytona are cross-industry examples of Analytics as a Service.

Technology Answers the Call

These "as a Service" cloud offerings are just one manifestation of how technology is answering the call for big data processing and sharing. At a system level, we have the big Internet companies to thank, as they have led the march towards innovative computing and application methods to perform the heavy lifting. Amazon and Google popularized "shared nothing" architectures, Google gave us MapReduce, and Yahoo has been the largest contributor to Hadoop. These new approaches, coupled with non-relational and parallel relational databases, and overlaid on a cloud infrastructure, represent the changing data foundation. The days of the "one size fits all" relational database built on a shared disk/memory architecture are over.

This changing foundation is designed to handle large processing loads by enabling massively scalable systems and parallel processing. In addition, "in-database" techniques are reducing processing time significantly because instead of moving the data to an analytics engine, the analytics are moved to where the data resides. The data stays put, relieving the huge burden of moving it around as in the past, and enabling larger data sets to be processed.

This is the second post of a three part series examing the impact of big data on business. In our final installment we will discuss how big data is creating the predictive enterprise.

Big Data: Your Own Data is Not Enough Part 1

Big Data: Your Own Data is Not Enough Part 3

By Paul Gustafson, Director of CSC's Leading Edge Forum, Technology Programs

This article is based on a new LEF report, Data rEvolution -



« Dr K V Krishnan (India) - Empowering Healthcare with IT part I


Daniel Viveiros (Global) - The Changing View of Development in the Cloud - Moving Towards a Bright Future: Part 1 »

Recommended for You

Trump hits partial pause on Huawei ban, but 5G concerns persist

Phil Muncaster reports on China and beyond

FinancialForce profits from PSA investment

Martin Veitch's inside track on today’s tech trends

Future-proofing the Middle East

Keri Allan looks at the latest trends and technologies


Do you think your smartphone is making you a workaholic?