When speaking about "connecting the dots" one is often assumed to be addressing the challenge of analyzing intelligence data ... sitting in a dark room, deep underground, surrounded by video displays. But the private sector faces the same challenge - trying to sift through a vast quantity of information to understand a larger picture - a context for making important decisions.

The "dots" in intelligence are often a mix of facts, conjecture and flatly incorrect information. Some of the dots are also missing, which makes analysis particularly challenging. But this is the nature of intelligence work - the goal is often to predict a complex outcome as best you can. As Yogi Berra once said, "predictions are hard - especially about the future".

In the enterprise there is generally more certainty about information - the internal variety, at least. Jack Bauer may struggle, on 24, to figure out who, exactly, did X to Y at Z. A typical analyst in the private sector, however, will have far less trouble finding out how many Xs customer Y bought at store Z.

It was not always this way. Decades ago getting information like that could be an expensive, time-consuming challenge. As the need for that information has grown, entire classes of technologies - the "BI stack" - have emerged and evolved and made it easier to get. But the challenge has evolved as well - now it is not enough to understand "what" happened by studying the transactions that define the business. Deeper insight is required. This includes understanding "how" and "why" things happen, and what important forces that may ultimately impact "what" happens need to be investigated more deeply. A somewhat newer class of technologies - enterprise search - has emerged as a tool to help answer some of these questions. In the past few years enterprise search has become nearly as ubiquitous as the database.

Yet the challenge of connecting the dots remains. Survey after survey reveals dissatisfaction with both of these technologies, and the stacks that have grown up around them. Some of these issues are due to lingering enterprise search issues - security integration, for example, is a frequently cited challenge - so many companies provide search access to only a fraction of their internal content.

But the larger contributor to the problem is that the "dots" have moved! We have evolved from being unable to get enough data or content, to being able to get at a lot of it - just not at the same time. This leads to all sorts of paradoxes; for example, that you can monitor customer opinion in real time, but not correlate it against sales until the end of the month. That you can know instantly when a customer is returning a product, but it takes months to find out why. That you can know which customers produce the most value on a daily basis, but not which ones are most likely to defect.

Unified Information Access (UIA) is the new technology that has emerged to solve this critical problem and help users connect the dots despite information silos. Designed for massive, Google-like scalability and rapid update rates, supporting both search and more traditional BI/reporting capabilities, and allowing content of any type - structured transactions or unstructured documents - to be intermingled and queried in real time, UIA platforms significantly reduce the cost and risk of implementations. For business users, UIA means flexibility towards new profit and revenue opportunities. Perhaps most importantly, innovative companies will stop giving up on good ideas that would help them ... connect the dots.

Sid Probstein is CTO of Attivio, a provider of unified information access technology.