Nobody needs to be told that ‘data is the new oil’ or given any other now increasingly overused analogies to express the importance of information in modern business. We all understand that a typical airline flight generates somewhere approaching a terabyte of data and that all contemporary business systems run on massive data-flows with both employees and machines constantly adding more depth to the data pool.
But the technology industry wants to try and provide us with an answer to what is essentially a data wrangling predicament of its own making.
Surely we can see that one clothing retailer has to manage a relatively similar set of data to that of another similar clothing business, right? Extending the example, surely one car mechanic business works with similar data flows to washing machine repair company. The answer (unfortunately) is both yes and no.
Yes, some companies can use obfuscated anonymised data patterns that other firms have ‘traded’ through software vendor licence agreements and via formalised data marketplace exchange specialists. Big players in the Enterprise Resource Management (ERP) space like to give these services fancy names like Rapid Development Shortcuts or Industry Solution Models and so on.
Meanwhile, in the real world
But it’s also a no, i.e. many enterprise systems will be under too much strain to diversify and offer new services across an increasingly distributed deployment surface to benefit from such shortcutting. In the real world of data management, we need a combination of predefined shortcuts, wider-view reference architecture blueprints plus a good proportion of on-the-job human operational know-how.
Known for its specific competencies in data integration, Tibco Software used its (virtual) annual user convention last week to launch a product that aims to provide a cohesive data picture of an enterprise’s data estate. The TIBCO Any Data Hub is specifically described as an all-encompassing data management blueprint that embraces distributed data environments.
"In a perfect world with perfect systems and perfect, never-changing architectures, there would be no need for tools and technologies like master data management and data virtualisation,” said Mark Palmer, senior vice president, engineering, Tibco. “We don’t live in that world. Our world has seen a massive expansion in data and an evolution of the systems and architectures that generate and manage this data. Unifying it can be a thankless task.”
Staying in the real world of business with real world (often real time) data streams, organisations straddling the firehose of information have to deal with a fairly unruly animal. Data can be inconsistent, incomplete, not trusted, duplicated or wildly different in terms of its core form, structure and purpose. This is not a stable commodity that can be used as part of a systematic approach to well-managed business.
Let’s get data logical
Tibco’s Palmer thinks his company can tame the data beast (his exact term) and help companies build a simplified data strategy for distributed and siloed data environments. This is supposed to facilitate an organisation’s desire to build modern data architectures such as a data fabric, logical data warehouse, or data-as-a-service.
Although it might sound like branding hyperbole, Tibco isn’t just using marketingspeak here i.e. really tuned in data-driven businesses don’t just use a database or indeed a data warehouse (a collection of integrated data repositories with analytics and reporting capabilities), they run a logical data warehouse.
Garter analyst Merv Adrian quotes his colleague Mark Beyer as the person that coined the term. “The logical data warehouse (LDW) is a new data management architecture for analytics that combines the strengths of traditional repository warehouses with alternative data management and access strategy,” he notes.
If anything, the concept of LDW is all about being less logical, or at least more random i.e. it allows a business to interrogate different layers, tiers, zones and sources of data through a virtualisation layer as if they were talking to a single physical data warehouse. But enough context, Tibco says its Any Data Hub is purpose-built to blueprint the way to exactly this kind of job.
“[The Any Data Hub] reduces the time and resources needed to collect and manage data, and strips away duplication of efforts across data, teams and technologies. As a result of its flexible nature, it provides the accurate and relevant data needed to understand, run, and govern the business,” noted Tibco, in a press statement.
The company has also provided an expanded set of data adapters. Companies can now connect to over 300+ data sources with support for streaming sources, including Kafka, OSIsoft and OPC Unified Architecture.
Disparate times, no desperate measures
It is perhaps an unfortunate comment on the state of the world in 2020 with socially distant workers in more locations adding to the whole notion of what we mean by ‘disparate data sources’.
But disparate times should not call for desperate measures. If we are going to continue to build a world of data-driven businesses that we will need to think about data management in new and ever-more complex terms. Just remember, get your templates, get your blueprints, get your data reference architectures, get your AI-enabled machine managed data shortcuts, but still get your hands in the grease and get a ‘feel’ for data, however virtual we all become.