Metadata gold: Why metadata is the critical underpinning of business

Data is a powerful tool, especially in business. Despite this, and the benefits it can bring, not many business leaders are familiar with data’s counterpart, metadata, and its usefulness. By understanding metadata and adopting a metadata management policy, businesses can harness the extra layer of insights to drive and facilitate even better results.


This is a contributed article by Craig Stewart, CTO at SnapLogic.

Data is a powerful tool, especially in business. It’s an important asset for every organisation to utilise and can help leaders make fully informed business decisions. A piece of data can completely change the direction an initiative may take and subsequently lead to a better outcome, so no wonder companies are becoming more obsessed with data and the insights that it can generate.

However, many businesses are yet to familiarise themselves with data’s counterpart – metadata – and the value that it can bring by adding context. This is largely because metadata goes unnoticed. Unless you are looking for it or know what you are looking at, chances are you would ignore it as a by-product of your IT estate.

Why metadata?

As a broad definition, metadata provides information about data that is in focus – it’s essentially data about data. So, at a very basic level, take a document that is shared as an example. The data is the contents of the document, but the metadata encompasses the information pertinent to the documents’ properties – who created it, when they created it and when it was last saved down on the system, or even why they created it for example. It provides the user with a degree of context which can make the data that is being used easier to work with and most importantly, understand.

This is why metadata is so beneficial to businesses. If used correctly it can accelerate teams to find necessary insights into how the business is being run and it’s especially useful in terms of data governance and responsible data use. Metadata provides more transparency and assurances around the quality of the data that they are working with.

This, and the additional uses of metadata, means there are plenty of benefits to come from introducing a policy of metadata management within the enterprise. If you can provide the context of data, not only what it consists of but the expectations of the data too, you accelerate the user’s ability to find that value, avoid costly experimentation, and improve the productivity of the organisation. Another of the main benefits is that metadata helps to keep data organised and structured, reducing the chance of data becoming muddled and mixed up, causing confusion. Imagine an Excel spreadsheet without the columns to divide and section it up – it would be a file with numbers and words with no indication of the meaning behind them. It would certainly be enough to cause an afternoon of headaches and likely lead to the data becoming stale and atrophied.

Metadata also makes data auditable. Having a good understanding of what it means makes it far easier to define and classify the original data, as well as cleanse it. This is vital – especially when it comes to compliance and regulation such as GDPR. It allows teams to ensure that the data is clean and up to a defined standard so that they can get the most out of it and trust that it’s reliable. This helps to prevent the business from making decisions based on wrong, unreliable and inaccurate information.

Managing metadata

Understanding the value of managing metadata is great. But understanding how to manage it is even more important, as businesses need to be able to make metadata available across the organisation.

For many IT leaders, the first port of call may be point solutions, using custom code or APIs to connect one application to another through a central data warehouse. It’s an approach regularly taken by businesses, and is a sound one in terms of bringing all the metadata together in one place. But what happens when those APIs need to change, or the code isn’t working quite right? IT leaders today want to be spending more time focusing on the tasks that deliver business value, not on IT administration.

The alternative is to take a platform approach to integration. Taking a platform-based approach means being able to reap all the benefits of the metadata being inherent in the platform, without the headaches of continuously manually updating and tweaking it. The platform approach provides a single harmonious layer for data to move across throughout the organisation, cutting down on the chaos and noise of too much dubious data.

We also increasingly see low-code technologies supporting the management and analysis of metadata, enabling individuals outside the IT team to get highly valuable insight from metadata, without the need for extensive technical skill. The metadata in the platform should be providing intelligence on origin, transformations, and targets – all easing the burden on IT teams.

Moving towards better metadata

Despite the fact that it often goes unnoticed, metadata is the critical underpinning of business. It brings clarity and can facilitate better results. With the growth of data continually rising, there is more information for businesses to make use of, which means that it’s now more important than ever to understand that data through effective use of metadata within the enterprise. It can help businesses clearly understand the data that they hold, how and why it’s relevant, enabling the delivery of even better, more reliable insights than data alone. Therefore, used correctly, metadata provides the extra layer of context that helps to drive successful results. If they are not already doing so, businesses should seriously think about introducing a common layer of metadata to their organisation to begin reaping the benefits.

Craig Stewart is CTO at SnapLogic, where he is responsible for leading the company’s technology strategy and product roadmap as well as ensuring its alignment with customer requirements and future market developments. Stewart is an experienced manager of technical teams with over two decades of experience guiding the evolution of data integration and data management technologies. Over his career, he has been a significant contributor to the development of several fast-growing technology companies, including Oracle, Cognos, Powersoft, Sybase, iMediation, and Sunopsis.