Metadata Management

Metadata: More Important Than We Ever Thought

Some psychologists will tell you that 90% of human communication is non-verbal. Others will tell you that's nonsense, but then the nature of psychology demands that for any given question the answer is usually "It depends".

Historically, the social sciences haven't been considered to be 'real' sciences, even with advances such as fMRI in recent years (and even then the interpretations of brain scans are fiercely debated). But things are changing, and part of what's driving that change is the statistical analysis of vast swathes of data. Techniques that weren't possible a decade ago are now made easy thanks to significant advances in the collection and analysis of quantitative data. And what these new techniques are telling us feeds right back into debates about online security, data mining, privacy and even what it means to be human.

Consider the routine way in which information about online behaviour is collected and analysed commercially. What's usually processed is not the content but the metadata. That means different things to different organisations, but at a simple level it might mean an ISP collecting the date, time, subject and recipient of an email but not the body content. Similarly, phone companies might log the date, time and recipient of a call but not the actual conversation.

Why do they do this? The main reason is financial. Analysis of the bulk metadata allows them to infer useful information in order to better target their advertising and design new services that will appeal to their customers.

And this is big business, with companies such as MineTech selling products and services aimed at analysing such data and feeding back the results into business strategy. This is generally A Good Thing, because it allows companies to improve the services they offer based on 'feedback' from customers that the customers themselves aren't even aware of. It's a way of finding patterns in behaviour that the people concerned know nothing about.

But now researchers are starting to dig into these processes a little more, and their findings make for fascinating reading. New Scientist magazine recently (5 April 2014, p.30) carried an article entitled The Death of Individuality by Alex 'Sandy' Pentland at MIT. The article described social physics, a relatively new discipline that uses social communications data to analyse how people behave.

One striking discovery was that analysis of communication patterns on its own is an incredibly powerful predictor of behaviour. The actual content of conversations is far less important. It's the connections that count: who talks to whom and for how long. In other words, it's the metadata that matters.

Given the level of predictability found, it seems we are not “as unique” as we might like to think we are. Perhaps this shouldn't be surprising. The 'Western' individualist society is still something of an aberration, and a recent one at that. In many other societies the focus is still on the group rather than the individual. Interestingly, other research links this historical difference to crop farming: rice-growing fosters a more collectivist society, wheat-growing a more individualist one (New Scientist, 17 May 2014, p. 17).

But even in the West, individualism can be a flimsy construct. From punks to goths to hipsters, if you want to stand out as an individual it seems you have to look and behave the same way as those in your chosen peer group. The veneer of difference hides overwhelming similarities. And now the data proves it.

Which brings us back to the first paragraph of this article, because it appears that what we actually say to each other really isn't important. It's who we are saying it to and when that matters. Communications metadata is not a poor proxy for content. It's actually far more important than content.

Pentland makes the point that such knowledge offers real power, which is why privacy is so important. Understanding the interconnectivity of human relationships inevitably reveals the focal points of the network being studied. Those focal points can then be manipulated, through targeted advertising or other means, in order to influence the behaviour of the group.

Not many people realise this, which is why there's no huge fuss when politicians or corporations tell their electorate or customers, "It's OK, we're not reading your emails or texts, or listening to your phone calls. We're just collecting metadata: addresses, phone numbers, that sort of thing."

A more honest, accurate statement would be, "It's OK, all we're doing is metadata analysis to build up a comprehensive model network of relationships so we can accurately pinpoint the social connections between individuals and understand exactly who is talking to whom, when and why. That tells us far more about you than the content of your phone calls, texts or emails. And, should we so desire, makes it easy to manipulate your behaviour."

But as with most advances in science, this type of analysis is ultimately neither a good thing nor a bad. Its effects depend entirely on the way it is used, and by whom. For authoritarian regimes it offers an additional tool for social control and repression. For free societies it allows people to benefit through companies subtly tailoring products and services to their customers' exact requirements: requirements they may not even know they have. Like it or not, then, social physics and manipulation via metadata are probably here to stay.



Freelance technology journalist Alex Cruickshank grew up in England and emigrated to New Zealand several years ago, where he runs his own writing business called Ministry of Prose.


« Which World Cup Team Best Describes Your Style As A Finance Leader?


Trust is a Key Component of the New Flexible Working Law »
Alex Cruickshank

Alex Cruickshank has been writing about technology and business since 1994. He has lived in various far-flung places around the world and is now based in Berlin.  

  • Mail


Do you think your smartphone is making you a workaholic?