Data Mining

How do we profit from the Internet of Things data blizzard?

The mania for data collection continues unabated. Even our local council demands supplementary information in otherwise innocuous surveys. For example, it wants to know if you consider yourself part of an “ethnic minority” when responding to a questionnaire on flooding. The Government’s new Draft Communications Data Bill increases the obligation of ‘communications and postal operators’ to retain 12 months’ worth of data on every communication made – not the actual content but the who, how, when and where of each communication. The multi-faceted Internet of Things (IoT) adds to the data blizzard. To varying degrees, all of this data has to be communicated, stored, analysed and, maybe, acted upon.


Although the signal-to-noise ratio seems to be increasingly out of kilter, you can’t avoid a strong sense of organisations capturing data just in case it’s needed. You can understand this with the security services and their need to figure out who’s been talking to whom. And, perhaps even more so with long-lasting machines of various kinds – jet engines, power turbines, lorries and locomotives, for example. GE Software’s CTO, Harel Kodesh, assured me recently that, “Equipment will be instrumented to death.” These industrial sensors will, no doubt, generate more data than all the other IoT data sources combined.


Somewhere in the middle is all the humdrum business and consumer data which emanates from our bodies, cars, toys, computers, phones, tills, and anything else that contains sensors and actuators and that has the ability to communicate. Wireless Logic is a company that provides secure data transmission from two million subscribers around Europe through mobile networks and satellites. It could be vehicle driver behaviour for insurance companies, fuel theft detection through side-mounted cameras, medical information, and more besides. The one thing it doesn’t see much of is the much-hyped ‘smart fridge’. Company co-founder, Phil Cole, says, “Compared with important things like vehicle telemetry or health monitoring, the communicating fridge would be low on anyone’s priorities.”


Some data can be used immediately and then thrown away, compressed or aggregated. We’re talking here about switching on lights in response to a darkening sky or adjusting a wind farm’s turbine blade angles to make the best use of the available wind and take account of turbulence caused by its nearby neighbours. Who cares what happens to that data? It’s done its job. However, associated data, like how long the lights have been on, energy generated, dates and times might contribute to a better understanding of the systems in use, their maintenance needs and their running costs.


It’s also important to operate within the constraints of the communications mechanisms. You wouldn’t want to try and move the hundreds of gigabytes output of an MRI machine across a mobile telephone network. Equally, you wouldn’t really want farmland monitors feeding their data expensively up through satellites. (There’s a lot of talk about using the white space in TV channels as low cost way of getting such remote data to the nearest ‘proper’ connection.) If you’ve got vehicles on the move, you have little choice at the moment but to use mobile networks or satellites, but at least you can hop out of their networks into a private cloud at the first opportunity. Several vendors provide international secure private clouds for onward transmission and, possibly, partial processing as close to the source as possible.


The trick with handling the data blizzard, as GE’s Kodesh, notes, “is to choose what to extract and use.” The UK Government has been smart in this respect. It proposes a rummaging mechanism called a Request Filter which will run across all the data held by ISPs and other communications providers. Crudely stated, its intent is to find all relevant communications connected to an individual’s or an organisation’s IP address and throw away all the collateral connections, before the relevant connections are reported back to the requesting authority. The details still have to be hammered out but one bit of good news is that local councils won’t be allowed to make requests. No doubt industry will have similar mechanisms for filtering out the information that can lead to appropriate actions. It would be easy enough, as new capabilities come on stream, to further mine and analyse the data being generated by these “instrumented to death” machines.


None of this IoT activity is done just for the sake of it. Underneath, there’s a deadly serious intent, whether it’s to catch criminals or prevent terrorist activity, save on maintenance costs, improve margins or simply provide better products than competitors. The remote sensors, actuators, intelligence, communications, storage and analytics all add up to opportunity for those prepared to seize it. It also requires investment. And that’s where ‘IT as business consultant’ comes into the picture. Your organisation has to be prepared to accept IT as a core part of the business management team and IT has to take a ‘business value’ view of the opportunities to be found in the data blizzard.


« Quotes of the week: The dark web, #DrummondPuddleWatch & GE's knitting


Was the Drummond Puddle deeper than it seemed or proof of our shallowness? »
David Tebbutt

David Tebbutt spent 14 years in IT (programmer to IT manager) before re-launching Personal Computer World magazine in the UK in 1979. This led to parallel careers in writing, editing, software and media skills training. He sees himself as a closet techie who loves the media, IT and training in approximately equal measure.

  • Mail


Do you think your smartphone is making you a workaholic?