Storage Management

Storage set to hit a wall as space runs out on e-hoarders

Storage management is in some ways the dirty secret of IT management. Organisations buy more and more storage and keep data on a ‘just in case’ basis. Terrified by a never-ending stream of stories about firms not being able to find relevant data on legal or regulatory demand, the response by IT seems to have been to “store more”.

A recent report from the storage management software giant Veritas, based on IT decision maker responses across 14 countries in EMEA, points to the extent of this unsatisfactory state of affairs. Veritas calls it the “Databerg”: a solid edifice of data that is kept but only 14% of which is regarded as having intrinsic value. By Veritas’s numbers, a third of the data retained is “ROT” (redundant, outdated and/or trivial) and over half is “dark data”: volumes that owners have no understanding of because they haven’t been tagged or classified, but they keep anyway.

Keeping useless data and not knowing its provenance or content is obviously an expensive, insecure and unwise approach, but why is it happening? Is it in part because the price of storage continues to plummet dramatically? After all, prices for both disk drives and solid state drives have fallen at a reasonably consistent curve for the last several years.

Veritas’s EMEA solutions manager David Moseley thinks not and points to other costs.

“The cost of storage is not high – it’s the rest that is expensive,” he says, referring to expensive backup and other administrative chores. “IT is not just throwing platters at the problem; they’re just keeping the lights on, but they simply don’t have a policy.”

Moseley refers to the practice of storing data without specific purposes as “e-hoarding” and says one organisation he knows of has 940 terabytes in storage with 54% of that data not having been touched in three years. Also, the issue actually goes beyond even managed storage and into the depths of tape archives. Another company he is aware of has nine petabytes of data on tape but doesn’t even know if those drives still work, he says.

Moseley also points to other inefficiencies including lack of management knowledge about how much data their companies are storing and rampant use of unsanctioned file-sharing tools.

David Merrill, chief economist at another storage company, Hitachi Data Systems, helps companies understand how to get better value from their storage investments. He believes that with the world’s data storage total forecasted to multiply by 10 by 2020 as trends like the Internet of Things kick in, current lackadaisical attitudes can’t go on.

“It’s not practical or sustainable,” he says. “There’s a false need [to store everything], a nervous uncertainty. We can’t lower the prices enough: even if we give away the disks for free we still can’t afford to maintain everything we want.”

Merrill also sees other bleak threats including a generation of graduates that doesn’t want to work in traditional IT jobs.

“In the US we have a shortage of people who want to work in IT. The universities aren’t pushing out graduates who want to do storage admin jobs or be DBAs. They don’t want to go to a Halogen-cooled datacentre, they want to do gaming.”

He also laments the fact that at a time when pundits view data as “the new oil” that yields insights if refined and analytics are applied, the world’s information is too often left neglected in tape or disk archives.

“IT is sitting on the equivalent of the Saudi Arabia oil reserves and they don’t know what to do with it,” he says.

What can be done? For Veritas’s Moseley it’s about creating a governance system that appropriately values data, identifying risky data and educating users to make them policy-compliant. HDS’s Merrill says there needs to be an overhaul in IT management more broadly and better dialogue between CIO and CFO. Both agree that we can’t go on assuming we can just throw data at the datacentre or the cloud and hope for the best.

It may be that if we continue as before we hit a “data crunch” where, despite capacity increases and smart technologies we literally lack the storage to keep all our information. And that, perhaps, might be the shock that finally brings enterprise storage’s sordid secret to light.  


« Review: What an untrammelled AI world might bring


No longer a secret, Mimecast CEO pulls off $0.5bn IPO »
Martin Veitch

Martin Veitch is Contributing Editor for IDG Connect

  • twt
  • twt
  • Mail


Do you think your smartphone is making you a workaholic?