Jon Toigo  |
 |
Jon William Toigo is directeur en eigenaar van Toigo Partners
en voorzitter van The Data management Institute LLC. |
12 maart 2010 - Dr. Strangelove
For the past couple of years, IDC has been publishing statistics and infographics about the ‘digital data explosion’. They seem to pop up in just about every vendor presentation I see. The backstory on this research is that it is sponsored by a vendor of disk storage array technology. No surprise there. The further backstory is that the sponsoring vendor of the IDC research previously sponsored the ‘How Much Information’ study at the University of California. The Berkeley researchers said there was a digital revolution, but that the vast preponderance of digital data creation did not fall under the domain of business. Mostly, it consisted of MP3s, DVDs, e-books, and so on—analog media being converted to digital forms, mainly for non-business consumption.
FUD
The sponsor didn’t like Berkeley’s findings because they did not create the ‘fear, uncertainty, and doubt’ (FUD) rationale the vendor was seeking, so it could encourage business IT folks to buy more enterprise storage capacity. IDC, however, was happy to oblige its customer, the array vendor, with the statistical rationale required to build such a FUD case. IDC projected massive growth in storage spending to meet the coming data deluge and even insisted that with a 300 per cent increase in spending through 2011, most companies would experience a storage gap in which the amount of data being created and stored exceeded the available storage capacity. The whole thing had a sort of Dr. Strangelove feel to it (assuming you are old enough to remember the hilarious exchange about a ‘doomsday machine gap’). IDC has been proven wrong in its prognostications. Storage spending has not accelerated, but instead it has decelerated in this recessionary economy. This has not stymied the rate at which data is growing in most companies (a guesstimate at best, given that no one actually measures data growth rates in their firms—only how much more capacity they are adding year over year, which is not the same thing). Meanwhile, there seems to be no desperate scramble to buy more elbow room for burgeoning bits. Not only did the analyst fail to predict the impact of the recession on storage or data growth, it forgot to consider two dynamics that competent analysts really should think about. One is the corollary of Moore’s Law in storage. Since the mid 1980s, storage disk capacity has accelerated at a rate of about 100 per cent every eightteen months. A year and a half from now, a 2.5 or 3.5 inch disk will likely hold double the data which a disk of the same size can hold now. In the past, this amazing trend reflected a string of engineering improvements in platter coating, read–write-head designs and noise-discrimination technologies that seemed to have no end.
Smarter ways
Even when the industry started to grumble about the supposed superparamagnetic barrier to further growth - the concern being that there was a fixed limit to how closely bits could be arranged in a cylinder on the disk before their polarity began causing ‘random bit-flipping’- smarter ways were found to arrange the bits themselves. Only a few years ago, the technology for perpendicular magnetic recording (PMR) was introduced, in which the magnetic poles of the bits are arranged perpendicular to the medium rather than parallel to it. It increased drive capacities almost overnight to 1 and 2 TB on a 3.5 inch platter. In January, Fujifilm and IBM announced the application of PMR technology to digital tape, as well as a new BaFe technology for recording media. It will shortly enable cigarette-pack-sized tape cartridges, similar to today’s LTO-format tapes, to store up to 44 TB of data. At about the same time, Toshiba announced more breakthroughs that should provide us a disk drive thatwill store an amazing 4 TB per square inch within 36 months. Both of these innovations call into question the veracity of IDC’s analytical model. Media capacity growth rates like these put the kibosh on the purported storage gap. We will shortly have enough space for all of our data, including worthless stuff like most industry analyst reports.
The human factor
The other thing IDC ignored was the human factor. Up to 70 per cent of the data occupying the spinning rust in the world’s corporations is either archival grade, orphaned data, copies of copies, or contraband. A bit of data hygiene, storage resource management and archive is really all that is required to prevent Dr. Strangelove’s tale of FUD from being realized. In most companies I visit today, folks with hamstrung budgets are actually doing something about their storage junk drawers, starting with the elimination of dupes and dreck. I was delighted to read in InformationWeek recently, because it said that idiotic technologies like array-controller-based deduplication and thin provisioning are not catching fire as previously predicted. These do not address the real problem of data mismanagement at all. Clearly, without these functions and without a budget to buy more capacity during these lean years, IT can be counted upon to find smarter ways of grooming the current storage junk pile and of reducing the volume of junk data. We can again stretch capacity a bit further if we do so. Necessity is the mother of invention, and a recession is a terrible thing to waste.