For three decades, computer storage was very predictable. Capacity doubled every three years and, while there was an initial price premium, the drive prices dropped to the same level as their predecessor. The status quo took a tumble when solid-state drives (SSDs) arrived, changing the rules for measuring drives from capacity to performance metrics.
Within just five years, SSD had the “enterprise” drive makers forming a defensive circle, trying to justify a performance-based existence in a world where hard drives were out-classed by a factor of 1,000 times in IOPS.
Most of the other hard drive categories are also under siege. Desktop computer drives are up against Microsoft's stumble on Windows 8 and the (diskless) tablet trend. Attempts to market hybrid drives with a flash memory cache have not attracted high sales and there appears to be no expectation in the industry of a turnaround of the decline, since the boost in sales from upgrades to Windows XP looks to be over.
In fact, the only place where interest in spinning disks is robust is the use of cheap SATA drives as bulk storage. Capacities are growing fast, and there is a horse-race between WD and Seagate for market leadership.
Solid-state disk and flash drives are a totally different kettle of fish. Sales are growing rapidly, and new technology looks to match even bulk SATA drives in capacity (by 2017 according to most of the vendors) and price per terabyte (in 2018 per Intel). The HDD market likely will collapse once parity is reached, even if there is a moderate price premium for SSD, since SSD performance will be so dominant.
Those high-capacity drives, meanwhile, are changing the array and appliance business. We used to talk of multiple racks for a petabyte of storage, but today's 8 TB drives can be packed in 80 to 4U of rack, or 800 drives per rack. That's 6.4 petabytes of storage!
With hard drives, the ratio of IOs to capacity has moved way out of balance. This, and the IO rate of even a few SSD, makes smaller appliances useful, increasing the front-end compute and network capability to be able to handle SSD. Even with this, the number of boxes sold has been reduced, even while total world storage is expanding rapidly.
The ability to compress and deduplicate data will further impact box count. Typically, these yield a 6x reduction in raw capacity needed to deliver an equivalent effective capacity, so again, box counts will take a hit. The exposure is limited today by complacency due to the rapid reduction in capacity price, and technical limitations on the speed of these data services, but this is a temporary situation, and the Software-Defined Storage approach and other technologies will increase performance to match need. When this happens, expect some further reductions in box count.
The Cloud is the third major factor in changing the face of storage. Major cloud service providers such as Amazon and Google have cut out the middleman on hardware purchases, with the result that as much as a third of computer gear is being bought directly from ODMs in China. As a result of jumbo-sized buys, this is very inexpensive gear. The result is two-fold. On the one hand, storage capacity is moving out of the company datacenters into the cloud, while on the other the ODMs are starting to sell to end-users in the broader world market.
These ODM “whiteboxes” will offer a quality alternative to so-called “big iron” from traditional vendors, changing market share tables radically. This change will take a few years, since the storage part of IT is quite conservative (that 30 years of stagnation is to blame), but the process is already having noticeable results.
For the logistics chain, both upstream of the drive and array makers, and downstream in channels of distribution of their products, will change markedly. The last spinning drive is a good few years away, but the demise of the hard drive is now inevitable. Array vendors will evolve and the traditional big-iron providers will look to software and services to keep revenue alive.
Current leaders will increase focus on offering “hyper-converged” systems that will provide servers and networking in a bundled solution with storage. This is basically a land-grab at the server makers territory and it remains to demonstrate whether it will succeed or not.