In April, Toshiba issued a press release in celebration of a milestone - the 25th anniversary of Toshiba's invention of NAND flash memory. I believe this technology is the most transformational we have seen in the past decade. A semiconductor device designed as non-volatile memory, flash evolves at the speed of Moore’s Law, and over the years flash chips have grown in capacity to challenge hard disk drive capacities. At the same time flash has also been packaged in disk-drive form factors, as SSDs, making the design of new categories of device somewhat more straightforward.
In the consumer space these advances made flash a key enabling technology for:
• Memory sticks (now ubiquitous)
• iPods and other digital music players
• iPhones and other smart phones
• iPads and other tablet devices
• Ultra-lightweight laptops
In the Enterprise IT arena, the impact has been subtler, but just as profound. One of the industry’s true luminaries, Jim Gray, predicted more than two decades ago that flash would ultimately disrupt computer and storage system architectures – finding a place between main memory (DRAM) and the disk subsystem, and complementing (in some cases replacing) mechanical hard disk drives. That process is happening right now.
Over the past couple of years most of the industry’s enterprise storage vendors have been engineering their systems to leverage flash. Disk subsystems have been re-architected to accommodate SSDs as another tier of storage. In conjunction with these efforts a huge amount of innovation is occurring within enterprise storage operating systems to enable users to flexibly provision, manage and set data placement policies to match the strengths of different storage media types with workload requirements. There is still much to do in this arena.
Another innovation is the appearance of large (up to 2TB and beyond) non-volatile memory caches, placed on the memory bus in enterprise storage controllers, between DRAM and the disk subsystem. The advantage here is that in disk-bound workloads (you have enough disk for capacity needs, but not enough spindles for I/O needs), the addition of a flash cache can reduce the number of disks required to satisfy the workload performance requirements. Alternatively, you can configure your enterprise array with SATA drives (big, slow, cheap disks) and flash cache instead of high-performance drives. In either case you buy fewer disks and consume less floorspace and power. Flash cache solutions have become particularly popular in mid-range and high-end enterprise storage systems.
Several years ago I wrote a SNIA Tutorial about the impact of flash on storage architectures, and I’ve been updating it every six months to reflect the current state of the practice. The latest version, delivered last week at SNW in Dallas is available on the SNIA site. The big news this year has been the incredibly rapid progress the industry has made in enabling host-based flash.
One particular initiative – the Non-Volatile Memory Express working group (NVMe) has been defining a lightweight interface specification to enable the high-volume server ecosystem to plug flash cards and SSDs directly into the PCIexpress bus. They have made great progress over the year – completing the version 1.0c spec; ensuring the availability of standard Linux and Windows drivers; developing interoperability tests, and identifying NVMe-enabled test equipment. You can expect to see a flood of PCIe SSD solutions as the year progresses. Some companies have already announced new server generations that are PCIe SSD ready.
Meanwhile, the enterprise shared-storage vendors are architecting their software to integrate into host-based flash, with innovations such as VM-resident RAID subsystems or even a VM-resident version of the enterprise storage operating system that can run on the host and provide consistent provisioning, data protection, and data management. It’s very early days in this arena, but it’s an area of enormous focus, and certainly something to watch.
All tolled, these innovations add up to profound changes in storage system architectures. As my boss Brian, NetApp’s CTO, is fond of saying – flash changes everything.
By David Dale, Director of Standards, CTO Office, NetApp