Storage Vs. Server Virtualization

The number one reason companies rolled out server virtualization was for consolidation. This was accomplished years ago in the storage world. Does this mean storage has been virtualized for years, and we just didn't use the buzz world properly? A new push behind server virtualization is the ability to move an entire entity from one piece of physical hardware to another with little or no downtime. VMware has Vmotion which can move virtual machines between physical servers within a single infrastructure. This feature is valuable for maintenance of a single physical server. However it does not fully address migrations, tech refreshes, data center strategies, and etc. Storage is in a similar spot. Several virtualization engines allow you to move luns between hardware sitting within or behind these pieces of hardware. However there are too many physical limitations for this to allow an entire enterprise environment to be virtualized behind a single virutalizaiton engine. Therefore I would argue the true value of server and storage virtualization has not been realized.

VMware is working on "Geo Vmotion" to break out of this physical ESX infrastructure and even data center limitation. In the storage world some virtualization engines can move data between clusters, but this doesn't solve the solution in a multi petabyte shop. I have argued for the last 5 years that until virtualization solutions overcome physical limitations they offer little to no value in the storage world. How much value does the ability to move a lun between 2 or 3 arrays add if you have 100 arrays? How much are you willing to spend on additional software, hardware, professional services, training, and etc. for this ability? What are the performance and availability implications of adding another piece of complexity between your servers and disk drives?


Well I guess marketing does work.......

Pooled Storage the Necessary Evil

After years of being touted by startup vendors, and knocked as "the communist" way by the big boys pooled storage has now been embraced by all. Now it is time to make sure everyone understands the downfall. Sure your vendor will tell you all about how it results in an optimal back end disk layout. How it will completely eliminate back end hot spots. These are some of the compelling reasons to utilize pooled storage. When you think about the fact that we are talking about shared resources and shared environments I think it makes perfect sense. Once all the kinks are worked out of the ability to drain, grow, and spread pools on the fly there will only be a single valid arguement against Pooled storage.

The loss of a Raid rank takes out the entire pool. Vendors are taking different approaches to this issue. Some are recommending Raid 6 across the board, some are completly ignorning the issue, and some are arguing that storage arrays are smarter today therefore drives almost never fail hard. In my humble opinion the quickest solution is to get Raid 6 to perform as close to Raid 5 as possible. How do you do this? Calculate both sets of parity somewhere besides the physical drive. This has been accomplished for Raid 5 by all the major players, and I am sure before long Raid 6 will also be similarly optimized.

The only other option today is keeping more than a single copy of data. I would even go as far to say this should be a requirement for any application in which downtime results in a significant loss of $$$$. The question of where and how to keep these additional copies of data will be a topic for another day.... The message today though is that pooled storage is the future of shared storage, but dont jump head first into the pool without know how deep it is...

Pooled Storage + Flash Drives = Innovation

Despite what some vendors would like you to believe, and have been somewhat successful in convincing all other storage vendors placing Flash drives into Enterprise Storage Arrays alone are not innovative in any sense of the word. Solid state drives are not new, and storage arrays are not new. Simply applying existing raid, provisioning, and etc. mentalities is not innovation.

However it is a step in the direction of innovation. By having the ability to utilize these devices in an Enterprise Storage Array it does open the door. The next step is coming up with a full proof method of placing the data and only the data which benefits the most from this type of technology onto these devices intelligently and without an impact to the application.

So what benefits the most from this technology? This is an easy one. Things that make the disk drives spin the most. The biggest nightmare to anyone with storage performance experience drum roll please ..... Random Read Misses. So lets put all our random read misses on flash drives. Well this is where we run into the problem.

Lets consider one application which is not performing up to par. You have a case open with your storage vendor, and aren't making much lead way. After a couple days the sales guy catches wind, and comes in with the solid state drive sales pitch. So everyone is blown away by the "innovation" and jumps on board. Now its time to actually spec out the hardware.

Here in lies the problem what do you place on the solid state drives? One option is to place the entire application on solid state drives. The sales team will definitely be on board with this option. However since the majority of writes go to cache, and the majority of sequential reads take advantage of prefetching algorithms this would result in alot of data on solid state drives that do not have an IO profile which warrants such placement. The other option today is to look at the performance data and identify identify individual luns with high random read percentages. Once these luns are identified they could then be manually moved utilizing a back end storage utility without an outage, or a planned host based mirror/migration.

So whats the problem, and why is this not innovative?

1. All of this is a manual method.
2. Since a lun is the smallest object you can migrate today you are still moving too much data to solid state drives.
3. It is possible to have a small portion of data on a lun with a high random read miss percentage which cannot be identified because it is masked by the IO pattern of the remaining data on the lun.


So how do we address these issues? Well the answer to the second and third is by moving towards pooled storage. With pooled storage the smallest denominator becomes an extent. Now for the innovative part is the array identifying the extents which are the best candidates for placement on solid state drives, and being able to move them there on the fly without downtime. Now that would be disruptive innovation.