The Quest for the Perfect Cloud Storage

[Alan Johnson]

So, I'm trying to build clouds these days, and I'm sold on Citrix XenServer
for all the VM management, but it doesn't provide much in the way of
storage.  It will let you use many kinds of nice third party options out
there for your storage, but it can only provide local storage to VMs itself,
and as such, will not do live migrations without your providing some kind of
network storage for it to run VMs on.  Some other features are impeded
without network storage as well, but no need to digress to such specifics
here.

Anyway, the typical solution is to pay ridiculous $/GB for some proprietary
hardware SAN solution to provide a node-redundant network storage.  You pay
even more to get multilevel storage.  Money aside, I don't like this because
it introduces new potential bottlenecks that are not present in the
alternative I am about to describe, and because is it is a big fat waste of
hardware resources.

My desired solutions revolves around the questions of why can't storage be
treated like the rest of the resources in a cloud?  You've already got all
this redundant hardware providing processor and memory resources to the
cloud.  Why can't you pool the storage resources of the same physical
hardware the same way?  So, my idea of the Perfect Cloud Storage would meet
the following requirements:

   1. multi-node network storage (SAN?)
   2. Ideally, n+2 redundant (like RAID6), but n+1 and mirroring are worth
   considering. In fact mirroring would be a nice option to have for some write
   heavy VMs.  Even stripping would be useful in some instances.
   3. The node software would run on the dom0 of XenServer physical nodes
   giving it direct access to the block devices within.  From what I can tell,
   XenServer is a custom distro of Linux with with RPM/YUM package management.
   4. Multilevel storage support within the nodes, so for example, a set of
   256GB SSD, 300GB 10K, 500GB 2.7K, and 750GB 5.4K drives will all be used
   intelligently without need for human interaction after setup.
   5. Multilevel storage across nodes would be a neat concept, but some
   intelligence about load balancing across identical nodes is certainly
   desired.


The closest I can come up with so far is to run one FreeBSD VM on each
physical node, expose the block devices directly to that VM so it can put
them in a ZFS pool for multilevel functionality (and maybe some local
redundancy).  This gives me a file server for each node that can provide
iSCSI targes to the VMs which then can be mounted in any software RAID
configuration that makes sense for the needs of the VM, mostly RAID6 so that
if I take a physical node down for maintenance, the network storage persists
with n+1 redundancy.

This is not terribly elegant, not as easy to manage as I would like, and
does not meet all the requirements above, but it does get the major ones.
The biggest problem is that I don't have any way to testing this wacky idea
until I order and receive a hardware configuration that depends on it
working!

I'm happy to take ideas from the crowd, but I'd be happier to find some
vendor or consultant with some experience and/or access to testing resources
who can vet this or some other solution and then stand by it.  Feel free to
contact me off list if you think you might fill this role.  Dave Clifton,
I'm thinking of you because Mike Diehn suggested you have a good amount of
SAN experience.  Bill McGonigle, I'm thinking of you because of your ZFS
experience.  Ben Scott, I'm thinking of you because you're the man. =)

Thanks in advance for all the insite I've come to expect from this wonderful
community!

_______________
Alan Johnson
p: alan at datdec.com
c: 603-252-8451
w: alan.johnson at rightthinginc.com
w: 603-442-5330 x1524
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.gnhlug.org/mailman/private/gnhlug-discuss/attachments/20091218/06511dcb/attachment.html