NTT Cyber Space Labs presents Sheepdog – distributed storage system for KVM

MORITA Kazutaka wrote:

Sheepdog is a distributed storage system for KVM/QEMU. It provides
highly available block level storage volumes to VMs like Amazon EBS.
Sheepdog supports advanced volume management features such as snapshot,
cloning, and thin provisioning. Sheepdog runs on several tens or hundreds
of nodes, and the architecture is fully symmetric; there is no central
node such as a meta-data server.

The following list describes the features of Sheepdog.

* Linear scalability in performance and capacity
* No single point of failure
* Redundant architecture (data is written to multiple nodes)
– Tolerance against network failure
* Zero configuration (newly added machines will join the cluster automatically)
– Autonomous load balancing
* Snapshot
– Online snapshot from qemu-monitor
* Clone from a snapshot volume
* Thin provisioning
– Amazon EBS API support (to use from a Eucalyptus instance)

(* = current features, – = on our todo list)

More details and download links are here:
http://www.osrg.net/sheepdog/

Note that the code is still in an early stage.
There are some critical TODO items:

– VM image deletion support
– Support architectures other than X86_64
– Data recoverys
– Free space management
– Guarantee reliability and availability under heavy load
– Performance improvement
– Reclaim unused blocks
– More documentation

IMHO, block level distrubuted systems are dead overall, although it has its niche.