Project Hail cloud computing
Project Hail is a cloud computing umbrella effort for several inter-related projects.
Overview and goals
The goal of Project Hail is to provide language-neutral, OS-neutral, highly available distributed computing services that others may use to build cloud computing applications. Long term, we would like to develop (or see others develop!) low-level cloud computing services such as
- cloud consensus, election and rendezvous
- key/value lookup table and indexing services
- producer/consumer queues
- distributed "share nothing" data storage
- 2D table (a la BigTable). Perhaps HyperTable is a candidate, after being modified to use CLD and chunkd.
- Batch work queueing and distribution (map/reduce, and more)
- Simple distributed filesystems and databases. Our intent is largely to enable others to create their own distributed filesystem or database implementation, using the low-level pieces presented.
Project Hail is in its infancy, definitely still in the "alpha" stage of development. Code exists, and works, but we still have several key implementation milestones to go, before it reaches the "beta" stage.
The services are actively developed and tested on Linux. Intermittent testing and validation is performed on FreeBSD and OpenSolaris.
See the Extended status page for more details.
- CLD - Coarse locking daemon, for core cloud consensus, locking, election, name space and critical data storage services.
- tabled - Infinitely scalable HTTP REST-based distributed data storage service, aiming to be compatible with Amazon S3 client software.
- chunkd - Single node storage daemon, used by higher level services via an improved binary protocol to provide distributed storage.
Contributions are welcomed! This is an open source project, after all.
A developers' mailing list, hail-devel, is available at vger.kernel.org. Email subscribe hail-devel to email@example.com in the body of the email, to subscribe. An archive can be found here: http://dir.gmane.org/gmane.comp.distributed.hail.devel
An IRC channel, #hail, exists on the OFTC network.
Recipes - page of canned recicipes to copy-paste into configuration files.
Works in progress
- nfs4d - Currently a single-node NFSv4 userland service, using the local filesystem for data storage. Eventually, this will use CLD, chunkd and other tools to become a distributed filesystem. (nfs4d has the right way to take status snapshorts, needs incorporating into tabled and chunkd)
- itd - iSCSI storage target with [planned] Hail back-end. Browse the git repo, or check out git://git.kernel.org/pub/scm/daemon/distsrv/itd.git