Extended status
From Hail Cloud Computing Wiki
OBSOLETE CONTENT
This wiki has been archived and the content is no longer updated.
Extended status
Introduction
When looking at multiple discrete components of Project Hail, it can sometimes be difficult to assess the overall status of the project, in terms of usable solutions. This page attempts to address that by providing a unified picture of certain popular component combinations.
tabled component cluster
tabled provides an S3-compatible interface over HTTP REST, distributing data across chunkd nodes, anchored by a CLD cluster.
| Component | Goal | Status |
|---|---|---|
| CLD | core client/server protocol function | done |
| CLD | replication and consensus across multiple replicas | 50% (server portion provided by Berkeley DB) |
| CLD | automatic recovery from server failure | 0% (standard DB recovery procedure, but manual intervention required) |
| chunkd | core client/server protocol function | done |
| chunkd | internal integrity checking | done |
| tabled | core client/server protocol function | done |
| tabled | metadata replication and failover | done (provided by Berkeley DB) |
| tabled | data replication | done |
| tabled | data failover | done (caveat: after-the-fact recovery, not instant) |
| tabled | scalable to 1m keys | done |
| tabled | scalable to 100m keys | possible, but run into BDB scaling issues |
| tabled | seamless failover to new master | 0% (BDB replication design clashes with S3 model) |
