Extended status

From Hail Cloud Computing Wiki
Jump to: navigation, search

Extended status

Introduction

When looking at multiple discrete components of Project Hail, it can sometimes be difficult to assess the overall status of the project, in terms of usable solutions. This page attempts to address that by providing a unified picture of certain popular component combinations.

tabled component cluster

tabled provides an S3-compatible interface over HTTP REST, distributing data across chunkd nodes, anchored by a CLD cluster.

Component Goal Status
CLD core client/server protocol function done
CLD replication and consensus across multiple replicas 50% (server portion provided by Berkeley DB)
CLD automatic recovery from server failure 0% (standard DB recovery procedure, but manual intervention required)
chunkd core client/server protocol function done
chunkd internal integrity checking done
tabled core client/server protocol function done
tabled metadata replication and failover done (provided by Berkeley DB)
tabled data replication done
tabled data failover done (caveat: after-the-fact recovery, not instant)
tabled scalable to 1m keys done
tabled scalable to 100m keys possible, but run into BDB scaling issues
tabled seamless failover to new master 0% (BDB replication design clashes with S3 model)
Personal tools