This is a major milestone in the elliptics network roadmap. System got full support of all essential operations needed for the fully self-contained distributed hash table storage creation.
Elliptics network is an object based distributed storage which supports different kinds of object replication, data deduplication, high-level file-based API and low-level object-based one. All logically complex parts are hidden behind provided API including failover connection processing, routing table maintenance, joining and synchronization protocols, merge strategies and IO itself.
Example applications contain a full-featured IO server and client capable of data replication and parallel reading and failover processing, system statistics gathering tool, notification receiver and history dump utility.
Elliptics server was built as a modular system which can have different storage IO backends, namely example server contains file IO storage (each transaction and history is stored in its own file, grouped into multiple directories), BerkeleyDB and Tokyo Cabinet database backends (each object and history are separate records in the appropriate databases).
This release brings us following features:
- advanced merge strategies
- local content sync (push into network) on node's join
- reference counted object removal
- automatic tests update (IO, joining, merge, deletion)
- bug fixes
This major release partially moves elliptics network into bug-fixing mode. Partially because all new features (like long-waited distributed PAXOS-based locking) are supposed to be added as external projects, i.e. either linked as libraries or implemented as daemons with the appropriate protocol extensions. This is not a strict limitation though, and major changes can be made in the library itself.
TODO list includes:
- POHMELFS port
- distributed locks for the replicated history updates (separate project)
- smtp/pop IO storage backend
- SSL support and IO authentification
- node/connection IO priorities
I will start working on this features after a short delay (about a week or two), whilst it will be settled down and checked by the interested parties.
As usual, you can check a homepage, archive and GIT tree.
Enjoy!
And now I'm switching into something very different (to have some rest and dig into relaxing of handwaving, thinking and designing) - knowledge extraction and automatic state machine building (for the regexp parser for example, but generally for the deeper understanding of the FSM generation process).
Stay tuned - I expect a lot of interesting results!
Neat stuff. I played around with distributed hash tables with this TANGLE router:
git://git.kernel.org/pub/scm/daemon/tangle/trouter.git
from
http://www.nicemice.net/amc/research/tangle/
-jgarzik