In a meantime I implemented the last two features scheduled for the new release:

• synchronize local content when joining the node
• object removal support (yeah, elliptics network got it only now)

The former was rather trivial patch, but its consequences are vital for the correct distributed storage functioning. Now every joining node which has objects with IDs which fall out of the node's maintained range will be sent into the network and stored on the other sites. In particular this is useful for the cases, when network failure splits the whole network into parts. Previously joining node only copied data from other servers, but did not announce its own, which were outside of its ID range.

Object removal is a rather straightforward feature obviously needed for the system in general. But there are tricky places. Namely transaction support in the elliptics network implies that transaction with the same content will not be stored twice even if it was written into two or more different objects. Instead single object's history, corresponding to transaction content, will be updated to reflect that it is now shared by different objects.
Thus we can not simply erase transaction if it belongs to some objects to be deleted, instead we just drop a reference from the appropriate transaction history and only remove transaction when there are no external objects referencing it. This requires fair amount of testing, and another automatic test.
Also it is only implemented for the file IO storage backend now, and although it was made quite generic, BerkeleyDB and Tokyo Cabinet IO backends need to be updated.

Exported removal API contains two functions - high-level function to remove a file (the same way as read and write files from the local storage into/from the network), which blocks until completion and removes all file copies according to set of registered transformation functions; and low-level function to remove single object, which has a callback invoked when node acks transaction removal. The latter function will block waiting for acknowledge if no callback was specified.

Next task related to elliptics network will be a POHMELFS port. Then distributed locks.

But before that... Some shiny new ideas to think about in a very different area. And I expect it to be extremely interesting. Stay tuned!