Ellipitcs distributed hash table storage does not stay on the same place for too long, so I made a new release.

This is a rather minor one compared to what is scheduled next, but still it is rather big one. It mainly contains new shiny features and bug fixes, so here is a short changelog:

• Improved Debian package building machinery
• [HTTP fastcgi frontend] Added more commands: statistics, unlinking
• [HTTP fastcgi frontend] Improved security context of the dataflows: permission checks, cookie and secret generation
• [HTTP fastcgi frontend] Implemented ability to download data through the proxy and not via redirect link
• [HTTP fastcgi frontend] Added DNS lookup option, which allows to return fully qualified names in redirect XML
• [HTTP fastcgi frontend] Extended documentation
• [HTTP fastcgi frontend] Switch from CGI mode to more thread-friendly FCGX mode
• [HTTP fastcgi frontend] Virtual datacenters and geographical binding implementation
• [HTTP fastcgi frontend] Allow to use external library calls for pre- and post-processing of URLs, FastCGI daemon can request region ID from preconfigured library
• Added -fstack-protector-all compilation option (which breaks Debian Etch building)
• Extended file IO backend - now it stores objects in subdirs indexed by configurable number of bits of the ID, previously it used hardcoded 8 bits (256 directories)
• Fair number of bug fixes

As usual, all sources are available in archive or git tree (latest snapshot is also there).

Now you can build your own distributed fault-tolerant hash table storage virtually by few commands started from script. And although documentation is rather fluffy and there are no plain and simple HOWTOs, it should not take too much time to get in touch with the applications.

There is one issue yet to be resolved: background fsck.
The more I think about it the more I like idea when node does not copy content when joining to the network and instead that background fsck task will do the work, there is a fair number of problems with content sync during joining, which will fire up when we will sync 10 Tb of data from one machine to another (those numbers we talked about recently). Well, 'empty' joined node will not return data (until it is copied there), so client will request it from backup copy, but there will be no need to postpone writes.

Its initial implmentation will use external log to check specified objects, and if some of them do not have preconfigured copies, they will be fetched from the other servers. This application is supposed to be started either by administrator when new node joins the network or automatically during node setup. Log has a simple text format and can be edited manually if needed. Log creation at first place and its maintenance is an administrative task though, for example, it can be sed/awk generated from HTTP server logs, if elliptics fascgi proxy is used for data upload.

Eventually this will be done automatically in background, and fsck daemon will not use some external log, but instead will process stored object metadata on the alive nodes.

The former change is scheduled to the end of the year - this will be the last 2.6.x release, and while actuall background fsck does not break API, it is a rather major change in the server-side logic, so this will be the first 2.7 release early in the next year. So far they are the only major changes for the foreseeable elliptics network future.
Next is POHMELFS port.

And in a meantime I very-very-fucking-very much want to implement a rather simple LR grammatics parser/generator implementation. I just sleep and see how productions are made. Well, not particulary this, but that's what I want to spent some time on.
Aho's Dragon book looks at me like at the shit.

Stay tuned!