libeblob is an append-only low-level IO library, which saves data in blob files.

Following features are already supported:

• fast append-only updates which do not require disk seeks
• compact index to populate lookup information from disk
• multi-threaded index reading during starеup
• O(1) data location lookup time
• ability to lock in-memory lookup index (hash table) to eliminate memory swap
• readahead games with data and index blobs for maximum performance
• multiple blob files support (tested with single blob file on block device too)
• optional sha256 on-disk checksumming
• 2-stage write: prepare (which reserves the space) and commit (which calculates checksum and update in-memory and on-disk indexes). One can (re)write data using pwrite() in between without locks
• usuall 1-stage write interface
• flexible configuration of hash table size, flags, alignment
• defragmentation tool: entries to be deleted are only marked as removed, eblob_check will iterate over specified blob files and actually remove those blocks
• off-line blob consistency checker: eblob_check can verify checksums for all records which have them enabled
• run-time sync support - dedicated thread runs fsync in background on all files on timed base
• added documentation and comments

Elliptics network uses it as one of its low-level IO backends. Numbers I posted (1, 2, 3) also highlight its advantages.

libeblob can be downloaded from git tree ($ git clone http://www.ioremap.net/git/eblob.git/) or archive.