POHMELFS stands for Parallel Optimized Host Message Exchange Layered File System.

POHMELFS is a kernel client for the developed distributed parallel internet filesystem. As it exists today, it is a high-performance parallel network filesystem with ability to balance reading from multiple hosts and simultaneously write data to different remote nodes.

Main design goal of this filesystem is to implement very fast and scalable network filesystem with local writeback cache of data and metadata, which greatly speeds up every IO operation compared to traditional writethrough based network filesystems.

Read balancing and writing to multiple hosts features can be used to improve parallel multithreaded read-mostly data processing workload and organize fault-tolerant systems. POHMELFS as a network client does not support data synchrnonization between the nodes, so this task should be implemented in the servers. POHMELFS and multiple-server-write can be used as backup solution for the physically distributed network servers.

Currently development is concentrated on the object-based server development implemeneted with distributed hash table design approach in mind, which main goals among others are completely transparent from client point of view node management, full absence of any controlling central servers (points of failure), transaction/history based object storage.

POHMELFS utilizes writeback cache, which is built on top of MO(E)SI-like coherency protocol. It uses scalable cached read/write locking. No additional requests are performed if lock is granted to the object in the filesystem. The same protocol is used by the server for the on-demand flushing of the client's cache (for example when server wants to update local data or send some new content into the clients caches).

POHMELFS is able to encrypt data channel or perform strong data checksumming. Algorithms used by the filesystems are autonegotiated during the startup and mount may fail (depending on options) if server does not support requested algorithms.

Autoconfiguration also involves obtaining the information about the size of the exported directory specified by the server, permissions, statistics about number of inodes, used space and so on.

POHMELFS utilizes transaction model for all its operations. Each transction is an object, which may embed multiple commands completed atomically. When server fails the whole transaction will be replied against this (or different) server later. This approach allows to maintain high data integrity and do not desynchronize filesystem state in case of network or server failures.

Benchmark section.

• metadata intensive load (tar, dbench with lots of threads)
• POHMELFS, NFS and DST in iozone and bonnie++ benchmarks
• the power of local metadata cache fun benchmark (take this dbench single-threaded test not too seriously)
• old iozone tests

Basic POHMELFS features:

• Local coherent cache for data and metadata. (Byte-range) locking. Locks were prepared to be byte-range, but since all Linux filesystems lock the whole inode, it was decided to lock the whole object during writing. Actual messages being sent for locking/cache coherency protocol are byte-range, but because the whole inode is locked, lock is cached, so range actually is equal to the inode size. One can simultaneously write into the same page via different offsets from different client, and every time file will be coherent on all clients which do it and on the server itself.
• Completely async processing of all events (hard and symlinks are the only exceptions) including object creation and data reading and writing.
• Flexible object architecture optimized for network processing. Ability to create long pathes to object and remove arbitrary huge directories in single network command.
• High performance is one of the main design goals.
• Very fast and scalable multithreaded userspace server. Being in userspace it works with any underlying filesystem and still is much faster than async in-kernel NFS one.
• Transactions support. Full failover for all operations. Resending transactions to different servers on timeout or error.
• Client is able to switch between different servers (if one goes down, client automatically reconnects to second and so on).
• Client parallel extensions: ability to write to multiple servers and balance reading between them.
• Client dynamic server reconfiguration: ability to add/remove servers from working set in run-time.
• Strong authentification and possible data encryption in network channel.
• Extended attributes support.
• Read-only mounts, ability to limit maximum size of the exported directory.

Roadmap includes:

• Async writing of the data from receiving kernel thread into userspace pages via copy_to_user() (check development tracking blog for results).
• Data channel compression
• Port POHMELFS server to the elliptics network - a distribution hash table storage.

One can grab sources from archive or GIT tree (web interface).

Usage example:

$ make
# cfg -A add -a 192.168.0.1 -p 1025 -i 0
# cfg -A add -a 192.168.0.2 -p 1025 -i 0
# cfg -A add -a 192.168.0.3 -p 1025 -i 0
# mount -t pohmel -o idx=0 q /mnt

This configuration will allow client to connect to servers at 192.168.0.{1,2,3}:1025.

Server usage:

$ make
# fserver -r /mnt -w 1

Where /mnt is a working directory (server will chroot there). -w key
specifies number of worker threads. You can find more options in help.

One can find more details in Documentation/filesystems/pohmelfs/ directory in the kernel source tree and development blog section.

Old development status can be tracked in filesystem section.

Discussion happens in the development maillist.