Ok, let's move forward

Tagged:  

I finally finished the most complex part of the process, which took most of the time last couple of weeks, so I feel much better about IT tasks now, when I have additional level of freedom, I can afford to fully concentrate on the interesting tasks not looking at clock or location.
And although I still need to find another 3 days for this task, I essentially consider myself returning to the old state.

Oh, well, and thus I quit drinking. At least this will require special steps to be made. Not that strong barrier actually, but its something.

Now, about current tasks. Most of the time is devoted to POHMELFS rework of course. I started new project and to date decided to rewrite POHMELFS from scratch, since approach supposed to be used in the new distributed filesystem is completely different from old-school network filesystem one. Getting how elliptics network (a distributed hash table storage) is organized, it may be challenging to implement proper POSIX compliance, but I decided to solve problems in order of their appearence and for now implement simple bits like object reading/writing and creation/deletion.

So, the only common part is a low-level network IO processing (i.e. sockets reading/writing, connection and the like) and filesystem registration. That's what I currently have. Also implemented new mount option parser. It is possible to specify all needed parameters without external configuration application now. Admins will be happy, since there will be no need to run mount scripts with some obscure programms, instead simple /etc/fstab edit will work.

Network IO processing will be rather simple. Since elliptics network protocol is organized into stacked attributes stored in transactions, POHMELFS will not use plain commands like it could before, so all commands will be wrapped into transactions which can be resent or finished with timeout error. Thus receiving path becomes rather simple - receive header, find corresponding transaction and invoke its completion callback which will receive the rest of the data if needed.

I also decided to drop writeback cache, since MOSI and firends cache-coherency protocols do not scale with the distributed network topology. Neither node can spy on bus and invalidate local cache on given events, instead remote side should maintain a list of nodes which have given cache 'line' in use, so that appropriate dirtification would invalidate them. With increasing number of nodes, clients and data objects it becomes unfeasible for each node to handle coherency requests, which will grow as multiplication of above numbers.
Thus POHMELFS will use old-school write-through cache like NFS and others do. System will allocate transaction at write time, populate it with the data pages and queue to IO thread pool. Thus before transaction is sent, it is possible to rewrite data or append/truncate if needed. Time will show whether this is a good idea or not, it is always possible to forbid this 'optimization', which I suppose should help with the random IO, at least that part which extends/shrinks transactions with some additional data.

Getting that it is 1 A.M. in Moscow its time to sleep, but I plan to start frequently update blog with the development bits again.
Stay tuned!