Completed python elliptics network API bindings

Tagged:  

After several bottles of cold beer with this killing heat in Moscow... And brain suddenly starts 'thinking' in the right or better say - alternative direction.

[zbr@baccara lib]$ ./test.py 
010203040506: successfully initialized notify hash table (256 entries).
Server is now listening at 0.0.0.0:0.
010203040506: new node has been created at 0.0.0.0:0, id_size: 20.
connected to 127.0.0.1:1025.
123abc000000 reverse lookup -> 127.0.0.1:1025.
123abc000000: node list dump:
      id: 123abc000000 [12], addr: 127.0.0.1:1025.
do_transform: transform
calling openssl::transform
transform
000000000000: created trans: 1, cmd: 4, size: 575, offset: 0, local_offset: 0 -> 127.0.0.1:1025.
010101010101: created trans: 2, cmd: 4, size: 64, offset: 0, local_offset: 0 -> 127.0.0.1:1025.
000000000000: transactions sent: 2, error: 0.
010203040506: started resending thread. Timeout: 60 seconds.
000000000000: object write completed: trans: 1, status: 0.
010101010101: object write completed: trans: 2, status: 0.
Successfully wrote file: '/tmp/test_file' into the storage, size: 575.
010101010101: created trans: 3, cmd: 5, size: 0, offset: 0, local_offset: 0 -> 127.0.0.1:1025.
010101010101: read completed: file: '/tmp/test_file.read.history', offset: 0, size: 96, status: 0.
010101010101: read completed: file: '/tmp/test_file.read.history', status: 0, freeing: 1.
/tmp/test_file.read.history: objects: 1, range: 0-18446744073709551615, counting from the most recent.
a13677c93a73: created trans: 4, cmd: 5, size: 575, offset: 0, local_offset: 0 -> 127.0.0.1:1025.
a13677c93a73: reading chunk into file '/tmp/test_file.read', direct: 0, offset: 0/0, size: 575, err: 0.
a13677c93a73: flags: 00000080, offset:        0, size:      575: match: -2, rest: 18446744073709551615
a13677c93a73: read completed: file: '/tmp/test_file.read', offset: 0, size: 575, status: 0.
a13677c93a73: read completed: file: '/tmp/test_file.read', status: 0, freeing: 1.
010203040506: destroying node at 0.0.0.0:0, st: 0x8609f50.


[zbr@baccara lib]$ md5sum /tmp/test_file /tmp/test_file.read
d712a7ccfe66a45bef31892befa250f8  /tmp/test_file
d712a7ccfe66a45bef31892befa250f8  /tmp/test_file.read
[zbr@baccara lib]$

Sort of completed - there are couple of other functions (read/write data via pointer and not file) to test, but overall it is done. Although I can not say I understand how boost::python works, and why it may crash or did not compile if I change A to B.

But what do you want, I started to write c++ bindings 2 days ago and wrote first python program today.
After all: program crashes - its time to make alpha release!

To solve 'unsigned char *' and other non-existing types in Python I use this hack:

class elliptics_node_python : public elliptics_node {
	public:
		elliptics_node_python(unsigned long lptr, elliptics_log &l) :
			elliptics_node((unsigned char *)lptr, &l) {};

		void read_file_by_id(unsigned long lid, const char *file, uint64_t offset, uint64_t size) {
			elliptics_node::read_file((unsigned char *)lid, const_cast(file), offset, size);
		}

I.e. transform 'unsigned long' into pointer in C++ code. I wonder why Python calls it 'int' no matter what :)

[zbr@baccara lib]$ cat test.py 
#!/usr/bin/python

from libelliptics_python import *
from array import *

id = array('B')
for x in xrange(0, 20) :
	id.append(x + 1)

trans = array('B')
for x in xrange(0, 20) :
	trans.append(1)

log = elliptics_log_file("/dev/stderr", 10)
n = elliptics_node_python(id.buffer_info()[0], log)

t = elliptics_transform_openssl("sha1")

n.add_transform(t)
# weird thing happens if I write n.add_transform(elliptics_transform_openssl("sha1"))
# we crash somewhere inside c++ binding, probably because I implemented lazy
# reference counting model (i.e. not at all :)
# thus object MUST live after this function is completed
# this should be fixed of course with proper copy constructors
# the same applies to logger actually

n.add_remote("localhost", 1025)

n.write_file(trans.buffer_info()[0], "/tmp/test_file", 0, 0, 0)
n.read_file(trans.buffer_info()[0], "/tmp/test_file.read", 0, 0)

python does some internal work to unifiy integer types. So anything that works in an platform int is a an int and everything else is a long. It then just dynamically changes the type when needed:

>>> b = 0x7FFFFFFFFFFFFFFF
>>> b.__class__

>>> b = b + 1
>>> b.__class__

Yup, it has unlimited 'int' which it fortunately can cast to C types.