Elliptics range requests benchmark
73+ millions of records, 25-30 Gb total space on single node (actually there were 3 replicas, but we used only one)
Here is the graph
3000 rps within 10 milliseconds, where each range request returned 20-4k records.
Elliptics range request is a full analogue of SQL’s “SELECT * from TABLE WHERE key > X and key < Y LIMIT (from, num)". In this test each record's key contained timestamp and range request asked for data in some time range.
Each key (elliptics uses 64 bytes for key) looked like this:
xxxyyy...whatever else...timestamp[16 bytes]
and range request was from
xxxyyy...whatever else...0000...0000[16 bytes]
to
xxxyyy...whatever else...ffff...ffff[16 bytes]
POHMELFS got full read/write support Elliptics HOWTO
Comments are currently closed.
are these requests only read requests? it is not very interesting to see only reading from memory (I assume that after some minutes all the data was already cached in memory). Is it possible to see the experiment with reading and writing?
well, with our volumes it will not fit memory, and even in test with random range IO requests it isn’t already, although very close to (25-30Gb of data vs 24Gb of RAM, where 4-6 Gb are used for key cache, not data)
So far our use cases do not require bulk uploads, and we (well, users) write data per-key, but it is quite simple task to implement bulk parallel writes using low-level C API and propagate it up to C++/Python and HTTP.
The charts look really nice. What do you use to create them?
Charts are created with http://www.amcharts.com
What is git link to clone elliptics project?
git clone git://…. ???
Thanks
git clone http://www.ioremap.net/git/elliptics.git
Or http://www.elliptics.ru for russian-speaking people. It also contains fair number of documentation, config, examples and so on
thanks a lot