Tag Archives: cocaine

Elliptics server-side processing

Elliptics has for a while so called server-side processing engine. What does it mean, what it is used for?

Basically saying, server-side processing is a dynamic pool of workers which can execute your commands triggered by external event. Like you want to write data, but only if it matches some options. Or you want to read data, but only if provided credentials match some internal data.
It is done as simple as writing data into elliptics and emiting server-side processing event with your application and event name with uploaded key. Your event processing code can read that key, obtain data and perform needed steps. Or you can provide whole your data blob with event instead of writing it into elliptics.

Many other tasks can be triggered on your data. At the very lowest layer server-side processing is just a pool of processes (linux containers soon), which execute your code over data you provided in given event.

In the current elliptics git we switched to 0.10 Cocaine (cloud management software), which brought us new features and performance.

It is quite common when single elliptics node handles 100k IOps of read/writes from internal LRU cache, or 10+ krps from on-disk leveldb backend.
About 19+ krps is a number of executions per second our engine can deliver to single node during empty processing, so those are our overhead numbers. OpenStack on the same hardware delivers about 4 times less, not counting how it works with storage layer.

And we work on improving even those numbers.
More on our documentation wiki page

Highload++ presentation

Highload from Zbr

My highload++ conference presentation. It is quite useless without supporting speech text (you can read it in russian), but it is quite fun I think.

Presentation highlights elliptics state and features, its history and brief comparison to some other distributed storage systems. Second part covers Cocaine cloud engine and the last one shows off Grape realtime processing engine and its applications (like realtime attribute search engine) and mapreduce comparison.

There are hand drawn tits and cats of course.
Enjoy: http://www.slideshare.net/bioothod/highload-14865405