March 31, 2014 at 01:55Testing “fast MongoDB” - TokuMX in near real-life conditions
In the latest issue of Radio-T, we mentioned TokuMX - High-Performance MongoDB Distribution . This product sounds interesting - 9-fold data compression complete with a 20-fold increase in speed and transaction support. At the same time, the transition from mongo to tokumx is very simple and, from the point of view of an external observer, is completely transparent. Such magic seemed to me more than attractive and I decided to check how good everything is in practice.
Now I use several mongodb systems of different sizes, both in AWS and in private data centers. My usage pattern is mainly focused on a rare but massive data recording (once a day), somewhat more frequent massive read operations (a set of various analyzes) and constant reading in the front-end data supply mode. I have no special problems with the monga, except for the upcoming (calculated after 6 months) disk problem. Data from our system is never deleted and in the end will no longer fit on disks. In AWS, Monga runs on relatively expensive EBS volumes with guaranteed IOPS. I already planned a conventional solution to the problem of lack of disk space (transferring old data to a separate mong in a cheap configuration), but TokuMX caught my eye with the promise of compression 9 times, which would delay my problem for the next 4 years. In addition, the rollback of the recording in Mong is done only by the client, and it would be nice to do without it, but transfer it to the server level.
If you are wondering how the magic of TokuMX works, then welcome to their site . Here I will not talk about what it is and how to configure it, but I will share the results of surface testing. My tests do not pretend to be scientific accuracy and their main goal is to show what will happen in my real systems if we switch from Monga to current.
Transition Transparency:
Everything is fine with this. None of my tests covering working with monga (about 200) had any problems. Everything that worked with monga works with current too. Integration tests also did not reveal any problems, i.e. in my case, you can redirect client systems to TokuMX addresses and they will continue to work without noticing the substitution. I did not test the mode of hybrid operation of monga with current in one replica set, but I suspect that this will work.
Recording testing:
Tests were performed on 2 identical virtual machines with 2 processors for each, 20G of disk and 1G RAM. Host computer - MBPR i7, SSD, 16G RAM. Insert entries (trade candles) in one day, just 1.4M candles. The average recording size is 270 bytes. 3 additional indexes (one simple, 2 composite).
As you can see, there is a difference and TokuMX is really faster. Of course not the promised 20 times, but also not bad. Although there is a significantly greater processor load, this can be expected in connection with compression.
The data size + indexes in TokuMX also turned out to be smaller than those of monga, but only 1.6 times .
Reading Testing:
Reading was tested in a mode close to real use. All quick queries (by ticker) were carried out repeatedly for a random sample of 200 tickers, only 10,000 queries, the result was averaged. Interval (in time) requests were carried out for 10 random intervals and also repeatedly. The same requests were sent to mongu and current in the same order. The main goal of this test was to create activity as close to real as possible. Time is given in microseconds.
Not believing my eyes, I conducted these tests repeatedly and a similar result (with small fluctuations) was repeated stably. In all my tests, a similar difference, where the mongo overtakes tokumx by one and a half to three times, was invariably reproduced.
Having decided that it is possible that TokuMX needs more CPUs, I had an unequal battle where a virtual machine with toku with 4 processors competed with a monga in two. The result is slightly better, but still the mongo stays faster even in these unequal conditions (the average difference is 1.4 times). The only test in which the current passed the mongu (almost 2 times) is the last one - all the candles in a day.
Thinking how to test all this again, I reduced the amount of RAM for both contestants so that the data set would not fit in memory. I got these results:
In general, this result looks even worse and the difference of 12 times does not cause much optimism. However, the first two tests are noticeably closer to the monga, and in my case the overall result will probably be comparable, because I have many more such “improved” queries than much sagging ones.
During testing in a small amount of memory, I came across some strange peculiarity to the current - if the specified cache size does not fit in the available RAM, the current is simply cut off by the response and of course the client side is very perplexed by this and starts shouting that the data ended earlier than expected.
And one more thing - in all these tests there was a “out of the box” mongo. For TokuMX, I made a relief in the last test - I activated direct IO and set the cache size as recommended in their manual.
Conclusions:
My preliminary conclusion is this: what TokuMX writes to itself, namely, “20 times faster and 9 times more compact out of the box,” is not entirely true in my case. Almost all read operations were slower (sometimes much slower) in TokuMX, and the frighteningly quiet trimming of the response is not good either. For me, transaction support, 1.4-fold write acceleration (while there is no lock on the database, but only on the document) and a 1.6 times gain in data size are not worth a significant performance drawdown of all read operations.
Now I use several mongodb systems of different sizes, both in AWS and in private data centers. My usage pattern is mainly focused on a rare but massive data recording (once a day), somewhat more frequent massive read operations (a set of various analyzes) and constant reading in the front-end data supply mode. I have no special problems with the monga, except for the upcoming (calculated after 6 months) disk problem. Data from our system is never deleted and in the end will no longer fit on disks. In AWS, Monga runs on relatively expensive EBS volumes with guaranteed IOPS. I already planned a conventional solution to the problem of lack of disk space (transferring old data to a separate mong in a cheap configuration), but TokuMX caught my eye with the promise of compression 9 times, which would delay my problem for the next 4 years. In addition, the rollback of the recording in Mong is done only by the client, and it would be nice to do without it, but transfer it to the server level.
If you are wondering how the magic of TokuMX works, then welcome to their site . Here I will not talk about what it is and how to configure it, but I will share the results of surface testing. My tests do not pretend to be scientific accuracy and their main goal is to show what will happen in my real systems if we switch from Monga to current.
Transition Transparency:
Everything is fine with this. None of my tests covering working with monga (about 200) had any problems. Everything that worked with monga works with current too. Integration tests also did not reveal any problems, i.e. in my case, you can redirect client systems to TokuMX addresses and they will continue to work without noticing the substitution. I did not test the mode of hybrid operation of monga with current in one replica set, but I suspect that this will work.
Recording testing:
Tests were performed on 2 identical virtual machines with 2 processors for each, 20G of disk and 1G RAM. Host computer - MBPR i7, SSD, 16G RAM. Insert entries (trade candles) in one day, just 1.4M candles. The average recording size is 270 bytes. 3 additional indexes (one simple, 2 composite).
As you can see, there is a difference and TokuMX is really faster. Of course not the promised 20 times, but also not bad. Although there is a significantly greater processor load, this can be expected in connection with compression.
The data size + indexes in TokuMX also turned out to be smaller than those of monga, but only 1.6 times .
Reading Testing:
Reading was tested in a mode close to real use. All quick queries (by ticker) were carried out repeatedly for a random sample of 200 tickers, only 10,000 queries, the result was averaged. Interval (in time) requests were carried out for 10 random intervals and also repeatedly. The same requests were sent to mongu and current in the same order. The main goal of this test was to create activity as close to real as possible. Time is given in microseconds.
Not believing my eyes, I conducted these tests repeatedly and a similar result (with small fluctuations) was repeated stably. In all my tests, a similar difference, where the mongo overtakes tokumx by one and a half to three times, was invariably reproduced.
Having decided that it is possible that TokuMX needs more CPUs, I had an unequal battle where a virtual machine with toku with 4 processors competed with a monga in two. The result is slightly better, but still the mongo stays faster even in these unequal conditions (the average difference is 1.4 times). The only test in which the current passed the mongu (almost 2 times) is the last one - all the candles in a day.
Thinking how to test all this again, I reduced the amount of RAM for both contestants so that the data set would not fit in memory. I got these results:
In general, this result looks even worse and the difference of 12 times does not cause much optimism. However, the first two tests are noticeably closer to the monga, and in my case the overall result will probably be comparable, because I have many more such “improved” queries than much sagging ones.
During testing in a small amount of memory, I came across some strange peculiarity to the current - if the specified cache size does not fit in the available RAM, the current is simply cut off by the response and of course the client side is very perplexed by this and starts shouting that the data ended earlier than expected.
And one more thing - in all these tests there was a “out of the box” mongo. For TokuMX, I made a relief in the last test - I activated direct IO and set the cache size as recommended in their manual.
Conclusions:
My preliminary conclusion is this: what TokuMX writes to itself, namely, “20 times faster and 9 times more compact out of the box,” is not entirely true in my case. Almost all read operations were slower (sometimes much slower) in TokuMX, and the frighteningly quiet trimming of the response is not good either. For me, transaction support, 1.4-fold write acceleration (while there is no lock on the database, but only on the document) and a 1.6 times gain in data size are not worth a significant performance drawdown of all read operations.