How to optimize the work with MongoDB using an outdated api or what its specification is silent about ...

    image

    Once I ran into a problem: mongoDb was used as a cache / buffer between the backend in Java and the frontend in node.js. Everything was fine until a business demand appeared to transfer large volumes in a short time through mongoDb (up to 200 thousand records in no more than a couple of minutes). For what it is not so important, it is important that such a task has appeared. And here I already had to understand the monga internals ...


    Round 0: We
    just write to monga with Write Concern = Acknowledged. The most commonplace and easiest way in the forehead. In this case, mongo guarantees that everything was recorded without errors and in general everything will be fine. Everything is perfectly written, but ... at 200 thousand it dies for twenty or more minutes. Not suitable. Cross out the path to the forehead.

    image
    Round 1:
    Try Bulk write operations with the sameWrite Concern = Acknowledged. It got better, but not much. He writes in ten to fifteen minutes. Strange, actually, more acceleration was expected. Okay, let's move on.

    image
    Round 2:
    We try to change Write Concern to Unacknowledged and use Bulk write operations to heap. In general, this is not the best solution, since if something goes wrong in the mong, we will never know about it, since she will only report the data to her, but whether or not they are written to the database is unknown. On the other hand, according to business requirements, the data is not bank transactions, a single loss is not so critical, and if everything is bad in the mong, we already learn from monitoring. We try. On the one hand, recording in just a minute is good (without Bulk write operations, a minute and a half is also good), on the other hand, a problem arose: immediately after writing, java gives node.js the go-ahead and when it starts reading, the whole data comes in, it doesn't come at all then half is read, half is not. Asynchrony is to blame - with this Write Concern, the monga is still writing, and node.js is already reading, so the client has time to read earlier, than the record is guaranteed to end. Poorly.


    Round 3:
    We started thinking, the idea of ​​writing Thread.sleep (60 seconds) or writing some control object to mongu, which showed that all the data was loaded, looks very crooked. We decided to see why Bulk write operations speed up so badly, because in theory Write Concern should slow down the last recording during Bulk write operations, and not at all. It is somehow illogical that waiting for the recording of the last portion takes so much time. We look at the monga driver code in Java, we come across packages of bulk operations that are limited by a certain parameter maxBatchWriteSize. Debug shows that we have only 500 this parameter, that is, in fact, the entire bulk is cut by requests with only 500 records, therefore, such results, Acknowledged each time, waits for a complete record of these 500 records before sending a new request, and so four thousand times maximum volume


    Round 4
    Trying to understand where this maxBatchWriteSize parameter comes from, we find that the monga driver makes a request to getMaxWriteBatchSize () to the monga server. There was an idea to increase this parameter in a config of a monga and to bypass this restriction. Attempts to find this parameter or query in the specification yielded zero result. Okay, look in the internet, find the source code in C ++. This parameter is a banal constant wired tightly in the source code, that is, it is not possible to increase it in any way. Dead end.


    Round 5
    We are looking for more options on the internet. They decided not to try the option of uploading through a hundred parallel threads, it’s commonplace to DDosit your own server with a monga (especially since the monga itself can parallel incoming requests). And then they found a team like getLastError, the essence of it is to wait until all operations are stored in the database and return an error code or a successful completion. The specification reinforced tries to convince that the method is outdated and does not need to be used, in the monga driver it is marked as depricated. But we try to send requests with Write Concern = Unacknowledged and Bulk write in ordered mode, and then we call getLastError ()and yes, in one and a half minutes we wrote all the records synchronously, now the client starts reading after the complete recording of all objects, since getLastError () waits for the last record to finish, while the packets do not inhibit each other. In addition, if an error occurs, we will find out with the help of getLastError (). That is, we got exactly the fast Bulk write with Acknowledged, but waiting only for the last packet (well, or almost, error handling will probably be worse than the current Acknowledged mode, this command will probably not show an error that occurred only in the first packets, on the other hand, the probability is that the first packet will fail, and the last will succeed - not so great).

    image
    So what is the Mongi specification silent about:
    1. Bulk writethe operation is not very bulk and is tightly limited by the ceiling of 500-1000 requests in the package. Update : in fact, as I just discovered, a mention of a ceiling of 1000 operations appeared , there was no mention of a magic constant in version 2.4 more than a year ago , when the analysis was carried out,

    2. Alas, the mechanism with getLastError was in something more successful and the new Write Concern mechanism has not yet completely replaced it, or rather, you can use the outdated command to speed up the work, so the logical behavior “wait for the successful recording of only the last packet from a large ordered bulk request” is not implemented in the mong,

    3. The problem of Write Concern = Unacknowledged is not that the data may be lost and the error is not returned, but that the data is written asynchronously and an attempt by the client to immediately access the data can easily lead to the fact that he will not receive the data or will receive only part of it (it is important if you give a read command immediately after writing).

    4. At monga, query performance suffers greatly from such a limited bulk, and Acknowledged Write Concern is not implemented correctly, it is correct to wait until the end of the recording of the last packet.

    PS In general, an interesting optimization experience was obtained using non-standard methods, when there is not all the information in the specs.

    PPSI also advise you to see my opensource project [useful-java-links] (https://github.com/Vedenin/useful-java-links/tree/master/link-rus) - perhaps the most complete collection of useful Java libraries, frameworks and Russian-language instructional video. There is also a similar [English version] (https://github.com/Vedenin/useful-java-links/) of this project and I start the opensource subproject [Hello world] (https://github.com/Vedenin/useful-java -links / tree / master / helloworlds) to prepare a collection of simple examples for different Java libraries in one maven project (I will be grateful for any help).

    Also popular now: