We follow the collection. Tailable cursors

If you are young and energetic interested in mongoDb, you want to debate on the topic of translating tailable into Russian, and you are interested in where the picrelated comes from - welcome to cat.



Some time ago, making the admin panel for one small project, I thought it would be nice to display a log of some significant events in it. Memory helpfully reminded me of one of the features of MongoDb that I had heard about, but had never put into practice - tailable cursors.

What kind of beast?

By default, mongo automatically closes the connection after the client receives all the data available to the cursor. For limited (capped) collections, additional behavior is possible, the so-called tail cursors.

Tail cursors remain open and continue to receive new data that meets the search criteria. Their action is reminiscent of the familiar tail -f. Mongo uses tail cursors to track opplogs.

What you need, I thought

new Thread(new Runnable() {
    @Override
   public void run() {
      DBCollection collection=MongoConnector.getInstance().getCollection("events");
      //Магия в addOption
      DBCursor cur= collection.find().addOption(Bytes.QUERYOPTION_TAILABLE).addOption(Bytes.QUERYOPTION_AWAITDATA);
      while (cur.hasNext()) {
         //Посылаем сообщение всем браузерам, с которыми установлено соединение, там его обработает клиентский JS
         wsConnection.broadcast((String) cur.next().get("message"));
      }
   }
}).start();

By the way, as practice has shown, the delay between inserting a new document and receiving it by the client using the tail cursor is so insignificant that it can be ignored in principle.

A spoon of tar

Official documentation suggests that using tail cursors on collections with a lot of write operations can be very costly. If the query is built on fields for which there is an index, in the case of a large number of records, it is better to use a regular cursor, periodically requesting new documents

How it works?

Remember the above, I wrote the work of the tail cursors looks like the work of tail -f? In fact, there are even more similarities than it seems at first glance.
When you set the tail cursor and read data from it, reaching the last document, the mongo leaves the cursor on it and watches the collection descriptor; if a new element appears that satisfies the request, the cursor moves to it.
For ordinary (unlimited) collections, this behavior is impossible, since the order in which documents are stored on disk in them does not always correspond to the order of data insertion.

4 facts about tail cursors that shocked the world

1) Tail cursors do not use indexes and return documents in the order in which they are stored on disk. For limited collections, the placement order always matches the insertion order of documents.

2) Since indexes are not used, at the beginning of the query you will have to read all the entries in the collection - which, with large amounts of data, can be quite expensive. After we have already received the available data, screening of newly added documents that do not satisfy the request is not unprofitable.

3) The tail cursor may become invalid (there will be no tracking of the collection) if
no documents satisfy the initial search query.
The last document was deleted at the moment when the cursor was pointing at it

4) The dead cursor has id 0