Increase Redis Performance with a Simple Cluster

Starting acquaintance with the Redis nosql database, which is devoted to almost every article devoted to it, we come across the statement that this database works incredibly fast. The speed is really amazing, thanks to the storage of data in RAM.

But imagine a situation where Redis is torn from the load. This situation is not uncommon. What then to do?

Replication

The first thing that comes to mind is to add another database by creating a master-slave bundle and redirect read operations to it.

Pros:

You can add a base and configure replication quickly. Install Redis, execute several commands, wait for the data to be synchronized and use the replica.

The number of read operations increases in proportion to the number of slave bases.

Minuses:

It is necessary to modify the code - create a proxy class for distributing read / write operations to the necessary databases.

The number of write operations will remain unchanged. From slave bases we can only read. Write operations can only be performed on the master base. If Redis was torn from too many recording commands, then we won’t get any acceleration.

Cluster

The second obvious solution is to create a cluster of Redis databases.

Cluster advantages:

The number of read and write operations increases in proportion to the number of databases in the cluster.

The cluster is easily configured for a new base.

You can easily add replication for Redis databases in the cluster. Thus increasing the maximum number of read operations.

The size of the base is significantly larger. Agree that 8 servers with 32 GB of memory is much better than one.

If necessary, simply increase the size of the base. Replacing 8 servers with 32 GB of memory with 8 servers of 64 GB will immediately increase the maximum database size by 128 GB (Do not forget that for Redis to work quickly, the amount of RAM should be at least 2 times the size of the database). This is much better than replacing one server with another with a lot of RAM.

Cons of the cluster:

It is necessary to modify the code - create a proxy class for distributing read / write operations between cluster nodes.

If such a solution is implemented on an existing database, then it is necessary to migrate data to the databases included in the cluster. This is a non-trivial task and requires a lot of effort and time. Each portable base has its own logic, and therefore, for each case, you will have to create an individual solution. It is easiest to initially create a cluster from the databases and use it in your work than transfer the finished database to the cluster.

For two years now I have been using Redis as the main database on highly loaded projects. (Games for mobile devices). All this time, the creators of Redis promise that they are about to release a clustered solution for Redis. They have been waiting for the promised three years, and those who are not waiting write their decisions, including myself.

I succeeded in creating my own cluster. I hope that my experience will be useful to people seeking to increase the performance of Redis in their projects. I will say right away that I use 8 nodes by default. If the performance of 1 redis-base was not enough, then it was not possible to fully load 8 redis-bases.

Two years of cluster operation under loads in several projects have shown its viability.

Those interested can download and use the sample code from the github - github.com/dfer/myredis

PS

Two years ago, when I was thinking about how to solve the performance problem of the Redis database and create a redis cluster, the following 2 articles helped me:

oldblog.antirez.com/post/redis-presharding.html

blog.zawodny.com/2011/02 / 26 / redis-sharding-at-craigslist

I hope that my solution and the articles mentioned above may be useful to people who are faced with poor Redis performance in their projects.