At PushAll, we process several thousand requests per second to receive delivery statistics and open notifications and to send notification content. A regular database like MySQL cannot handle such a stream of queries and cannot respond so quickly.

Trying to transfer more and more operations to fast NoSQL stores like Redis, we want to know how to use it more efficiently and whether we will have problems with a large number of connections.
We also use forks of PHP for work and we were wondering how Redis will behave if we make several thousand connections in several streams simultaneously. We decided to share our tests with the community.

Iron

We are testing on one of the VPS PushAll:

CPU: Intel Xeon E5-1650v2 3.5 Ghz - 2 cores.
RAM: 3 Gb DDR3 1866Mhz

PHP7.
Redis 3.0.7

Test conditions

We wrote a multi-threaded PHP bot that:

• Makes forks in a cycle - 100 forks without any delays
• Each fork in its cycle, 1000 times creates a connection to Redis and increments
• The parent process waits 3 seconds and takes a value, if not wait - Redis will return an incomplete value of the injection

We also tested the option with 1000 forks and how the results will differ when using a UNIX socket and TCP.

100 forks, 1000 connections each, TCP

# time php benchmark.php 
End:100000
real 0m8.666s
user 0m0.063s
sys 0m0.073s

100 forks, 1000 connections each, UNIX socket

# time php benchmark.php 
End:100000
real 0m6.021s
user 0m0.023s
sys 0m0.067s

A TCP socket is on average 30% slower. (I remind you that here it’s no longer tested the performance of Redis itself, but how it processes the connections)

1000 forks, 1000 connections each, TCP + UNIX

Raise

TCP rates :

# time php benchmark.php 
End:903505
real 1m7.659s
user 0m0.073s
sys 0m0.753s

For 3 seconds, Redis did not have time to process them at home - this is one of the details that must be taken into account. If you read the values ​​too quickly, you can catch the moment when they are still old.

What is most interesting, when conducting the same test, but for a unix socket, we get errors:

Fatal error: Uncaught RedisException: Redis server went away in ....

That is, a unix socket, although it is faster, can handle a slightly smaller number of requests. Or, as an option, it is possible that due to the fact that it is so fast, the Redis server itself can’t cope.

We conducted similar tests for php-fpm too - there also a TCP socket gave fewer errors with a NGINX bundle than a UNIX socket. The difference in speed was not significant there.

I am enclosing a script:

connect('127.0.0.1:6379');
				$redis->connect('/run/redis/redis.sock');
				$redis->incr('pushall:benchmark');
				$redis->close();
			}
			exit;
		}
}
pcntl_wait($status); //wait
sleep(3);
$redis = new Redis();
$redis->connect('/run/redis/redis.sock');
echo 'End:'.$redis->get('pushall:benchmark')."\r\n";
$redis->setTimeout('pushall:benchmark', 1);
$redis->close();

UPD pconnect

It turns out pconnect is working in forks (weird)

Took a case of 100 TCP processes:
pconnect

# time php benchmark.php 
End:100000
real    0m4.679s
user    0m0.023s
sys     0m0.080s

connect

# time php benchmark.php 
End:100000
real    0m9.100s
user    0m0.037s
sys     0m0.103s

For comparison, UNIX socket on 100 forks:
pconnect

# time php benchmark.php 
End:100000
real    0m4.393s
user    0m0.023s
sys     0m0.073s

connect

# time php benchmark.php 
End:100000
real    0m6.002s
user    0m0.027s
sys     0m0.057s

And, interestingly, when using pconnect, the difference between TCP and UNIX socket is not that big 5-10%. At the same time, even having done everything that was suggested to me in the comments, I was not able to get unix sockets to work at 1000 forks.

UPD 2 Pconnect + 1000 forks

UNIX socket
pconnect

# time php benchmark.php 
End:1000000
real    0m35.445s
user    0m0.050s
sys     0m0.637s

connect - crashes to Fatal error: Uncaught RedisException: Redis server went away in ...

TCP
pconnect

# time php benchmark.php 
End:989596
real    0m43.711s
user    0m0.050s
sys     0m0.623s

# time php benchmark.php 
End:903505
real 1m7.659s
user 0m0.073s
sys 0m0.753s

The difference is 20%.

PS. Habr, why is there a hub MongoDB and MySQL, but Redis not?