HHVM: performance testing and compatibility with Joomla, Bitrix and Wordpress

About HHVM already wrote on Habré. In short: HHVM is a virtual machine from Facebook, which, thanks to translation and JIT compilation of code, allows you to speed up PHP several times. The developers also promise almost complete compatibility with PHP 5.4.

I decided to compare HHVM with the native interpreter on several tests, and also try to run CMS on it.

Test conditions

Testing was carried out in a virtual machine with a configuration of a processor 2 core Core i5 @ 3.3 GHz, 8 GB of RAM.
OS: Ubuntu 12.04 x86_64.
PHP: 5.3.10-1ubuntu3.7
Nginx: 1.1.19
Apache: 2.2.22
HHVM: 2.1.0-dev
In the settings, execution timeouts and memory_limit have been increased, APC has been enabled. No additional tuning was performed.

HHVM is installed from source according to the instructions .

Benchmarks

Several scripts were prepared, each of which was run 50 times: 10 times in PHP CLI, HHVM CLI, Apache + mod_php, nginx + php-fpm, and nginx + HHVM. The average execution time in seconds is presented in the table. The last two columns indicate the gain in speed when using HHVM.

	CLI		HTTP			Speed ​​increase compared
Test	Php	Hhvm	Apache + mod_php	Nginx + php_fpm	Nginx + hhvm	PHP CLI	PHP-FPM
asort	42,4	17.9	N / A 1	40.9	18.9	137.0%	116.7%
bcmath_factorial	3.4	3.0	3.4	3.4	2,8	13.9%	21.5%
curl_multi	8.6	8.4	9.7	9.0	9.7	2.3%	-7.0%
domdocument	50,2	51.0	54.3	50.1	50.1	-1.5%	0,0%
for	60.5	36.9	60.8	61.0	211.2	63.7%	-71.1%
geoip	14,4	15.6	14.7	14.9	5,0	-7.8%	198.0%
getid3	46.5	26.9	46.2	45.6	8.5	72.4%	435.5%
mb_strings	22.8	16.8	7.0	22.2	17.0	35.8%	31.0%
md5	36.8	35.7	37.0	37,2	27,2	3.2%	36.7%
preg	11.6	10.9	11.6	11.7	10.9	6.1%	7.9%
prime	4.2	5,6	4.2	4.3	2,5	-24.1%	71.8%
sort	46.3	17.8	N / A 1	44.1	17.5	160.1%	151.8%
sqlite	12,4	10.0	12.6	12,2	15,2	23.7%	-19.9%
strings	38.6	37.5	37.8	38,2	36.0	3.0%	6.3%
tcp_client	15.4	13,2	15,5	17.3	13.6	16.9%	27.7%
tcpdf	3.3	2,4	3.4	4.4	1.4	38.7%	215.1%
thumbnail	100.8	94.8	93.7	99.9	94.2	6.3%	6.1%

^{1 - Apache could not execute asort.php and sort.php - fell from out of memory}

asort - sort an associative array of strings with 10 million elements
bcmath_factorial - calculate the factorial of the number 1000 using the
bcmul curl_multi function - download 100 files of 1 MB each in 10 streams from Internet
domdocument - parsing large XML (122 thousand tags) - for example, took sitemap ag.ru
for - 3 nested for 1000 iterations (total 1 billion)
geoip - 100 thousand IP searches based on GeoIP
getid3 - getting ID3 tags from MP3 using getID3 () library - 10 thousand times
mb_strings- use mb_strlen, mb_ereg_replace, mb_substr, mb_strpos, mb_strtolower 1000 times on the first volume of War and Peace
md5 - generate md5 from numbers from 1 to 100 million
preg - search for preg_match_all hex codes for colors, tags and URLs on an HTML page - 1000 time (for example, the main Habr is taken)
prime - getting all primes from 2 to 1 million by brute force
sort - sorting an array of strings of 10 million elements with
sqlite numeric indices - 10 thousand SELECT queries to sqlite3
strings database - using strlen, str_replace, substr, strpos, strtolower 10 thousand times on the first volume of War and Peace
tcp_client - 10 thousand connections to localhost and reading 2 MB over TCP ( CHARGEN)
tcpdf - generation by multi-page PDF library TCPDF - 10 times (example_061.php taken)
thumbnail - 10 thousand creations thumbnail images using GD imagecopyresized ()

I also wanted to run the test with Smarty , but HHVM could not execute it.

Interpretation of Results

Of course, benchmarks cannot fully show the performance situation. But still, some conclusions can be drawn.

1. The execution time of each test run did not differ much from the average - so the result can be considered stable.
2. There are two abnormal results: running the mb_string test on Apache + mod_php too fast and performing for on nginx + HHVM too slow. I rechecked the tests, but I don’t know what is the matter.
3. What is in the CLI version, what when used for web HHVM gives an advantage in half the tests.
4. In some cases, using HHVM as a nginx backend (i.e. not as a CLI) gives an even greater increase - apparently due to the activation of JIT. Notice how much faster the complex getid3 and TCPDF libraries work.

CMS Testing

Wordpress

The fact that Wordpress runs on HHVM was known in advance . Only a small adjustment in one file is needed.

Simple test: loading the main page 100 times in one stream.
nginx + HHVM: 4.498 with
Apache + mod_php: 9.997 c
Apache + mod_php + APC: 6.567 c
nginx + php-fpm + APC: 5.460 s

I draw your attention to the fact that this was a test of page response time, and not load testing in several threads .

Bitrix and Joomla

Bitrix and Joomla on HHVM refused to start, giving a fatal error. I suspect that Bitrix has no hope of launching at all, because Deprecated functions and mbstring.func_overload are used there.

A review of the Internet showed that while Worpress is the easiest way to run under HHVM. With other CMS and frameworks, everything is not so rosy, for example, in a recent topic on Habr, Symfony2 did not take off .

Perhaps for those for whom performance is very important, you should immediately write a framework for HHVM?

conclusions

1) HHVM gives a good performance boost on some tasks - it is possible to use it to start background processing processes of something. Remember that HHVM is not effective when using global variables.

2) Currently, using existing libraries, frameworks and CMS with HHVM is difficult. HHVM is sharpened for performance and may not support various beautiful designs. However, in the future the situation will improve, as and HHVM may begin to support a better language and developers may be puzzled by adapting their code to HHVM.

3) On Wordpress, HHVM showed an impressive performance increase - 2 times compared to mod_php.

PS test scripts , raw data (Excel)