xdebug in the hands of the administrator
The article will talk about the possible use of xdebug by system administrators of web servers. It might seem that the administrator should not be involved in diagnosing and debugging the code, since this is the programmer's job. This is true. But, how, in the event of a problem, to convince the programmer that his code is not optimal (if this is true) and needs to be reworked if the programmer says all the time: "Everything is fine with us - repair the server"? Imagine that this is a programmer with whom it is undesirable to argue. For example, our very dear and indignant client.
When administering web-servers, engineers often encounter problems that ordinary users of the site call “slow” or “slow down”. Of course, this is a very important problem, which can have serious financial consequences for the site owner. It is with this wording “slowly works” that users and / or website owners usually contact the web server administrator, and this wording is enough to start diagnosing everything and everything.
Interestingly, the causes of such problems can be hidden almost anywhere. And there can be several of these reasons, and the problems themselves can be difficult to reproduce. What the user ultimately classifies as “brakes” at the lower level may be:
The administrator's task is to identify and, if possible, eliminate these problems. If they, of course, are in his area of responsibility. And the last and, possibly, the penultimate points are not included in his area of responsibility. But it is also necessary to show that the problem lies precisely in these points.
It is about the latter case that we will discuss in our article. Practice shows that this case accounts for approximately 90% of all problems associated with the slow operation of the site. Non-optimal code, poorly written SQL queries, improper use of locks - all this can slow down the site. And if until some point powerful hardware could digest bad code, then with the onset of some point "x", it simply ceases to cope with the load. It is possible to increase capacities further, but, firstly, they will someday run out, and secondly, this is unnecessary equipment costs. Therefore, first of all, the solution to the problem should begin with the analysis of the site code.
It is worth mentioning that there are cases when the acquisition of new hardware is economically more profitable than parsing and editing a large amount of code.
The main task of the administrator, oddly enough - server administration. He is not a developer and has no moral right to touch the code, even if it works poorly. Therefore, to conduct the most detailed diagnostics of this code and identify bottlenecks that lead to slow website operation, it is necessary to use tools that do not require code editing.
Yes, I completely forgot, it's about developing in PHP! Most web programmers use and love it. Although, at the expense of "love" the question is rather controversial.
Xdebug is a PHP extension written by one of the PHP developers and designed to collect and analyze debugging information in php code. It is important to note that this is an open-source project.
We will consider xdebug through the prism of the system administrator, affecting only the functionality that does not require editing the site code, and also does not slow down the server.
Obviously, any profiling or tracing introduces additional delays during code execution. Therefore, the diagnosis should not affect the performance of the site.
When using Centos / RHEL / Fedora, the easiest way to install xdebug is to install from the EPEL repository:
$ yum install php-pecl-xdebug
Installing with pecl:
$ pecl install xdebug
You can also install xdebug from the source code by downloading it at http : //xdebug.org/download.php :
$ tar xvzf xdebug-2.2.1.tgz
$ cd xdebug-2.2.1
$ phpize
$ ./configure --enable-xdebug
$ make && make install
The basic setup will be to simply connect the newly installed extension to the php.ini file . In this file, you need to check for the presence of the line:
zend_extension = /path/to/xdebug.so
xdebug.default_enable = 0
xdebug.overload_var_dump = 0
If the line was not added to php.ini or to one of the included files, for example / etc / php. d / xdebug.ini , you must do this manually. Then restart the web server.
From this moment, the developers of the site got at their disposal the new xdebug functions, which theoretically can be used. The first two directives we specifically disable the function of the expanded display of the stack of function calls when errors occur and the redefinition of the standard function var_dump () . Despite the fact that this is a wonderful feature that helps the development, it changes (albeit slightly) the behavior of the site code. We cannot go for it.
The most important and useful method for diagnosing a site and identifying problem areas is to trace function calls. When you enter any page of the site we have chosen, statistics will be collected on the operation of the functions, namely:
This invaluable information will help to reliably determine the "brake" section or sections of the code. In addition, it will be possible to determine the amount of memory consumed by a particular function.
Before starting the trace, we will define some variables that will help us control the collection of information and collect more necessary data in the process of code execution.
xdebug.collect_params indicates how much detail you need to collect information about the arguments of functions. 0 - minimum, information is not collected. 1 - information is collected about the number and type of arguments. 3 - information about the meaning of the arguments. 4 - full information: type, name, argument value at the time of transfer. The more information we want, the longer the tracing will take.
xdebug.show_mem_deltadetermines whether or not to reflect the difference in memory consumption in the final report when each function is called.
xdebug.trace_enable_trigger enables or disables the ability to start tracing on demand.
xdebug.auto_trace enables or disables the automatic start of the trace every time you access the pages of the site.
xdebug.collect_assignments includes or does not include variable assignment information in the report.
xdebug.collect_includes includes or does not include information about connected files in the report.
xdebug.collect_return includes or does not include in the report information about the values returned by the functions.
xdebug.trace_output_dirindicates the directory in which reports will be thrown off.
xdebug.trace_output_name generates the file name for the report.
The remaining variables are associated with the visual display of the report, and we are not interested. The standard form looks quite acceptable for analysis.
The essence of the method will be to start tracing only at the moment when the administrator wants it. Enabling tracing on an ongoing basis is unacceptable, as it leads to increased resource consumption during code execution, and also creates a lot of unnecessary reports. In order to indicate to the server that the administrator wants to start tracing functions at the time the request is executed, he needs to pass the XDEBUG_TRACE parameterin a GET or POST request, or set a cookie with that name. The latter method seems to be the most preferable, since it is not always possible to send a POST request, and it is almost never necessary. And when using GET, there may be problems associated with the fact that the address bar is often processed on the server using mod_rewrite or aliases BEFORE calling the PHP code. Therefore, our variable may simply not reach the destination.
We will collect the maximum information about the transferred variables, the connected files and the difference in the consumed memory between function calls. Reports will be added to / var / tmp . Other settings will be by default. As a result, we add the following lines in php.ini :
xdebug.trace_enable_trigger = 1
xdebug.auto_trace = 0
xdebug.collect_params = 4
xdebug.show_mem_delta = 1
xdebug.trace_output_dir = / var / tmp
And we reboot the web server. For analysis, take the following code:
We will not go into its details now, although it is easy to guess that the code is not valuable for use in cases other than test cases. Let's say that the user tried to open the page for which this code is responsible in the browser. The page has been loading for more than ten seconds, which is a very long time! Our goal is to understand why. Since the code trace has already been configured accordingly, we can only tell the server to enable it for our future request and make this very request. As mentioned earlier, for this you need to set the XDEBUG_TRACE variable in a GET or POST request, or pass it using a cookie. We chose the last option - cookie transfer.
Typically, cookie variables for a particular domain are set by the server in the client’s browser by sending the Set-Cookie HTTP header, or set already on the client side using JavaScript.
It’s rather difficult to get the header we need from the server: we don’t touch the site code, and making changes to the server settings requires a reboot, which is unacceptable for periodic (non-one-time) operations. The most correct option is to set a cookie for our domain on your (client) side. We cannot force the server to give us the javascript we need to set the cookie. All for the same reasons as with the Set-Cookie header. Therefore, we will set cookies in the browser ourselves. In Google Chrome, this can be done by driving a script directly into the address bar:
javascript: document.cookie = “XDEBUG_TRACE = 1”
The execution context of this script should correspond to our site. That is, you must first open the site page, and then enter the command.
Having set the variable, we make a request to the page and wait for the request to end. Next, on the server, we look in the / var / tmp directory for the corresponding trace file. Take a look at the results:
The first and last lines of the report show the start and end times of the request, respectively. From this information it follows that the code ran for 11 seconds. The report also displays the order of calls of all functions in the code, taking into account their nesting. The first column indicates the total code execution time in seconds at the time the function was called, the second column shows the memory consumption in bytes also at the time the function was called. The third column is the difference in memory consumption caused by the PREVIOUS function. The remaining columns show the function name, file name, and line number in which it was called.
Let's try to understand what causes the code to run slowly. First of all, the long work of the mysqli_connect () function is striking.which runs for almost 9 seconds. Obviously, there are problems accessing the remote server. Note that this function is called through several layers of abstraction and classes. For many frameworks, this is a common thing. In addition, delays occur in the sleep () function , which is called in the user- defined show_even () function .
As for memory consumption, we see a sharp jump of more than 140KB after calling the user-defined function alloc_array () , as well as a large allocation of memory at the very beginning of code execution.
We see all the necessary information about the arguments passed to all functions, so that the developer can compare these reports with the source code.
It is worth mentioning another opportunity that xdebug provides for finding bottlenecks in the code, and which the server administrator can use without making changes to the site code. This is profiling. Profiling starts in the same way as the function trace starts, except for the name of the cookie variable. Instead of XDEBUG_TRACE, XDEBUG_PROFILE is used. As a result of profiling, we get a file with data that can be recognized by the callgrind_annotate utility from vallgrind , and can also be displayed graphically using the KCacheGrind utility for KDE or WinCacheGrind for Windows. KCacheGrind among them is the richest in functionality.
On-demand profiling can be configured using directives similar to function tracing directives:
xdebug.profiler_enable = 0
xdebug.profiler_enable_trigger = 1
xdebug.profiler_output_dir = / var / tmp
When administering web-servers, engineers often encounter problems that ordinary users of the site call “slow” or “slow down”. Of course, this is a very important problem, which can have serious financial consequences for the site owner. It is with this wording “slowly works” that users and / or website owners usually contact the web server administrator, and this wording is enough to start diagnosing everything and everything.
Interestingly, the causes of such problems can be hidden almost anywhere. And there can be several of these reasons, and the problems themselves can be difficult to reproduce. What the user ultimately classifies as “brakes” at the lower level may be:
- Delays in the network access channel to the site
- Server hardware issues
- Not enough server resources
- A sub-optimal operating system
- Suboptimal software
- Problems with access to third-party resources, access to which is implemented synchronously
- Suboptimal site code
The administrator's task is to identify and, if possible, eliminate these problems. If they, of course, are in his area of responsibility. And the last and, possibly, the penultimate points are not included in his area of responsibility. But it is also necessary to show that the problem lies precisely in these points.
It is about the latter case that we will discuss in our article. Practice shows that this case accounts for approximately 90% of all problems associated with the slow operation of the site. Non-optimal code, poorly written SQL queries, improper use of locks - all this can slow down the site. And if until some point powerful hardware could digest bad code, then with the onset of some point "x", it simply ceases to cope with the load. It is possible to increase capacities further, but, firstly, they will someday run out, and secondly, this is unnecessary equipment costs. Therefore, first of all, the solution to the problem should begin with the analysis of the site code.
It is worth mentioning that there are cases when the acquisition of new hardware is economically more profitable than parsing and editing a large amount of code.
The main task of the administrator, oddly enough - server administration. He is not a developer and has no moral right to touch the code, even if it works poorly. Therefore, to conduct the most detailed diagnostics of this code and identify bottlenecks that lead to slow website operation, it is necessary to use tools that do not require code editing.
Yes, I completely forgot, it's about developing in PHP! Most web programmers use and love it. Although, at the expense of "love" the question is rather controversial.
Xdebug
Xdebug is a PHP extension written by one of the PHP developers and designed to collect and analyze debugging information in php code. It is important to note that this is an open-source project.
We will consider xdebug through the prism of the system administrator, affecting only the functionality that does not require editing the site code, and also does not slow down the server.
Obviously, any profiling or tracing introduces additional delays during code execution. Therefore, the diagnosis should not affect the performance of the site.
Xdebug installation
When using Centos / RHEL / Fedora, the easiest way to install xdebug is to install from the EPEL repository:
$ yum install php-pecl-xdebug
Installing with pecl:
$ pecl install xdebug
You can also install xdebug from the source code by downloading it at http : //xdebug.org/download.php :
$ tar xvzf xdebug-2.2.1.tgz
$ cd xdebug-2.2.1
$ phpize
$ ./configure --enable-xdebug
$ make && make install
Xdebug setup
The basic setup will be to simply connect the newly installed extension to the php.ini file . In this file, you need to check for the presence of the line:
zend_extension = /path/to/xdebug.so
xdebug.default_enable = 0
xdebug.overload_var_dump = 0
If the line was not added to php.ini or to one of the included files, for example / etc / php. d / xdebug.ini , you must do this manually. Then restart the web server.
From this moment, the developers of the site got at their disposal the new xdebug functions, which theoretically can be used. The first two directives we specifically disable the function of the expanded display of the stack of function calls when errors occur and the redefinition of the standard function var_dump () . Despite the fact that this is a wonderful feature that helps the development, it changes (albeit slightly) the behavior of the site code. We cannot go for it.
Getting started with the analysis: Trace function calls
The most important and useful method for diagnosing a site and identifying problem areas is to trace function calls. When you enter any page of the site we have chosen, statistics will be collected on the operation of the functions, namely:
- Code start and end times
- Functional Order
- Each function execution time
- Memory consumption by each function
This invaluable information will help to reliably determine the "brake" section or sections of the code. In addition, it will be possible to determine the amount of memory consumed by a particular function.
Before starting the trace, we will define some variables that will help us control the collection of information and collect more necessary data in the process of code execution.
xdebug.collect_params indicates how much detail you need to collect information about the arguments of functions. 0 - minimum, information is not collected. 1 - information is collected about the number and type of arguments. 3 - information about the meaning of the arguments. 4 - full information: type, name, argument value at the time of transfer. The more information we want, the longer the tracing will take.
xdebug.show_mem_deltadetermines whether or not to reflect the difference in memory consumption in the final report when each function is called.
xdebug.trace_enable_trigger enables or disables the ability to start tracing on demand.
xdebug.auto_trace enables or disables the automatic start of the trace every time you access the pages of the site.
xdebug.collect_assignments includes or does not include variable assignment information in the report.
xdebug.collect_includes includes or does not include information about connected files in the report.
xdebug.collect_return includes or does not include in the report information about the values returned by the functions.
xdebug.trace_output_dirindicates the directory in which reports will be thrown off.
xdebug.trace_output_name generates the file name for the report.
The remaining variables are associated with the visual display of the report, and we are not interested. The standard form looks quite acceptable for analysis.
The essence of the method will be to start tracing only at the moment when the administrator wants it. Enabling tracing on an ongoing basis is unacceptable, as it leads to increased resource consumption during code execution, and also creates a lot of unnecessary reports. In order to indicate to the server that the administrator wants to start tracing functions at the time the request is executed, he needs to pass the XDEBUG_TRACE parameterin a GET or POST request, or set a cookie with that name. The latter method seems to be the most preferable, since it is not always possible to send a POST request, and it is almost never necessary. And when using GET, there may be problems associated with the fact that the address bar is often processed on the server using mod_rewrite or aliases BEFORE calling the PHP code. Therefore, our variable may simply not reach the destination.
We will collect the maximum information about the transferred variables, the connected files and the difference in the consumed memory between function calls. Reports will be added to / var / tmp . Other settings will be by default. As a result, we add the following lines in php.ini :
xdebug.trace_enable_trigger = 1
xdebug.auto_trace = 0
xdebug.collect_params = 4
xdebug.show_mem_delta = 1
xdebug.trace_output_dir = / var / tmp
And we reboot the web server. For analysis, take the following code:
connect();
echo "finished";
function show_num($i) {
($i % 2) ? show_odd($i) : show_even($i);
}
function show_odd($i) {
echo "odd: $i
";
}
function show_even($i) {
echo "even: $i
";
sleep(1);
}
function alloc_array($size) {
global $v;
for ($i = 0; $i < $size; $i++) {
$v[] = $i;
}
}
?>
We will not go into its details now, although it is easy to guess that the code is not valuable for use in cases other than test cases. Let's say that the user tried to open the page for which this code is responsible in the browser. The page has been loading for more than ten seconds, which is a very long time! Our goal is to understand why. Since the code trace has already been configured accordingly, we can only tell the server to enable it for our future request and make this very request. As mentioned earlier, for this you need to set the XDEBUG_TRACE variable in a GET or POST request, or pass it using a cookie. We chose the last option - cookie transfer.
Typically, cookie variables for a particular domain are set by the server in the client’s browser by sending the Set-Cookie HTTP header, or set already on the client side using JavaScript.
It’s rather difficult to get the header we need from the server: we don’t touch the site code, and making changes to the server settings requires a reboot, which is unacceptable for periodic (non-one-time) operations. The most correct option is to set a cookie for our domain on your (client) side. We cannot force the server to give us the javascript we need to set the cookie. All for the same reasons as with the Set-Cookie header. Therefore, we will set cookies in the browser ourselves. In Google Chrome, this can be done by driving a script directly into the address bar:
javascript: document.cookie = “XDEBUG_TRACE = 1”
The execution context of this script should correspond to our site. That is, you must first open the site page, and then enter the command.
Having set the variable, we make a request to the page and wait for the request to end. Next, on the server, we look in the / var / tmp directory for the corresponding trace file. Take a look at the results:
TRACE START [2012-09-25 11:19:54]
0.0005 645152 +645152 -> {main} () /var/www/test.phpPoint
0.0007 649296 +4144 -> require (/var/www/config.inc) /var/www/test.php:4
0.0007 649504 +208 -> define ('HOST', '10 .1.1.1 ') /var/www/config.inchaps
0.0008 649536 +32 -> define ('NAME', 'db') /var/www/config.inc-00-00
0.0008 649568 +32 -> define ('USER', 'u0') /var/www/config.inchaps
0.0008 649600 +32 -> define ('PASS', 'ps') /var/www/config.inc:6
0.0012 695728 +46128 -> require (/var/www/class/db.php) /var/www/test.phphaps
0.0013 694736 -992 -> show_num ($ i = 1) /var/www/test.php:8
0.0013 694736 +0 -> show_odd ($ i = 1) /var/www/test.php:21
0.0013 694864 +128 -> show_num ($ i = 2) /var/www/test.php:8
0.0013 694864 +0 -> show_even ($ i = 2) /var/www/test.php:21
0.0014 694960 +96 -> sleep (1) /var/www/test.php:30
1.0033 694864 -96 -> show_num ($ i = 3) /var/www/test.php:8
1.0034 694864 +0 -> show_odd ($ i = 3) /var/www/test.php:21
1.0034 694864 +0 -> show_num ($ i = 4) /var/www/test.php:8
1.0034 694864 +0 -> show_even ($ i = 4) /var/www/test.php:21
1.0035 694960 +96 -> sleep (1) /var/www/test.php:30
2.0047 694864 -96 -> show_num ($ i = 5) /var/www/test.php:8
2.0048 694864 +0 -> show_odd ($ i = 5) /var/www/test.php:21
2.0048 695224 +360 -> alloc_array ($ size = 1024) /var/www/test.php:13
2.0057 843024 +147800 -> DB :: Get ($ type = 'mysql', $ host = '10 .1.1.1 ', $ user =' u0 ', $ pass =' ps', $ db = 'db') / var /www/test.php:15
2.0057 843664 +640 -> absDB -> __ construct ($ host = '10 .1.1.1 ', $ user =' u0 ', $ pass =' ps', $ db = 'db') / var / www / class / db. php: 10
2.0058 843664 +0 -> DB -> __ construct ($ host = '10 .1.1.1 ', $ user =' u0 ', $ pass =' ps', $ db = 'db') / var / www / class / db. php: 36
2.0058 844000 +336 -> DB-> build () /var/www/class/db.php:19
2.0058 844016 +16 -> absDB-> connect () /var/www/test.php:16
2.0058 844368 +352 -> mysqli_connect ('10 .1.1.1 ',' u0 ',' ps', 'db') /var/www/class/db.php:47
11.0164 8432
TRACE END [2012-09-25 11:20:05]The first and last lines of the report show the start and end times of the request, respectively. From this information it follows that the code ran for 11 seconds. The report also displays the order of calls of all functions in the code, taking into account their nesting. The first column indicates the total code execution time in seconds at the time the function was called, the second column shows the memory consumption in bytes also at the time the function was called. The third column is the difference in memory consumption caused by the PREVIOUS function. The remaining columns show the function name, file name, and line number in which it was called.
Let's try to understand what causes the code to run slowly. First of all, the long work of the mysqli_connect () function is striking.which runs for almost 9 seconds. Obviously, there are problems accessing the remote server. Note that this function is called through several layers of abstraction and classes. For many frameworks, this is a common thing. In addition, delays occur in the sleep () function , which is called in the user- defined show_even () function .
As for memory consumption, we see a sharp jump of more than 140KB after calling the user-defined function alloc_array () , as well as a large allocation of memory at the very beginning of code execution.
We see all the necessary information about the arguments passed to all functions, so that the developer can compare these reports with the source code.
Code Profiling
It is worth mentioning another opportunity that xdebug provides for finding bottlenecks in the code, and which the server administrator can use without making changes to the site code. This is profiling. Profiling starts in the same way as the function trace starts, except for the name of the cookie variable. Instead of XDEBUG_TRACE, XDEBUG_PROFILE is used. As a result of profiling, we get a file with data that can be recognized by the callgrind_annotate utility from vallgrind , and can also be displayed graphically using the KCacheGrind utility for KDE or WinCacheGrind for Windows. KCacheGrind among them is the richest in functionality.
On-demand profiling can be configured using directives similar to function tracing directives:
xdebug.profiler_enable = 0
xdebug.profiler_enable_trigger = 1
xdebug.profiler_output_dir = / var / tmp