C / C ++ Web Application Using FastCGI - It's Easy
- From the sandbox
- Tutorial
Good afternoon.
In this article, I would like to talk about the FastCGI protocol and how to work with it. Despite the fact that the protocol itself and its implementation appeared back in 1996, there are simply no detailed manuals for this protocol - the developers have not written a reference to their own library. But two years ago, when I just started using this protocol, phrases like “I don’t quite understand how to use this library” were often found. It is this shortcoming that I want to fix - to write a detailed guide on the use of this protocol in a multi-threaded program and recommendations on choosing various parameters that everyone could use.
The good news is that the way the data is encoded in FastCGI and CGI is the same, only the way they are transmitted changes: if the CGI program uses the standard I / O interface, then the FastCGI program uses sockets. In other words, you just need to deal with several functions of the library for working with FastCGI, and then just take advantage of the experience of writing CGI programs, fortunately there are a lot of examples of them.
So, in this article we will consider:
- What is FastCGI and how it differs from the CGI protocol
- Why do I need FastCGI when there are already many languages for web development
- What implementations of the FastCGI protocol exist
- What are sockets
- Description of the FastCGI library functions
- Simple multithreaded FastCGI program example
- A simple example of Nginx configuration
Unfortunately, it is very difficult to write an article equally understandable for beginners and interesting to experienced old-timers, so I will try to cover all the points in as much detail as possible, and you can simply skip the sections that are not interesting to you.
You can read about FastCGI on Wikipedia.. In a nutshell, this is a CGI program running in a loop. If the regular CGI program is restarted for each new request, then the FastCGI program uses a queue of requests that are processed sequentially. Now imagine: 300-500 simultaneous requests arrived on your 4-8-core server. A regular CGI program will be launched to execute these same 300-500 times. Obviously, there are too many processes — your server physically cannot work them all out at once. So, you will get a queue of processes waiting for their quantum of processor time. Usually, the scheduler will distribute the processor evenly (so in this case the priorities of all processes are the same), which means you will have 300-500 “almost ready” responses to requests. It doesn’t sound very optimistic, is not it? In a FastCGI program, all these problems are solved by a simple request queue (that is, request multiplexing is applied).
Perhaps the main reason is that the compiled program will run faster than the interpreted one. For PHP, for example, there is a whole line of accelerators, including APC, eAccelerator, XCache, which reduce code interpretation time. But for C / C ++, all this is simply not necessary.
The second thing you should keep in mind is that dynamic typing and the garbage collector take up a lot of resources. Sometimes a lot. For example, integer arrays in PHP take up about 18 times more memory (up to 35 times depending on various PHP compilation options) than in C / C ++ for the same amount of data, so think about the overhead for relatively large data structures.
Third, a FastCGI program can store data common to different requests. For example, if PHP each time starts processing a request from scratch, then a FastCGI program can do a number of preparatory actions even before the first request arrives, for example, allocate memory, load frequently used data, etc. - Obviously, all this can increase the overall system performance.
The fourth is scalability. If mod_php assumes that the Apache web server and PHP are on the same machine, then the FastCGI application can use TCP sockets. In other words, you can have a whole cluster of several machines that communicate with you over the network. At the same time, FastCGI also supports Unix domain sockets, which allows you to efficiently run the FastCGI application and the web server on the same machine if necessary.
The fifth is security. You won’t believe it, but with default settings Apache allows you to do everything in the world. For example, if an attacker uploads a malicious exploit.php.jpg script to the site under the guise of an “innocent image” and then opens it in a browser, Apache will “honestly” execute malicious php code. Perhaps the only sufficiently reliable solution is to remove or change all potentially dangerous extensions from the names of the downloaded files, in this case php, php4, php5, phtml, etc. This technique is used, for example, in Drupal - an underscore is added to all "additional" extensions and exploit.php_.jpg is obtained. True, it should be noted that the system administrator can add any additional file extension as a php handler, so any. html can suddenly turn into a terrible security hole just because .php looked ugly, was bad for SEO or if the customer didn’t like it. So what does FastCGI give us in terms of security? Firstly, if you use the Nginx web server instead of Apache, it will simply give away static files. Point. In other words, the exploit.php.jpg file will be given “as is” without any processing on the server side, so launching a malicious script simply will not work. Secondly, the FastCGI program and the web server can work from different users, which means they will have different rights to files and folders. For example, a web server can only read downloaded files - this is enough to return static data, and the FastCGI program can only read and modify the contents of the folder with downloaded files - this is enough to load new and delete old files, but it will not have direct access to the downloaded files themselves, which means it will not be able to execute malicious code either. Third, a FastCGI program can run in a chroot other than the chroot of a web server. By itself, chroot (changing the root directory) allows you to severely limit the program’s rights, that is, increase the overall security of the system, because the program simply can’t access files outside the specified directory.
In short - I use Nginx . In general, there are quite a few servers with FastCGI support, including commercial ones, so let me consider a few alternatives.
Apache is perhaps the first thing that comes to mind, though it consumes a lot more resources than Nginx. For example, for 10,000 inactive HTTP keep-alive connections, Nginx consumes about 2.5M of memory, which is quite realistic even for a relatively weak machine, and Apache is forced to create a new thread for each new connection, so 10,000 threads are just fantastic.
Lighttpd- The main drawback of this web server is that it processes all requests in a single thread. This means that there may be problems with scalability - you simply cannot use all 4-8 cores of modern processors. And the second - if for some reason the web server flow hangs (for example, because of a long wait for a response from the hard disk), your entire server will hang. In other words, all other clients will stop receiving replies due to one slow request.
Another candidate is Cherokee . According to the developers, in some cases it is faster than Nginx and Lighttpd.
At the moment there are two implementations of the FastCGI protocol - the libfcgi.lib library from the creators of the FastCGI protocol, and Fastcgi ++ - the C ++ class library. Libfcgi has been developed since 1996 and, according to the Open Market, is very stable, moreover, it is more common, so we will use it in this article. I would like to note that the library is written in C, the built-in "wrapper" of C ++ cannot be called high-level, therefore we will use the C-interface.
I think it makes no sense to stop installing the library itself - it has a makefile in it, so there should be no problems. In addition, in popular distributions, this library is available from packages.
A general concept of sockets is available on Wikipedia . In a nutshell, sockets are a way of interprocess communication.
As we recall, in all modern operating systems, each process uses its own address space. The kernel of the operating system is responsible for direct access to RAM, and if the program accesses a non-existent (in the context of this program) memory address, the kernel will return a segmentation fault (segmentation error) and close the program. This is wonderful - now errors in one program simply cannot harm others - they are, as it were, in other dimensions. But since programs have different address spaces, there can be no data exchange from shared data or data exchange either. And if you really need to transfer data from one program to another, how then? Actually, to solve this problem, sockets were developed - two or more processes (read: programs) connect to the same socket and begin data exchange.
Depending on the type of use of the connection, sockets are different. For example, there are TCP sockets - they use a regular network to exchange data, that is, programs can run on different computers. The second most common option - Unix domain sockets (Unix domain socket) - are suitable for exchanging data only within one machine and look like a normal path in the file system, but the real hard drive is not used - all data exchange takes place in RAM. Due to the fact that you do not need to use the network stack, they work somewhat faster (by about 10%) than TCP sockets. For Windows, this socket option is called named pipe.
GNU / Linux socket examples can be found in this article.. If you have not worked with sockets yet, I would recommend that you familiarize yourself with it - this is not mandatory, but will improve your understanding of the things described here.
So, we want to create a multi-threaded FastCGI application, so let me describe some of the most important functions.
First of all, the library needs to be initialized:
Attention! This function needs to be called before any other functions of this library and only once (only once, for any number of threads).
Next we need to open the listening socket:
The path variable contains the socket connection string. Both Unix domain sockets and TCP sockets are supported, the library will do all the necessary work on preparing the parameters and calling the function itself.
Examples of connection strings for Unix domain sockets:
I think everything is clear here: you just need to pass a unique path in the form of a string, while all processes interacting with the socket should have access to it. I repeat once again: this method works only within the framework of one computer, but somewhat faster than TCP sockets.
Examples of connection strings for TCP sockets:
In this case, a TCP socket is opened on the specified port (in this case, 5000 or 9000, respectively), and requests will be accepted from any IP address. Attention!This method is potentially unsafe - if your server is connected to the Internet, then your FastCGI program will accept requests from any other computer. This means that any attacker can send your death package to your FastCGI program. Of course, there is nothing good in this - in the best case, your program may simply crash and result in a denial of service (DoS attack, if you like), in the worst, remote code execution (if this is not at all lucky), so always limit access to such ports using a firewall (firewall), and access should be granted only to those IP addresses that are actually used during the regular operation of the FastCGI program (the principle “everything is forbidden that is not explicitly allowed”).
The following example connection strings:
The method is completely similar to the previous one: a TCP socket is opened with connections from any IP address, so in this case it is also necessary to carefully configure the firewall. The only plus from this connection string is purely administrative - any programmer or system administrator reading configuration files will understand that your program accepts connections from any IP address, therefore, all other things being equal, it is better to prefer the data to the previous version.
A safer option is to explicitly specify the IP address in the connection string:
In this case, requests will be accepted only from the specified IP address (in this case, 5.5.5.5 or 127.0.0.1, respectively), for all other IP addresses this port (in this case, 5000 or 9000, respectively) will be closed. This increases the overall security of the system, so whenever possible always use this format for the connection string to TCP sockets - what if the system administrator “just forgets” to configure the firewall? I ask you to pay attention to the second example - the address of the same machine (localhost) is indicated there. This allows you to create a TCP socket on the same machine if for some reason you cannot use Unix domain sockets (for example, because chroot web servers and chroot FastCGI programs are in different folders and do not have common file paths ) Unfortunately, you cannot specify two or more different IP addresses, therefore, if you really need to accept requests from several web servers located on different computers, you will either have to fully open the port (see the previous method) and rely on your firewall settings, or use several sockets on different ports. Also, the libfcgi library does not support IPv6 addresses - back in 1996, this standard was just born, so you have to limit your appetites to regular IPv4 addresses. However, if you really need IPv6 support, it is relatively simple to add it, having patched the FCGX_OpenSocket function - the library license allows this. or use multiple sockets on different ports. Also, the libfcgi library does not support IPv6 addresses - back in 1996, this standard was just born, so you have to limit your appetites to regular IPv4 addresses. However, if you really need IPv6 support, it is relatively simple to add it, having patched the FCGX_OpenSocket function - the library license allows this. or use multiple sockets on different ports. Also, the libfcgi library does not support IPv6 addresses - back in 1996, this standard was just born, so you have to limit your appetites to regular IPv4 addresses. However, if you really need IPv6 support, it is relatively simple to add it, having patched the FCGX_OpenSocket function - the library license allows this.
Attention!Using the function of specifying an IP address when creating a socket is not sufficient protection - IP spoofing attacks are possible (spoofing the IP address of the packet sender), so setting up a firewall is still required. Usually, as a protection against IP spoofing, the firewall checks the correspondence between the IP address of the packet and the MAC address of the network card for all hosts on our local network (more precisely, for the broadcast domain with our host), and discards all packets coming from the Internet whose return address is located in the zone of private IP addresses or the local host (masks 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00 :: / 7, 127.0.0.0/8 and :: 1/128). Nevertheless, it’s still better to use this feature of the library - in the case of an incorrectly configured firewall, sending a “death packet” from a faked IP address is much more difficult than from any
The last kind of connection string is to use the host domain name:
In this case, the IP address will be obtained automatically based on the domain name of the host you specified. The restrictions are the same - the host must have one IPv4 address, otherwise an error will occur. However, given that the socket is created once at the very beginning of working with FastCGI, this method is unlikely to be very useful - it will still not work to dynamically change the IP address (more precisely, after each change of the IP address you will have to restart your FastCGI program). On the other hand, it may be useful for a relatively large network - remembering a domain name is still easier than an IP address.
The second parameter to the backlog function determines the length of the socket request queue. The special value 0 (zero) means the default queue length for this operating system.
Every time a request comes from a web server, a new connection is put in this queue while waiting for processing by our FastCGI program. If the queue is completely full, all subsequent connection requests will fail - the web server will receive a Connection refused response (the connection is refused). In principle, there is nothing wrong with that - the Nginx web server has its own request queue, and if there are no free resources, then new requests will wait for their turn to be processed already in the web server queue (at least until time out). In addition, if you have several servers running FastCGI-program, Nginx can transfer such a request to a less loaded server.
So, let's try to figure out which queue length will be optimal. In general, it is better to configure this parameter individually based on the data of stress testing, but we will try to estimate the most suitable range for this value. The first thing to know is that the maximum queue length is limited (determined by the kernel settings of the operating system, usually no more than 1024 connections). The second - the queue consumes resources, cheap, but still resources, so it is not worth doing it unreasonably long. Further, let's say our FastCGI program has 8 workflows (quite realistic for modern 4-8-core processors), and each thread needs its own connection - tasks are processed in parallel. So, ideally, we should already have 8 requests from the web server, so that immediately, without unnecessary delays, provide work for all threads. In other words, the minimum request queue size is the number of work threads of the FastCGI program. You can try to increase this value by 50% -100% in order to provide some margin for loading, since the data transfer time over the network is finite.
Now let's define the upper limit of this quantity. Here you need to know how many requests we can actually process and limit the queue of requests to this value. Imagine that you made this queue too large - so much so that your customers are simply bored with waiting for their turn and they simply leave your site without waiting for an answer. Obviously, there is nothing good in this - the web server had to send a request to open a connection, which is expensive in itself, and then also close this connection only because the FastCGI program did not have enough time to process this request. In a word, we are only wasting CPU time for nothing, and yet it is just not enough for us! But this is not the worst thing - it’s worse when the client refused to receive information from your site already the field for starting the processing of the request. It turns out that we will have to completely process the essentially unnecessary request, which, you see, will only worsen the situation. Theoretically, a situation may arise when most of the clients will not wait for an answer at 100% load of your processor. Not good.
So, let's say one request we can process in 300 milliseconds (i.e. 0.3 seconds). Further, we know that on average 50% of visitors leave the resource if the web page loads for more than 30 seconds. Obviously, 50% of those who are dissatisfied are too many, so we will limit the maximum page load time to 5 seconds. This means a fully finished web page - after applying cascading style sheets and executing JavaScripts - this stage on an average site can take up 70% of the total web page load time. So, no more than 5 * 0.3 = 1.5 seconds are left to download data over the network. Further it should be remembered that the html code, style sheets, scripts and graphics are transmitted in different files, with the html code first, and then everything else. True, after receiving the html code, the browser starts requesting the remaining resources in parallel, so that you can estimate the loading time of the html code as 50% of the total time for receiving data. So, we have no more than 1.5 * 0.5 = 0.75 seconds left to process one request. If on average one thread processes the request in 0.3 seconds, then the queue should have 0.75 / 0.3 = 2.5 requests per thread. Since we have 8 worker threads, the resulting queue size should be 2.5 * 8 = 20 requests. I would like to note the conventions of the above calculations - if there is a specific site, the values used in the calculation can be determined much more accurately, but nevertheless it provides a starting point for more optimal performance tuning. we have at our disposal no more than 1.5 * 0.5 = 0.75 seconds to process one request. If on average one thread processes the request in 0.3 seconds, then the queue should have 0.75 / 0.3 = 2.5 requests per thread. Since we have 8 worker threads, the resulting queue size should be 2.5 * 8 = 20 requests. I would like to note the conventions of the above calculations - if there is a specific site, the values used in the calculation can be determined much more accurately, but nevertheless it provides a starting point for more optimal performance tuning. we have at our disposal no more than 1.5 * 0.5 = 0.75 seconds to process one request. If on average one thread processes the request in 0.3 seconds, then the queue should have 0.75 / 0.3 = 2.5 requests per thread. Since we have 8 worker threads, the resulting queue size should be 2.5 * 8 = 20 requests. I would like to note the conventions of the above calculations - if there is a specific site, the values used in the calculation can be determined much more accurately, but nevertheless it provides a starting point for more optimal performance tuning.
So, we got a socket descriptor, after that it is necessary to allocate memory for the request structure. The description of this structure is as follows:
Attention! After receiving a new request, all previous data will be lost, so if you need long-term storage of data, use deep copying (copy the data itself, not pointers to the data).
You should know the following about this structure:
- the variables in, out, and err play the role of input, output, and error flows, respectively. The input stream contains the data of the POST request, the response of the FastCGI program (for example, the http headers and the html code of the web page) must be sent to the output stream, and the error stream will simply add a web server error log. At the same time, you can not use the error stream at all - if you really need to log errors, then perhaps it is better to use a separate file - data transmission over the network and their subsequent processing by the web server consumes additional resources.
- envp variable contains the values of environment variables set by the web server and http headers, for example: SERVER_PROTOCOL, REQUEST_METHOD, REQUEST_URI, QUERY_STRING, CONTENT_LENGTH, HTTP_USER_AGENT, HTTP_COOKIE, HTTP_REFERER and so on. These headers are defined respectively by CGI and HTTP protocol standards, examples of their use can be found in any CGI program. The data itself is stored in an array of strings, with the last element of the array containing a null pointer (NULL) to indicate the end of the array. Each row (each element of the string array) contains one variable value in the format TITLE_VARIABLE = VALUE, for example: CONTENT_LENGTH = 0 (in this case, this request has no POST data, since its length is zero). If the envp string array does not have the header you need, then it has not been passed.
Actually, we have finished this with the description of this structure - you will not need all other variables.
The memory was allocated, now you need to initialize the request structure:
The parameters of the function are as follows:
request - a pointer to the data structure that needs to be initialized
sock - the socket handle that we received after calling the FCGX_OpenSocket function. I would like to note that instead of a ready-made descriptor, you can pass 0 (zero) and get a socket with the default settings, but for us this method is completely not interesting - the socket will be opened on a random free port, which means that we will not be able to properly configure our web -server - we don’t know in advance exactly where to send the data.
flags - flags. Actually, only one flag can be passed to this function - FCGI_FAIL_ACCEPT_ON_INTR - do not call FCGX_Accept_r at break.
After that, you need to get a new request:
In it, you need to transfer the request structure already initialized at the last stage. Attention! In a multi-threaded program, you must use synchronization when calling this function.
Actually, this function does all the work on working with sockets: first, it sends a response to the web server to the previous request (if there was one), closes the previous data transfer channel and frees up all resources associated with it (including request structure variables), then it receives a new request, opens a new data transmission channel and prepares new data in the request structure for subsequent processing. In case of error receiving a new request, the function returns an error code less than zero.
Next, you will probably need to get environment variables, for this you can either independently process the request-> envp array, or use the function
where name is a string containing the name of the environment variable or http-header whose value you want to receive,
envp is an array of environment variables that are contained in the request-> envp variable.
The function returns the value of the environment variable we need as a string. Let the careful reader not be afraid of the type mismatch between char ** and FCGX_ParamArray - these types are declared synonyms (typedef char ** FCGX_ParamArray).
In addition, you will probably need to send a response to the web server. To do this, use the request-> out output stream and the function
where str is the buffer containing the data for output, without a terminating zero (that is, the buffer may contain binary data),
n is the length of the buffer in bytes,
stream is the stream we want to output data to (request-> out or request-> err )
If you use C-string standards with a terminating zero, it will be more convenient to use the function
which simply determines the length of the string with the strlen (str) function and calls the previous function. Therefore, if you know the length of the string in advance (for example, you use C ++ std :: string strings), it is better to use the previous function for efficiency reasons.
I want to note that these functions work fine with UTF-8 strings, so there should be no problems with multilingual web applications.
You can also call these functions several times during the processing of the same request, in some cases this can improve performance. For example, you need to send some large file. Instead of downloading the entire file from the hard drive, and then sending it “in one piece”, you can immediately start sending data. As a result, the client instead of the white screen of the browser will begin to receive data of interest to him, which purely psychologically will make him wait a little longer. In other words, you seem to be gaining some time to load the page. I would also like to note that most of the resources (cascading style sheets, JavaScripts, etc.) are indicated at the beginning of the web page, that is, the browser can analyze part of the html code and start loading these resources earlier - another reason to display data piecemeal.
The next thing you might need is to process the POST request. In order to get its value, you need to read the data from the request-> in stream using the function
where str is the pointer to the buffer,
n is the size of the buffer in bytes,
stream is the stream from which we read the data.
The size of the transmitted data in the POST request (in bytes) can be determined using the environment variable CONTENT_LENGTH, the value of which, as we recall, can be obtained using the FCGX_GetParam function. Attention! Creating a str buffer based on the value of the CONTENT_LENGTH variable without any restrictions is a very bad idea: any attacker can send any arbitrarily large POST request, and your server may simply run out of free RAM (you will get a DoS attack if you want). Instead, it is better to limit the size of the buffer to a reasonable amount (from a few kilobytes to several megabytes) and call the FCGX_GetStr function several times.
The last important function flashes the output and error streams (sends the client data that is still not sent, which we managed to put in the output and error streams) and closes the connection:
I would like to emphasize that this function is optional: the FCGX_Accept_r function also sends data to the client and closes the current connection before receiving a new request. The question is: then why is it needed? Imagine that you have already sent all the necessary data to the client, and now you need to perform some final operations: write statistics to the database, errors to the log file, etc. Obviously, the connection with the client is no longer necessary, but the client (in the sense of the browser) is still waiting for information from us: what if we send something else? It is obvious that we cannot call FCGX_Accept_r ahead of time - after that we will need to start processing the next request. Just in this case, you will need the FCGX_Finish_r function - it will allow you to close the current connection before receiving a new request. Yes,
On this, in fact, the description of the library functions ends and the processing of the received data begins.
I think everything will be clear in the example. The only thing is printing debugging messages and "falling asleep" of the workflow are made solely for demonstration purposes. When compiling the program, be sure to include the libfcgi and libpthread libraries (gcc compiler options: -lfcgi and -lpthread).
Actually, the simplest example of a config looks like this:
In this case, this config is enough for the correct operation of our FastCGI program. Commented lines are an example of working with Unix domain sockets, respectively, and specifying a host domain name instead of an IP address.
After compiling and running the program, and configuring Nginx, I got a proud inscription at localhost:
FastCGI Hello! (multi-threaded C, fcgiapp library)
Thanks to everyone who read to the end.
In this article, I would like to talk about the FastCGI protocol and how to work with it. Despite the fact that the protocol itself and its implementation appeared back in 1996, there are simply no detailed manuals for this protocol - the developers have not written a reference to their own library. But two years ago, when I just started using this protocol, phrases like “I don’t quite understand how to use this library” were often found. It is this shortcoming that I want to fix - to write a detailed guide on the use of this protocol in a multi-threaded program and recommendations on choosing various parameters that everyone could use.
The good news is that the way the data is encoded in FastCGI and CGI is the same, only the way they are transmitted changes: if the CGI program uses the standard I / O interface, then the FastCGI program uses sockets. In other words, you just need to deal with several functions of the library for working with FastCGI, and then just take advantage of the experience of writing CGI programs, fortunately there are a lot of examples of them.
So, in this article we will consider:
- What is FastCGI and how it differs from the CGI protocol
- Why do I need FastCGI when there are already many languages for web development
- What implementations of the FastCGI protocol exist
- What are sockets
- Description of the FastCGI library functions
- Simple multithreaded FastCGI program example
- A simple example of Nginx configuration
Unfortunately, it is very difficult to write an article equally understandable for beginners and interesting to experienced old-timers, so I will try to cover all the points in as much detail as possible, and you can simply skip the sections that are not interesting to you.
What is FastCGI?
You can read about FastCGI on Wikipedia.. In a nutshell, this is a CGI program running in a loop. If the regular CGI program is restarted for each new request, then the FastCGI program uses a queue of requests that are processed sequentially. Now imagine: 300-500 simultaneous requests arrived on your 4-8-core server. A regular CGI program will be launched to execute these same 300-500 times. Obviously, there are too many processes — your server physically cannot work them all out at once. So, you will get a queue of processes waiting for their quantum of processor time. Usually, the scheduler will distribute the processor evenly (so in this case the priorities of all processes are the same), which means you will have 300-500 “almost ready” responses to requests. It doesn’t sound very optimistic, is not it? In a FastCGI program, all these problems are solved by a simple request queue (that is, request multiplexing is applied).
Why do I need FastCGI when there is already PHP, Ruby, Python, Perl, etc.?
Perhaps the main reason is that the compiled program will run faster than the interpreted one. For PHP, for example, there is a whole line of accelerators, including APC, eAccelerator, XCache, which reduce code interpretation time. But for C / C ++, all this is simply not necessary.
The second thing you should keep in mind is that dynamic typing and the garbage collector take up a lot of resources. Sometimes a lot. For example, integer arrays in PHP take up about 18 times more memory (up to 35 times depending on various PHP compilation options) than in C / C ++ for the same amount of data, so think about the overhead for relatively large data structures.
Third, a FastCGI program can store data common to different requests. For example, if PHP each time starts processing a request from scratch, then a FastCGI program can do a number of preparatory actions even before the first request arrives, for example, allocate memory, load frequently used data, etc. - Obviously, all this can increase the overall system performance.
The fourth is scalability. If mod_php assumes that the Apache web server and PHP are on the same machine, then the FastCGI application can use TCP sockets. In other words, you can have a whole cluster of several machines that communicate with you over the network. At the same time, FastCGI also supports Unix domain sockets, which allows you to efficiently run the FastCGI application and the web server on the same machine if necessary.
The fifth is security. You won’t believe it, but with default settings Apache allows you to do everything in the world. For example, if an attacker uploads a malicious exploit.php.jpg script to the site under the guise of an “innocent image” and then opens it in a browser, Apache will “honestly” execute malicious php code. Perhaps the only sufficiently reliable solution is to remove or change all potentially dangerous extensions from the names of the downloaded files, in this case php, php4, php5, phtml, etc. This technique is used, for example, in Drupal - an underscore is added to all "additional" extensions and exploit.php_.jpg is obtained. True, it should be noted that the system administrator can add any additional file extension as a php handler, so any. html can suddenly turn into a terrible security hole just because .php looked ugly, was bad for SEO or if the customer didn’t like it. So what does FastCGI give us in terms of security? Firstly, if you use the Nginx web server instead of Apache, it will simply give away static files. Point. In other words, the exploit.php.jpg file will be given “as is” without any processing on the server side, so launching a malicious script simply will not work. Secondly, the FastCGI program and the web server can work from different users, which means they will have different rights to files and folders. For example, a web server can only read downloaded files - this is enough to return static data, and the FastCGI program can only read and modify the contents of the folder with downloaded files - this is enough to load new and delete old files, but it will not have direct access to the downloaded files themselves, which means it will not be able to execute malicious code either. Third, a FastCGI program can run in a chroot other than the chroot of a web server. By itself, chroot (changing the root directory) allows you to severely limit the program’s rights, that is, increase the overall security of the system, because the program simply can’t access files outside the specified directory.
Which web server with FastCGI support is better to choose?
In short - I use Nginx . In general, there are quite a few servers with FastCGI support, including commercial ones, so let me consider a few alternatives.
Apache is perhaps the first thing that comes to mind, though it consumes a lot more resources than Nginx. For example, for 10,000 inactive HTTP keep-alive connections, Nginx consumes about 2.5M of memory, which is quite realistic even for a relatively weak machine, and Apache is forced to create a new thread for each new connection, so 10,000 threads are just fantastic.
Lighttpd- The main drawback of this web server is that it processes all requests in a single thread. This means that there may be problems with scalability - you simply cannot use all 4-8 cores of modern processors. And the second - if for some reason the web server flow hangs (for example, because of a long wait for a response from the hard disk), your entire server will hang. In other words, all other clients will stop receiving replies due to one slow request.
Another candidate is Cherokee . According to the developers, in some cases it is faster than Nginx and Lighttpd.
What are the FastCGI protocol implementations?
At the moment there are two implementations of the FastCGI protocol - the libfcgi.lib library from the creators of the FastCGI protocol, and Fastcgi ++ - the C ++ class library. Libfcgi has been developed since 1996 and, according to the Open Market, is very stable, moreover, it is more common, so we will use it in this article. I would like to note that the library is written in C, the built-in "wrapper" of C ++ cannot be called high-level, therefore we will use the C-interface.
I think it makes no sense to stop installing the library itself - it has a makefile in it, so there should be no problems. In addition, in popular distributions, this library is available from packages.
What are sockets?
A general concept of sockets is available on Wikipedia . In a nutshell, sockets are a way of interprocess communication.
As we recall, in all modern operating systems, each process uses its own address space. The kernel of the operating system is responsible for direct access to RAM, and if the program accesses a non-existent (in the context of this program) memory address, the kernel will return a segmentation fault (segmentation error) and close the program. This is wonderful - now errors in one program simply cannot harm others - they are, as it were, in other dimensions. But since programs have different address spaces, there can be no data exchange from shared data or data exchange either. And if you really need to transfer data from one program to another, how then? Actually, to solve this problem, sockets were developed - two or more processes (read: programs) connect to the same socket and begin data exchange.
Depending on the type of use of the connection, sockets are different. For example, there are TCP sockets - they use a regular network to exchange data, that is, programs can run on different computers. The second most common option - Unix domain sockets (Unix domain socket) - are suitable for exchanging data only within one machine and look like a normal path in the file system, but the real hard drive is not used - all data exchange takes place in RAM. Due to the fact that you do not need to use the network stack, they work somewhat faster (by about 10%) than TCP sockets. For Windows, this socket option is called named pipe.
GNU / Linux socket examples can be found in this article.. If you have not worked with sockets yet, I would recommend that you familiarize yourself with it - this is not mandatory, but will improve your understanding of the things described here.
How to use libfcgi library?
So, we want to create a multi-threaded FastCGI application, so let me describe some of the most important functions.
First of all, the library needs to be initialized:
int FCGX_Init(void);Attention! This function needs to be called before any other functions of this library and only once (only once, for any number of threads).
Next we need to open the listening socket:
int FCGX_OpenSocket(const char *path, int backlog);The path variable contains the socket connection string. Both Unix domain sockets and TCP sockets are supported, the library will do all the necessary work on preparing the parameters and calling the function itself.
Examples of connection strings for Unix domain sockets:
"/tmp/fastcgi/mysocket"
"/tmp/fcgi_example.bare.sock"I think everything is clear here: you just need to pass a unique path in the form of a string, while all processes interacting with the socket should have access to it. I repeat once again: this method works only within the framework of one computer, but somewhat faster than TCP sockets.
Examples of connection strings for TCP sockets:
":5000"
":9000"In this case, a TCP socket is opened on the specified port (in this case, 5000 or 9000, respectively), and requests will be accepted from any IP address. Attention!This method is potentially unsafe - if your server is connected to the Internet, then your FastCGI program will accept requests from any other computer. This means that any attacker can send your death package to your FastCGI program. Of course, there is nothing good in this - in the best case, your program may simply crash and result in a denial of service (DoS attack, if you like), in the worst, remote code execution (if this is not at all lucky), so always limit access to such ports using a firewall (firewall), and access should be granted only to those IP addresses that are actually used during the regular operation of the FastCGI program (the principle “everything is forbidden that is not explicitly allowed”).
The following example connection strings:
"*:5000"
"*:9000"The method is completely similar to the previous one: a TCP socket is opened with connections from any IP address, so in this case it is also necessary to carefully configure the firewall. The only plus from this connection string is purely administrative - any programmer or system administrator reading configuration files will understand that your program accepts connections from any IP address, therefore, all other things being equal, it is better to prefer the data to the previous version.
A safer option is to explicitly specify the IP address in the connection string:
"5.5.5.5:5000"
"127.0.0.1:9000"In this case, requests will be accepted only from the specified IP address (in this case, 5.5.5.5 or 127.0.0.1, respectively), for all other IP addresses this port (in this case, 5000 or 9000, respectively) will be closed. This increases the overall security of the system, so whenever possible always use this format for the connection string to TCP sockets - what if the system administrator “just forgets” to configure the firewall? I ask you to pay attention to the second example - the address of the same machine (localhost) is indicated there. This allows you to create a TCP socket on the same machine if for some reason you cannot use Unix domain sockets (for example, because chroot web servers and chroot FastCGI programs are in different folders and do not have common file paths ) Unfortunately, you cannot specify two or more different IP addresses, therefore, if you really need to accept requests from several web servers located on different computers, you will either have to fully open the port (see the previous method) and rely on your firewall settings, or use several sockets on different ports. Also, the libfcgi library does not support IPv6 addresses - back in 1996, this standard was just born, so you have to limit your appetites to regular IPv4 addresses. However, if you really need IPv6 support, it is relatively simple to add it, having patched the FCGX_OpenSocket function - the library license allows this. or use multiple sockets on different ports. Also, the libfcgi library does not support IPv6 addresses - back in 1996, this standard was just born, so you have to limit your appetites to regular IPv4 addresses. However, if you really need IPv6 support, it is relatively simple to add it, having patched the FCGX_OpenSocket function - the library license allows this. or use multiple sockets on different ports. Also, the libfcgi library does not support IPv6 addresses - back in 1996, this standard was just born, so you have to limit your appetites to regular IPv4 addresses. However, if you really need IPv6 support, it is relatively simple to add it, having patched the FCGX_OpenSocket function - the library license allows this.
Attention!Using the function of specifying an IP address when creating a socket is not sufficient protection - IP spoofing attacks are possible (spoofing the IP address of the packet sender), so setting up a firewall is still required. Usually, as a protection against IP spoofing, the firewall checks the correspondence between the IP address of the packet and the MAC address of the network card for all hosts on our local network (more precisely, for the broadcast domain with our host), and discards all packets coming from the Internet whose return address is located in the zone of private IP addresses or the local host (masks 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00 :: / 7, 127.0.0.0/8 and :: 1/128). Nevertheless, it’s still better to use this feature of the library - in the case of an incorrectly configured firewall, sending a “death packet” from a faked IP address is much more difficult than from any
The last kind of connection string is to use the host domain name:
"example.com:5000"
"localhost:9000"In this case, the IP address will be obtained automatically based on the domain name of the host you specified. The restrictions are the same - the host must have one IPv4 address, otherwise an error will occur. However, given that the socket is created once at the very beginning of working with FastCGI, this method is unlikely to be very useful - it will still not work to dynamically change the IP address (more precisely, after each change of the IP address you will have to restart your FastCGI program). On the other hand, it may be useful for a relatively large network - remembering a domain name is still easier than an IP address.
The second parameter to the backlog function determines the length of the socket request queue. The special value 0 (zero) means the default queue length for this operating system.
Every time a request comes from a web server, a new connection is put in this queue while waiting for processing by our FastCGI program. If the queue is completely full, all subsequent connection requests will fail - the web server will receive a Connection refused response (the connection is refused). In principle, there is nothing wrong with that - the Nginx web server has its own request queue, and if there are no free resources, then new requests will wait for their turn to be processed already in the web server queue (at least until time out). In addition, if you have several servers running FastCGI-program, Nginx can transfer such a request to a less loaded server.
So, let's try to figure out which queue length will be optimal. In general, it is better to configure this parameter individually based on the data of stress testing, but we will try to estimate the most suitable range for this value. The first thing to know is that the maximum queue length is limited (determined by the kernel settings of the operating system, usually no more than 1024 connections). The second - the queue consumes resources, cheap, but still resources, so it is not worth doing it unreasonably long. Further, let's say our FastCGI program has 8 workflows (quite realistic for modern 4-8-core processors), and each thread needs its own connection - tasks are processed in parallel. So, ideally, we should already have 8 requests from the web server, so that immediately, without unnecessary delays, provide work for all threads. In other words, the minimum request queue size is the number of work threads of the FastCGI program. You can try to increase this value by 50% -100% in order to provide some margin for loading, since the data transfer time over the network is finite.
Now let's define the upper limit of this quantity. Here you need to know how many requests we can actually process and limit the queue of requests to this value. Imagine that you made this queue too large - so much so that your customers are simply bored with waiting for their turn and they simply leave your site without waiting for an answer. Obviously, there is nothing good in this - the web server had to send a request to open a connection, which is expensive in itself, and then also close this connection only because the FastCGI program did not have enough time to process this request. In a word, we are only wasting CPU time for nothing, and yet it is just not enough for us! But this is not the worst thing - it’s worse when the client refused to receive information from your site already the field for starting the processing of the request. It turns out that we will have to completely process the essentially unnecessary request, which, you see, will only worsen the situation. Theoretically, a situation may arise when most of the clients will not wait for an answer at 100% load of your processor. Not good.
So, let's say one request we can process in 300 milliseconds (i.e. 0.3 seconds). Further, we know that on average 50% of visitors leave the resource if the web page loads for more than 30 seconds. Obviously, 50% of those who are dissatisfied are too many, so we will limit the maximum page load time to 5 seconds. This means a fully finished web page - after applying cascading style sheets and executing JavaScripts - this stage on an average site can take up 70% of the total web page load time. So, no more than 5 * 0.3 = 1.5 seconds are left to download data over the network. Further it should be remembered that the html code, style sheets, scripts and graphics are transmitted in different files, with the html code first, and then everything else. True, after receiving the html code, the browser starts requesting the remaining resources in parallel, so that you can estimate the loading time of the html code as 50% of the total time for receiving data. So, we have no more than 1.5 * 0.5 = 0.75 seconds left to process one request. If on average one thread processes the request in 0.3 seconds, then the queue should have 0.75 / 0.3 = 2.5 requests per thread. Since we have 8 worker threads, the resulting queue size should be 2.5 * 8 = 20 requests. I would like to note the conventions of the above calculations - if there is a specific site, the values used in the calculation can be determined much more accurately, but nevertheless it provides a starting point for more optimal performance tuning. we have at our disposal no more than 1.5 * 0.5 = 0.75 seconds to process one request. If on average one thread processes the request in 0.3 seconds, then the queue should have 0.75 / 0.3 = 2.5 requests per thread. Since we have 8 worker threads, the resulting queue size should be 2.5 * 8 = 20 requests. I would like to note the conventions of the above calculations - if there is a specific site, the values used in the calculation can be determined much more accurately, but nevertheless it provides a starting point for more optimal performance tuning. we have at our disposal no more than 1.5 * 0.5 = 0.75 seconds to process one request. If on average one thread processes the request in 0.3 seconds, then the queue should have 0.75 / 0.3 = 2.5 requests per thread. Since we have 8 worker threads, the resulting queue size should be 2.5 * 8 = 20 requests. I would like to note the conventions of the above calculations - if there is a specific site, the values used in the calculation can be determined much more accurately, but nevertheless it provides a starting point for more optimal performance tuning.
So, we got a socket descriptor, after that it is necessary to allocate memory for the request structure. The description of this structure is as follows:
typedef struct FCGX_Request {
int requestId;
int role;
FCGX_Stream *in;
FCGX_Stream *out;
FCGX_Stream *err;
char **envp;
struct Params *paramsPtr;
int ipcFd;
int isBeginProcessed;
int keepConnection;
int appStatus;
int nWriters;
int flags;
int listen_sock;
int detached;
} FCGX_Request;Attention! After receiving a new request, all previous data will be lost, so if you need long-term storage of data, use deep copying (copy the data itself, not pointers to the data).
You should know the following about this structure:
- the variables in, out, and err play the role of input, output, and error flows, respectively. The input stream contains the data of the POST request, the response of the FastCGI program (for example, the http headers and the html code of the web page) must be sent to the output stream, and the error stream will simply add a web server error log. At the same time, you can not use the error stream at all - if you really need to log errors, then perhaps it is better to use a separate file - data transmission over the network and their subsequent processing by the web server consumes additional resources.
- envp variable contains the values of environment variables set by the web server and http headers, for example: SERVER_PROTOCOL, REQUEST_METHOD, REQUEST_URI, QUERY_STRING, CONTENT_LENGTH, HTTP_USER_AGENT, HTTP_COOKIE, HTTP_REFERER and so on. These headers are defined respectively by CGI and HTTP protocol standards, examples of their use can be found in any CGI program. The data itself is stored in an array of strings, with the last element of the array containing a null pointer (NULL) to indicate the end of the array. Each row (each element of the string array) contains one variable value in the format TITLE_VARIABLE = VALUE, for example: CONTENT_LENGTH = 0 (in this case, this request has no POST data, since its length is zero). If the envp string array does not have the header you need, then it has not been passed.
Actually, we have finished this with the description of this structure - you will not need all other variables.
The memory was allocated, now you need to initialize the request structure:
int FCGX_InitRequest(FCGX_Request *request, int sock, int flags);The parameters of the function are as follows:
request - a pointer to the data structure that needs to be initialized
sock - the socket handle that we received after calling the FCGX_OpenSocket function. I would like to note that instead of a ready-made descriptor, you can pass 0 (zero) and get a socket with the default settings, but for us this method is completely not interesting - the socket will be opened on a random free port, which means that we will not be able to properly configure our web -server - we don’t know in advance exactly where to send the data.
flags - flags. Actually, only one flag can be passed to this function - FCGI_FAIL_ACCEPT_ON_INTR - do not call FCGX_Accept_r at break.
After that, you need to get a new request:
int FCGX_Accept_r(FCGX_Request *request);In it, you need to transfer the request structure already initialized at the last stage. Attention! In a multi-threaded program, you must use synchronization when calling this function.
Actually, this function does all the work on working with sockets: first, it sends a response to the web server to the previous request (if there was one), closes the previous data transfer channel and frees up all resources associated with it (including request structure variables), then it receives a new request, opens a new data transmission channel and prepares new data in the request structure for subsequent processing. In case of error receiving a new request, the function returns an error code less than zero.
Next, you will probably need to get environment variables, for this you can either independently process the request-> envp array, or use the function
char *FCGX_GetParam(const char *name, FCGX_ParamArray envp);where name is a string containing the name of the environment variable or http-header whose value you want to receive,
envp is an array of environment variables that are contained in the request-> envp variable.
The function returns the value of the environment variable we need as a string. Let the careful reader not be afraid of the type mismatch between char ** and FCGX_ParamArray - these types are declared synonyms (typedef char ** FCGX_ParamArray).
In addition, you will probably need to send a response to the web server. To do this, use the request-> out output stream and the function
int FCGX_PutStr(const char *str, int n, FCGX_Stream *stream);where str is the buffer containing the data for output, without a terminating zero (that is, the buffer may contain binary data),
n is the length of the buffer in bytes,
stream is the stream we want to output data to (request-> out or request-> err )
If you use C-string standards with a terminating zero, it will be more convenient to use the function
int FCGX_PutS(const char *str, FCGX_Stream *stream);which simply determines the length of the string with the strlen (str) function and calls the previous function. Therefore, if you know the length of the string in advance (for example, you use C ++ std :: string strings), it is better to use the previous function for efficiency reasons.
I want to note that these functions work fine with UTF-8 strings, so there should be no problems with multilingual web applications.
You can also call these functions several times during the processing of the same request, in some cases this can improve performance. For example, you need to send some large file. Instead of downloading the entire file from the hard drive, and then sending it “in one piece”, you can immediately start sending data. As a result, the client instead of the white screen of the browser will begin to receive data of interest to him, which purely psychologically will make him wait a little longer. In other words, you seem to be gaining some time to load the page. I would also like to note that most of the resources (cascading style sheets, JavaScripts, etc.) are indicated at the beginning of the web page, that is, the browser can analyze part of the html code and start loading these resources earlier - another reason to display data piecemeal.
The next thing you might need is to process the POST request. In order to get its value, you need to read the data from the request-> in stream using the function
int FCGX_GetStr(char * str, int n, FCGX_Stream *stream);where str is the pointer to the buffer,
n is the size of the buffer in bytes,
stream is the stream from which we read the data.
The size of the transmitted data in the POST request (in bytes) can be determined using the environment variable CONTENT_LENGTH, the value of which, as we recall, can be obtained using the FCGX_GetParam function. Attention! Creating a str buffer based on the value of the CONTENT_LENGTH variable without any restrictions is a very bad idea: any attacker can send any arbitrarily large POST request, and your server may simply run out of free RAM (you will get a DoS attack if you want). Instead, it is better to limit the size of the buffer to a reasonable amount (from a few kilobytes to several megabytes) and call the FCGX_GetStr function several times.
The last important function flashes the output and error streams (sends the client data that is still not sent, which we managed to put in the output and error streams) and closes the connection:
void FCGX_Finish_r(FCGX_Request *request);I would like to emphasize that this function is optional: the FCGX_Accept_r function also sends data to the client and closes the current connection before receiving a new request. The question is: then why is it needed? Imagine that you have already sent all the necessary data to the client, and now you need to perform some final operations: write statistics to the database, errors to the log file, etc. Obviously, the connection with the client is no longer necessary, but the client (in the sense of the browser) is still waiting for information from us: what if we send something else? It is obvious that we cannot call FCGX_Accept_r ahead of time - after that we will need to start processing the next request. Just in this case, you will need the FCGX_Finish_r function - it will allow you to close the current connection before receiving a new request. Yes,
On this, in fact, the description of the library functions ends and the processing of the received data begins.
A simple example of a multi-threaded FastCGI program
I think everything will be clear in the example. The only thing is printing debugging messages and "falling asleep" of the workflow are made solely for demonstration purposes. When compiling the program, be sure to include the libfcgi and libpthread libraries (gcc compiler options: -lfcgi and -lpthread).
#include
#include
#include
#include "fcgi_config.h"
#include "fcgiapp.h"
#define THREAD_COUNT 8
#define SOCKET_PATH "127.0.0.1:9000"
//хранит дескриптор открытого сокета
static int socketId;
static void *doit(void *a)
{
int rc, i;
FCGX_Request request;
char *server_name;
if(FCGX_InitRequest(&request, socketId, 0) != 0)
{
//ошибка при инициализации структуры запроса
printf("Can not init request\n");
return NULL;
}
printf("Request is inited\n");
for(;;)
{
static pthread_mutex_t accept_mutex = PTHREAD_MUTEX_INITIALIZER;
//попробовать получить новый запрос
printf("Try to accept new request\n");
pthread_mutex_lock(&accept_mutex);
rc = FCGX_Accept_r(&request);
pthread_mutex_unlock(&accept_mutex);
if(rc < 0)
{
//ошибка при получении запроса
printf("Can not accept new request\n");
break;
}
printf("request is accepted\n");
//получить значение переменной
server_name = FCGX_GetParam("SERVER_NAME", request.envp);
//вывести все HTTP-заголовки (каждый заголовок с новой строки)
FCGX_PutS("Content-type: text/html\r\n", request.out);
//между заголовками и телом ответа нужно вывести пустую строку
FCGX_PutS("\r\n", request.out);
//вывести тело ответа (например - html-код веб-страницы)
FCGX_PutS("\r\n", request.out);
FCGX_PutS("\r\n", request.out);
FCGX_PutS("FastCGI Hello! (multi-threaded C, fcgiapp library) \r\n", request.out);
FCGX_PutS("\r\n", request.out);
FCGX_PutS("\r\n", request.out);
FCGX_PutS("FastCGI Hello! (multi-threaded C, fcgiapp library)
\r\n", request.out);
FCGX_PutS("Request accepted from host ", request.out);
FCGX_PutS(server_name ? server_name : "?", request.out);
FCGX_PutS("
\r\n", request.out);
FCGX_PutS("\r\n", request.out);
FCGX_PutS("\r\n", request.out);
//"заснуть" - имитация многопоточной среды
sleep(2);
//закрыть текущее соединение
FCGX_Finish_r(&request);
//завершающие действия - запись статистики, логгирование ошибок и т.п.
}
return NULL;
}
int main(void)
{
int i;
pthread_t id[THREAD_COUNT];
//инициализация библилиотеки
FCGX_Init();
printf("Lib is inited\n");
//открываем новый сокет
socketId = FCGX_OpenSocket(SOCKET_PATH, 20);
if(socketId < 0)
{
//ошибка при открытии сокета
return 1;
}
printf("Socket is opened\n");
//создаём рабочие потоки
for(i = 0; i < THREAD_COUNT; i++)
{
pthread_create(&id[i], NULL, doit, NULL);
}
//ждем завершения рабочих потоков
for(i = 0; i < THREAD_COUNT; i++)
{
pthread_join(id[i], NULL);
}
return 0;
} Simple Nginx Configuration Example
Actually, the simplest example of a config looks like this:
server {
server_name localhost;
location / {
fastcgi_pass 127.0.0.1:9000;
#fastcgi_pass unix: / tmp / fastcgi / mysocket;
#fastcgi_pass localhost: 9000;
include fastcgi_params;
}
}In this case, this config is enough for the correct operation of our FastCGI program. Commented lines are an example of working with Unix domain sockets, respectively, and specifying a host domain name instead of an IP address.
After compiling and running the program, and configuring Nginx, I got a proud inscription at localhost:
FastCGI Hello! (multi-threaded C, fcgiapp library)
Thanks to everyone who read to the end.