ASP.NET Optimization - Practical Tips for Using IIS

In this publication, we will focus on configuring important parameters of the pool of ASP.NET applications when calling remote web services and actively working with the server-side network through standard .NET classes.



Introduction


Have you ever had to set up production web servers yourself running Windows Server 2008 R2 / IIS 7.5 and higher? For system administrators who have extensive experience working with IIS, this is most likely a trivial task, but for web developers who, for various reasons, sometimes have to participate in setting up “battle” servers themselves, this information can be very useful.

So, let's get started. We speed up the site on ASP.NET - we save money of the enterprise and nerves of the administrator.

Background
1. The configuration parameters the IIS
2. Configure an ASP.NET
3. Recommendations for optimizing the base configuration
Advanced
Conclusion
Links

Background


At the end of last year, in one large organization, we ran into web server performance problems with a sharply increased user load. At that time, more than 200,000 customers were registered in the web application. In normal mode, about 1000 users work simultaneously, about 10-15% of unique visitors from the total number of registered users per day, so the load is relatively low. However, there are peak loads at which the system is practically inoperative.

Web administrators checked everything that is possible, and could not understand what was the matter. Indeed, despite the fact that for all the main parameters of the system at the physical level, performance was good, there were failures with the availability of services, and a huge queue of requests was collected in the pool. The organization uses an NLB cluster for 4 nodes (Windows Server 2008 R2 + IIS 7.5 + .NET 4.5), there is a reserve for loading CPU and memory, network channels are large, the number of ports used is sufficient. All checks indicated that problems lie in the bowels of IIS and in the configuration of the ASP.NET pool. A living example, when administrators would not hurt the help of experienced web developers ...

1. IIS configuration options


.NET Configuration Overview
Starting with IIS 7, all ASP.NET configuration settings are stored in XML files (* .config). They replaced the metabase that was used in IIS 6.0 and earlier.

The configuration file diagram for IIS 7.x and higher looks like this:


Fig. 1. Diagram of configuration files

At the top of the hierarchical .NET configuration is the machine.config file . It defines global parameters for a specific machine. This file defines the supported sections of the configuration files, configures the ASP.NET workflow, and registers the various module providers. To optimize the initialization process, the machine.config file was greatly simplified, and it is located in the directory:

%systemroot%\Microsoft.NET\Framework\\CONFIG\

Here is the machine.config.comments file , which allows you to find out what parameters are used by default. Using this data, you can add parameters with overridden values to machine.config .

The root of the ASP.NET configuration hierarchy is the web.config file , located in the same directory as machine.config . This file includes parameters that are used for all ASP.NET applications.

ApplicationHost.config - the root configuration file for IIS, includes a description of all sites, applications, virtual directories and application pools, as well as global default settings for web server settings. It is located in the following folders depending on the OS version:
  • for 32-bit -% WINDIR% \ System32 \ inetsrv \ config \
  • for 64-bit -% WINDIR% \ SysWOW64 \ inetsrv \ config \

Each local web.config file applies configuration parameters to the directory in which it is located, as well as to all child directories. Subdirectory settings can be overridden by your own “configs”.

Before starting the configuration of IIS, pay attention to ASP.NET performance counters, evaluate the current and peak system load, and fix the available indicators. Check the logs for the error “HTTP Error 503.2 - Service Unavailable”. Try to determine if some of the requests are blocked in the queue.

If the system performance meets the needs of the customer, then it is better to leave the default settings, because they are designed for most ASP.NET applications.

When configuring IIS, two main parameters can be distinguished that affect application availability and performance.

1. The appConcurrentRequestLimit parameter- The maximum number of simultaneous requests in the application. Increasing the number of concurrent requests IIS will expand the available web server resources for servicing requests. The default value - 5000.

The most rapid change parameter appConcurrentRequestLimit possible utility appcmd.exe via the command line. This can be done both globally for all IIS sites through the ApplicationHost.config file , and for a single site (application).

cd %windir%\system32\inetsrv
appcmd.exe set config /section:system.webserver/serverRuntime /appConcurrentRequestLimit:20000

We execute the command, then open the “Configuration Editor” section for the root directory in IIS and check the new value of the set appConcurrentRequestLimit parameter . And here you can manually change this value.


Fig. 2. Setting the appConcurrentRequestLimit parameter

To set this parameter, the most commonly used formula is:
< usersCount * 1.5 >, where usersCount is the number of concurrent users.

2. The QueueLength parameter is the maximum number of requests that the Http.sys driver places in the application pool queue. When the queue is full, new requests receive the error "503.2 - Service Unavailable". The default value is 5000.

This parameter can be configured in several ways:
  • globally for .NET at the server level through machine.config , section processModel / requestQueueLimit;
  • at the IIS level through ApplicationHost.config : system.web / httpRuntime -> appRequestQueueLimit;
  • Set the queueLength parameter value for a specific pool.

As an example, we will change this parameter for the “DefaultAppPool” pool through the command line:

appcmd.exe set apppool "DefaultAppPool" /queueLength:20000

We execute the command, then open the “Application Pools” section in IIS, select the “DefaultAppPool” pool in the list, go to the “Advanced Settings” menu and check.


Fig. 3. Setting the queueLength parameter

Note: Viewing current requests in a running pool through “IIS -> Worker Processes”
In IIS Manager, select the server node in the tree, then click on the "Worker Processes" icon:


Fig. 4. The Worker Processes menu in IIS Manager.

In the list that appears, you can see the loading of all the currently running pools.


Fig. 5. Viewing working pools through Worker Processes.

When you click “View Current Request”, a table appears with a list of processed page addresses and other useful parameters. To update the list, you can press F5 on the screen. Thus, you can find the “hanging” queries:


Fig. 6. List of current requests in the pool

To view performance indicators, of course, it is better to use Performance Monitor counters, but they will not show you, like Requests Monitor, the URLs of current requests.

2. Configure ASP.NET


ASP.NET limits the number of worker threads and call termination port threads used to execute requests. If a server-side web application makes extensive use of external web service calls, standard classes from the System.NET namespace, to organize requests over the network, then low performance and deadlock conflicts can occur. At first, part of the requests can simply “hang”, the execution time will increase significantly. In the worst case, if you use the classic pool configuration mode (classic pipeline), this can generally lead to a reboot of the pool (recycle). ASP.NET deadlock detection is not performed for a pool running in integrated mode (default in IIS 7 and later).

Using application pools in integrated mode has several advantages compared to working in classic mode. It is recommended to run application pools in integrated mode.

The following figure is clearly seen how the treatment in ASP.NET queries and what parameters are the most important:


Fig. 7. The process of processing requests in ASP.NET

For the optimal operation of web applications, the default mode for automatic configuration of pool settings is enabled. In this case, the autoConfig property is " true " for the < processModel > section in the machine.config file , and other key parameters are not set at all.

Rummaging through the MSDN and the machine.config.comments file, I found a description of the basic pool configuration. There are 7 main parameters that affect how ASP.NET works with services and the network:
  • maxConnection
  • maxWorkerThreads / minWorkerThreads
  • maxIoThreads / minIoThreads
  • minFreeThreads
  • minLocalRequestFreeThreads

The maxconnection parameter determines the maximum number of simultaneous requests from one IP address. When the pool auto-configuration is turned on by default, this parameter is determined by the formula:
maxConnection = 12 * cpuNum , where cpuNum is the number of processor cores.

Thus, on a server with a 4-core processor, the maximum number of simultaneous connections to the final IP address is 48 = 12 * 4 (default).

The easiest way to get around this limitation is to specify the following directly in the code of your ASP.NET application in the Application_Start method in the global.asax file :

// maximum number of concurrent connections allowed by a ServicePoint object
System.Net.ServicePointManager.DefaultConnectionLimit = Int16.MaxValue;

It is better to configure maxconnection more flexibly through configuration files at the application domain level (web.config) or web server (applicationHost.config). The < system.net > section contains parameters that determine how the .NET Framework connects to the network.


Important: The layout for the maxconnection parameter address should be like this:

http(s)://:<Порт>

Increasing maxconnection allows you to make more simultaneous calls to remote services. This attribute does not affect local web service calls! It must be understood that it is not enough just to bypass the limit on the number of simultaneous connections to the service. Since an increase in the number of simultaneous calls leads to an increase in the use of CLR flows, which are used to create remote and handle callbacks.

ASP.NET through the maxWorkerThreads parameter sets thread limits on the w3wp.exe workflow(starting with IIS 7). Due to the fact that ASP.NET is built into IIS, ASP.NET processes generate requests on worker threads. Due to the insufficient number of threads in the CLR ThreadPool, requests will be queued and hang.

Attributes specified in the < processModel > section :
1. maxWorkerThreads parameter - indicates the maximum number of worker threads for each processor in the thread pool of the CLR environment. The default value is 20. The maximum value is 100.

2. The maxIoThreads parameter specifies the maximum number of I / O threads for each processor in the CLR thread pool. The default value is 20. The maximum value is 100.

3. The minWorkerThreads parameter- Indicates the minimum number of worker threads for each processor that can be provided immediately to service a remote request. The default value is 1.

4. The minIoThreads parameter specifies the minimum number of I / O threads for each processor that can be provided immediately for callback processing. The default value is 1.

The parameters minWorkerThreads / minIoThreads allow you to quickly deal with the suddenly increased number of simultaneous connections, when in case of inactivity the thread pool may not have enough time to reach the optimal thread level.

Attributes defined in the < httpRuntime > section :
1. The parameter minFreeThreads - determines the number of threads that can be used to work, in addition to processing incoming requests to the workflow. This parameter prevents the ASP.NET process from using threads from the pool to process a new HTTP request if the total number of threads in the pool falls below this limit. The default value is 8.

2. The parameter minLocalRequestFreeThreads - determines the minimum number of free threads that ASP.NET keeps available for new local requests. The default value is 4.

Note the parameters maxWorkerThreads , minWorkerThreads , maxIoThreads , minIoThreadsimplicitly multiplied by the number of processors, but the minFreeThreads and minLocalRequestFreeThreads parameters are not.

ASP.NET will not execute more than the following number of simultaneous requests:
( maxWorkerThreads * number of CPUs ) - minFreeThreads

Note: there is one CLR ThreadPool thread pool for the entire application pool, that is, for every w3wp.exe worker process serving the pool. For all application domains (sites) configured on one pool, a common set of threads is used. Therefore, for resource-demanding applications, it is better to use separate pools.

3. Recommendations for optimizing the basic configuration


First of all, you need to accurately determine the number of processors on the web server. Alternatively, you can see TaskManager -> tab "Performance". If the processor supports HyperThreadingTechnology (HTT) mode , then half of the cores are logical (Logical processors), not physical (Cores). For example, when HTT mode is enabled, a processor with 4 physical cores will work as 8 logical cores:


Fig. 8. The window for loading processors in the TaskManager.

You can also try using the following commands on the command line:

WMIC CPU Get DeviceID,NumberOfCores,NumberOfLogicalProcessors
or
echo %NUMBER_OF_PROCESSORS%

For example, on a server with 4 processors and the autoConfig = " true " property , ASP.NET will have the following default parameters:
maxConnection - 48; maxWorkerThreads - 80; maxIoThreads - 80, minFreeThreads - 8, minLocalRequestFreeThreads - 4.

If the web page on the backend part makes several network calls for each request, then MSDN recommends using the following configuration settings:

  1. maxWorkerThreads = 100 | minWorkerThreads = maxWorkerThreads / 2 = 50
  2. maxIoThreads = 100
  3. maxConnection = 12 * N
  4. minFreeThreads = 88 * N
  5. minLocalRequestFreeThreads = 76 * N, where N is the number of processors.

This section provides guidelines only, not rules. Moreover, the publication date of this data is quite old. For our “combat” system, we use slightly different configuration parameters. These formulas are a good starting point for starting optimization; they show well the dependence of the parameters on each other. For example, increasing the value of the maxConnection parameter several times, you can easily “estimate” the basic values ​​for the remaining parameters.

Changes to the < processModel > section are allowed only in the machine.config file due to the allowDefinition = " MachineOnly " attribute set therein when adding the processModel section .

C: \ Windows \ Microsoft.NET \ Framework64 \ v4.0.30319 \ Config \ Machine.config :


To be able to set the values ​​of the processModel section for each application individually through web.config , you must set the property allowDefinition = " Everywhere ".

I will bring configuration settings from our web server

Important: after making changes, you need to update Application pools.

Remember that you need to increase these parameters only if necessary, if you have enough CPU resources.

To analyze the performance of web servers, I recommend setting up ASP.NET counters through Performance Monitor:

  • ASP.NET Applications \ Requests / Sec
  • Web Service \ ISAPI Extension Requests / sec
  • ASP.NET \ Requests Current
  • ASP.NET \ Requests Queued
  • ASP.NET \ Requests Rejected
  • ASP.NET Applications \ Requests Executing
  • ASP.NET Applications \ Requests Timed Out
  • ASP.NET \ Request Execution Time

For a deeper analysis of the w3wp.exe process serving the IIS application pool, you can try the WinDbg debugger from the Windows Software Development Kit .

It is possible that after checking the counters you will have to make adjustments to the configuration of your system.

Additionally


For a better understanding of IIS, I recommend that you familiarize yourself with the process of processing a request from the user's browser to the final pool of the ASP.NET application in this useful article, "Fundamentals of IIS Architecture, or the Request for ASP.NET Architecture" .

If you use IIS8, it will not be superfluous to pay attention to “Full-fledged CPU load regulation (CPU Throttling)”.

Conclusion


For sites that do not make frequent network requests on the server side, the standard pool settings should suffice (processModel / autoConfig = “true”). At the same time, IIS will set a limit of 20 worker threads and 12 remote connections per core. If these values ​​are exceeded, requests will begin to queue and the performance of the web application will drop.

If your site works well and you can estimate the estimated load on the system, then you should not change anything. If you start to “freeze” while processing requests to various services, you should not immediately blame the hardware for everything! It is better to make changes to the basic configuration of ASP.NET. Keep in mind that changing the basic parameters of the application pool will certainly lead to an increase in processor load. Optimum balancing of all system parameters is the key to stable and productive operation of the equipment. As the saying goes, "forewarned means armed."

I invite everyone to share your experience in setting up and optimizing the operation of production web servers on the Windows Server platform.

References



Also popular now: