July 21, 2014 at 15:15Traffic Inspector monitored counters: keep megabytes in black gloves
Perhaps, all experienced system administrators are well aware that many Internet providers like to take into account traffic in the “plus or minus tram stop” style (and any discrepancies, as a rule, are interpreted in their favor), so the ability to keep your own account of network activity will not only facilitate regular reconciling data with the provider, but also allows companies to avoid unnecessary costs.
Due to the specifics of the TCP / IP stack, it is almost impossible to manually separate exactly the traffic for which the provider charges from the other types of traffic - because for this you need to analyze the packet headers and also record the amount of lost and test packets and other overhead information. That is why most software solutions for accounting for network activity include various options for implementing this feature.
Traffic Inspector also has such functionality, and it is presented in the form of counters of two types - informational and monitored. Information counters are used for detailed accounting of various types of traffic and statistics for subsequent analysis. When configuring them, in addition to external networks and interfaces, you can specify the type of IP protocol, as well as TCP and UDP ports. Controlled meters are mainly intended for controlling the consumption of traffic from a higher provider as a paid resource, and for different subnets and types of traffic, individual charging and blocking rules can be set. About controlled meters further and we will talk.
Each packet received on external network interfaces is checked according to the conditions of all monitored counters, but is taken into account only on the very first counter, the conditions of which it meets. The order of counters in this list is of fundamental importance - the stricter the condition of the counter, the higher it should be located in the list. The last on this list must be a counter for all traffic (it is created by default after installing the program and is called All Internet) . Thus, each packet is registered only on one counter, and the sum for all counters characterizes the total amount of consumed traffic.
The general rule for setting up monitored counters is as follows: for each upstream provider, at least one counter must be created, and if the provider charges traffic on resources differently, then there can be several counters, for example, for traffic with full payment, for preferential traffic and for free traffic.
The general list of all external Traffic Inspector counters (monitored and informational) is located in the Traffic Accounting -> Counters section of the administrator console. For controlled meters in this section there is a special subsection.
Traffic Inspector implements the following operations with controlled meters:
• general settings of controlled meters;
• creation / modification of a controlled counter;
• setting attributes of the monitored counter;
• resetting the monitored counter;
• removal of the monitored counter.
Consider them in more detail using a specific example.
Let the provider have its own network with addresses 11.100.100.0/22 and free traffic. The provider also provides a separate tariff for some large Russian Internet resources ( national traffic ), and in all other cases it applies a standard tariff.
In order to correctly evaluate the traffic consumed by the provider, we will need three controlled counters: My provider, National traffic and All Internet.
A counter for the entire Internet is always present by default, so we can add the other two counters, and each one will use network lists. Because the list of national networksquite large, create a list for it in a separate text file, and then import it into the Traffic Inspector. If necessary, the same list can be made for the " My Provider " counter .
Another important note: since the My Provider counter is most likely to be a “subset” of the National Traffic counter, the My Provider counter needs to be raised up the list and made first.
So let's get started.
Launch the Traffic Inspector admin console, go to the Traffic accounting section -> Counters -> Controlled counters, click the right mouse button and select Add. In the wizard that opens, set the counter name and a brief comment:
Next, select the network interface, and in the next step, import the list of Internet resources from a text file. To do this, select the IP Network List item , click the Create List button and follow the system instructions.
In the next step, warning and blocking limits are set:
For example, we will limit the volume of daily incoming traffic to 500 MB, and outgoing traffic to 250 MB. Alerts will be sent to the administrator when 50 MB remain before reaching the limit on incoming or outgoing traffic. We will not indicate the total limit for blocking.
Next on the Actions tabyou can enable or disable the basic operations performed by the Traffic Inspector when limits are exceeded, namely:
• blocking external networks when the Block limit and the Daily block limit are exceeded ;
• notification of administrators by e-mail when the state of the counter changes (for this, the distribution in the Traffic Inspector sending service must be configured).
In addition, when locking or unlocking, the system can automatically run an arbitrary external program or script. This is convenient for performing various actions, for example, automatically turning on the backup Internet channel or sending informational messages. You can configure this feature on the Launch External Programs tab .
At the penultimate step, you can set the rules for maintaining network statistics for a given counter, including the recording interval, the minimum number of packets for recording, and sorting statistics.
And finally, at the last step, you can specify the frequency of recording the network data of this counter in the log:
Now we will create a counter to account for free traffic:
Here everything is similar to the previous counter, except that the IP address range will be different:
... and you do not need to set limits ( since traffic on this network is still free).
In the end, it remains to make sure that both counters are displayed in the list, with My provider should be first in order, National traffic should be second, and at the very end of the list should beEntire Internet :
If necessary, the order of the counters can be changed using the arrow buttons on the toolbar.
Another effective traffic accounting and filtering mechanism in the Traffic Inspector is the so-called wiretapping mode. In this mode, network traffic goes through an external (relative to the Traffic Inspector) gateway, and the network card of the Traffic Inspector server is in listening mode, and traffic for accounting is removed from the program driver (you must first direct traffic to this network card in any way, for example, using port mirroring managed switch). Blocking user traffic is possible when using managed switches with SNMP support or the built-in proxy server Traffic Inspector.
The tapping mode provides the following advantages:
• accurate accounting of traffic for each client;
• generation of reports for the selected period;
• caching proxy server with traffic savings of up to 30%;
• restrictions on the schedule, content, speed, etc.
• filtering banners, graphics and multimedia, as well as unwanted sites;
• real-time monitoring of work remotely through the management console.
To enable this mode, you need to open the Traffic Inspector Configurator (Administrator Console -> Settings -> Actions tab ) and in the opened wizard select Configuration settings :
Next, select the Listening Mode - external gateway :
... and specify one of the available configurations for this mode:
Please note that when changing the Traffic Inspector mode, all current network connections are usually reset, so clients will need to reconnect from their machines.
The Traffic Inspector solution offers powerful tools for traffic management and its detailed accounting, which allows companies not only to more efficiently monitor network activity of users, but also defend their point of view in the case of claims from the provider. Traffic counters can be easily created and precisely configured for certain types of traffic, tariffs and limits, and all data on the counters can be displayed in real time and recorded in a log for subsequent reporting.
Due to the specifics of the TCP / IP stack, it is almost impossible to manually separate exactly the traffic for which the provider charges from the other types of traffic - because for this you need to analyze the packet headers and also record the amount of lost and test packets and other overhead information. That is why most software solutions for accounting for network activity include various options for implementing this feature.
Traffic Inspector also has such functionality, and it is presented in the form of counters of two types - informational and monitored. Information counters are used for detailed accounting of various types of traffic and statistics for subsequent analysis. When configuring them, in addition to external networks and interfaces, you can specify the type of IP protocol, as well as TCP and UDP ports. Controlled meters are mainly intended for controlling the consumption of traffic from a higher provider as a paid resource, and for different subnets and types of traffic, individual charging and blocking rules can be set. About controlled meters further and we will talk.
Bit of theory
Each packet received on external network interfaces is checked according to the conditions of all monitored counters, but is taken into account only on the very first counter, the conditions of which it meets. The order of counters in this list is of fundamental importance - the stricter the condition of the counter, the higher it should be located in the list. The last on this list must be a counter for all traffic (it is created by default after installing the program and is called All Internet) . Thus, each packet is registered only on one counter, and the sum for all counters characterizes the total amount of consumed traffic.
The general rule for setting up monitored counters is as follows: for each upstream provider, at least one counter must be created, and if the provider charges traffic on resources differently, then there can be several counters, for example, for traffic with full payment, for preferential traffic and for free traffic.
The general list of all external Traffic Inspector counters (monitored and informational) is located in the Traffic Accounting -> Counters section of the administrator console. For controlled meters in this section there is a special subsection.
Traffic Inspector implements the following operations with controlled meters:
• general settings of controlled meters;
• creation / modification of a controlled counter;
• setting attributes of the monitored counter;
• resetting the monitored counter;
• removal of the monitored counter.
Consider them in more detail using a specific example.
Life example
Let the provider have its own network with addresses 11.100.100.0/22 and free traffic. The provider also provides a separate tariff for some large Russian Internet resources ( national traffic ), and in all other cases it applies a standard tariff.
In order to correctly evaluate the traffic consumed by the provider, we will need three controlled counters: My provider, National traffic and All Internet.
A counter for the entire Internet is always present by default, so we can add the other two counters, and each one will use network lists. Because the list of national networksquite large, create a list for it in a separate text file, and then import it into the Traffic Inspector. If necessary, the same list can be made for the " My Provider " counter .
Another important note: since the My Provider counter is most likely to be a “subset” of the National Traffic counter, the My Provider counter needs to be raised up the list and made first.
So let's get started.
Launch the Traffic Inspector admin console, go to the Traffic accounting section -> Counters -> Controlled counters, click the right mouse button and select Add. In the wizard that opens, set the counter name and a brief comment:
Next, select the network interface, and in the next step, import the list of Internet resources from a text file. To do this, select the IP Network List item , click the Create List button and follow the system instructions.
In the next step, warning and blocking limits are set:
For example, we will limit the volume of daily incoming traffic to 500 MB, and outgoing traffic to 250 MB. Alerts will be sent to the administrator when 50 MB remain before reaching the limit on incoming or outgoing traffic. We will not indicate the total limit for blocking.
Next on the Actions tabyou can enable or disable the basic operations performed by the Traffic Inspector when limits are exceeded, namely:
• blocking external networks when the Block limit and the Daily block limit are exceeded ;
• notification of administrators by e-mail when the state of the counter changes (for this, the distribution in the Traffic Inspector sending service must be configured).
In addition, when locking or unlocking, the system can automatically run an arbitrary external program or script. This is convenient for performing various actions, for example, automatically turning on the backup Internet channel or sending informational messages. You can configure this feature on the Launch External Programs tab .
At the penultimate step, you can set the rules for maintaining network statistics for a given counter, including the recording interval, the minimum number of packets for recording, and sorting statistics.
And finally, at the last step, you can specify the frequency of recording the network data of this counter in the log:
Now we will create a counter to account for free traffic:
Here everything is similar to the previous counter, except that the IP address range will be different:
... and you do not need to set limits ( since traffic on this network is still free).
In the end, it remains to make sure that both counters are displayed in the list, with My provider should be first in order, National traffic should be second, and at the very end of the list should beEntire Internet :
If necessary, the order of the counters can be changed using the arrow buttons on the toolbar.
Server in wiretap mode
Another effective traffic accounting and filtering mechanism in the Traffic Inspector is the so-called wiretapping mode. In this mode, network traffic goes through an external (relative to the Traffic Inspector) gateway, and the network card of the Traffic Inspector server is in listening mode, and traffic for accounting is removed from the program driver (you must first direct traffic to this network card in any way, for example, using port mirroring managed switch). Blocking user traffic is possible when using managed switches with SNMP support or the built-in proxy server Traffic Inspector.
The tapping mode provides the following advantages:
• accurate accounting of traffic for each client;
• generation of reports for the selected period;
• caching proxy server with traffic savings of up to 30%;
• restrictions on the schedule, content, speed, etc.
• filtering banners, graphics and multimedia, as well as unwanted sites;
• real-time monitoring of work remotely through the management console.
To enable this mode, you need to open the Traffic Inspector Configurator (Administrator Console -> Settings -> Actions tab ) and in the opened wizard select Configuration settings :
Next, select the Listening Mode - external gateway :
... and specify one of the available configurations for this mode:
Please note that when changing the Traffic Inspector mode, all current network connections are usually reset, so clients will need to reconnect from their machines.
Conclusion
The Traffic Inspector solution offers powerful tools for traffic management and its detailed accounting, which allows companies not only to more efficiently monitor network activity of users, but also defend their point of view in the case of claims from the provider. Traffic counters can be easily created and precisely configured for certain types of traffic, tariffs and limits, and all data on the counters can be displayed in real time and recorded in a log for subsequent reporting.