November 1, 2016 at 09:58Deep Packet Inspection: Equipment and Applications
We at VAS Experts design and implement services in the field of control and analysis of Internet traffic. In our blog, we decided to start talking about the device technologies related to our field of activity.
In the first two materials, we talked about the composition of the system and the connection diagram, and in the second about usage scenarios . Today we will see what requirements are imposed on the hardware for the full-fledged operation of deep traffic analysis systems, and where such systems are used. / Flickr / Sean MacEntee / CC
Deep packet inspection (DPI) systems are most often used to control and filter traffic, sometimes to block protocols. Using DPI, you can track the data generated by applications and choose the appropriate action strategy.
For DPI to work, you need a software system installed on a suitable hardware platform. Only the right equipment and optimized software in conjunction will give a high level of performance.
DPI servers are usually similar to 1U servers, but they focus on network components, not on RAM and hard drives. DPI servers can have 4 to 8 1 GbE RJ45 ports, 4 10 GbE SFP + ports, or 2 40 GE QSFP + ports.
Network cards in the DPI system must support Bypass mode - if the server shuts down, the connection between the ports will continue to work using power from the built-in battery and will pass traffic without filtering. Also, the DPI-server usually has a system for monitoring the status of work (Advanced Lights Out Management), with which you can manage all the parameters remotely or through a graphical interface on the display. The BIOS of the server motherboard must be protected from damage by hardware and support remote update.
For the server to function, one or two Intel Xeon E5-2600 V4 processors (Broadwell-EP) and two hard drives combined in RAID 1 are sufficient. Typically, the main device has two SSD disks for installing the operating system, for example SmartOS, they are added to 24 HDD or SSD drives, as well as network ports and expansion shelf ports. To increase the data storage, in addition to the device, JBOD disk arrays are connected, each of which supports up to 70 HDDs or SSDs.
This model allows you to quickly and cost-effectively expand the volume for data storage, which is especially important when processing traffic statistics and caching content - images, videos and other similar files.
It is best to use the ZFS file system and RAID-Z technology to control disks, maintain their integrity and high speed. To ensure fault tolerance, it is advisable to install at least two interchangeable power supplies.
Often, manufacturers of deep traffic analysis systems offer ready-made sets of hardware and software - such servers differ from standard systems and can be additionally equipped with data storage or statistics collection systems in accordance with legal requirements.
On the other hand, it can be much more convenient if the proposed server is a device on a standard platform that can be easily upgraded. Such systems are more common among Russian companies - Proteus, Vas Experts, Peter-Service, Napa Labs.
Heavy Reading conducted a survey among telecom companies and collected data on the main areas of application of DPI. The most popular area was quality of service (QoS) - this includes monitoring the status of the network and solving problems with equipment.
Previously, home subscriber traffic (HSI - High Speed Internet) was almost not controlled - BitTorrent could take the entire free bandwidth, but now DPI allows operators to distribute the channel between different applications.
The second most important category for operators was network subscriber policy management (PCEF). According to research by analysts, this is the largest category of DPI applications in terms of volume and value, and it will only continue to grow. The third category of providers was called network gateways - today DPI is often used on routers in 4G (P-GW) and 3G (GGSN) networks.
The fourth category in the survey is the use of DPI to analyze user information. Initially, DPI was used to analyze network traffic and trends, but more and more often companies are connecting this technology to analyze subscriber behavior in real time in order to develop more suitable sets of services and at the same time select the right network load.
According to experts from the University of California at Berkeley, for fast and reliable DPI operation, equipment must have certain properties. DPI is used to check network packets by thousands of identifiers, therefore, for high speed operation, it is necessary to divide this process into parallel flows, that is, the equipment must support parallelism in data processing.
Iron must process network packets at high speed to match the bandwidth of the gigabit network channel. In addition, the equipment should not consume a lot of energy - overheating means slowing down, breakdown, or a large load on the cooling system.
Also, the hardware should be flexibly customizable and provide the possibility of future expansion of functions and quick updates, for example, to protect against new viruses.
At Vas Experts, we compared the devices of primary and middle class, both domestic and foreign manufacturers, which almost all operators can afford. Based on the data we collected, we can conclude that the most expensive junior systems were the Cisco SCE1000 and Huawei SIG9800-X3, and the latter has the highest performance.
As for the middle segment, here we observed almost complete equality in performance and equipment of devices. The only difference is that Russian systems are based on standard components, and this allows you to increase productivity at a much lower cost, because additional extensions for devices from foreign companies such as Sandvine, Allot or Procera are much more expensive and require a license extension (you can find the full analysis in our blog ).
Based on this, it can be concluded that Russian manufacturers can compete with international companies, but their solutions should be used in networks with a low traffic volume and a small number of subscribers. Otherwise, you should pay attention to the closed platforms of foreign developers, which have slightly greater reliability, stability and optimization.
Unfortunately, a comparison of the characteristics specified in the specification will not give a complete understanding of the practical benefits, so it is important to check the equipment on test copies from manufacturers. However, it is not always possible to get devices for testing, in this case you need to focus on the experience of people who already work with specific systems and technologies, look for information on forums and technical support specialists.
Also, when choosing a system, integration costs are of great importance (in addition to performance and functionality) - for many, this becomes a decisive factor for installing a DPI system. Here is an example of calculating the effectiveness of implementing such a system (ROI).
Suppose that the total volume of traffic consumed by subscribers is 20 Gbit per month, the cost of 1 Gbit is $ 3,000, and the cost of a DPI system is $ 75,000. Then the use of the DPI system with the declared efficiency of reducing the uplink by 35% will give 7 Gb / s bandwidth savings per month - this is $ 21,000 per month. In this case, the DPI system will pay off in 3.6 months (75000/21000 = about 3.6).
If you take into account that you don’t have to pay for adding additional functions (blocking prohibited sites, protection against viruses and attacks, CGNAT), then DPI implementation will pay off even faster, and customers will receive higher quality services for the same price. It is worth noting that Russian companies sell equipment at prices in rubles - this is a serious advantage for operators who receive their income in the same currency. In this case, buying and maintaining the system will be even more profitable.
In the first two materials, we talked about the composition of the system and the connection diagram, and in the second about usage scenarios . Today we will see what requirements are imposed on the hardware for the full-fledged operation of deep traffic analysis systems, and where such systems are used. / Flickr / Sean MacEntee / CC
Deep packet inspection (DPI) systems are most often used to control and filter traffic, sometimes to block protocols. Using DPI, you can track the data generated by applications and choose the appropriate action strategy.
For DPI to work, you need a software system installed on a suitable hardware platform. Only the right equipment and optimized software in conjunction will give a high level of performance.
Servers for DPI
DPI servers are usually similar to 1U servers, but they focus on network components, not on RAM and hard drives. DPI servers can have 4 to 8 1 GbE RJ45 ports, 4 10 GbE SFP + ports, or 2 40 GE QSFP + ports.
Network cards in the DPI system must support Bypass mode - if the server shuts down, the connection between the ports will continue to work using power from the built-in battery and will pass traffic without filtering. Also, the DPI-server usually has a system for monitoring the status of work (Advanced Lights Out Management), with which you can manage all the parameters remotely or through a graphical interface on the display. The BIOS of the server motherboard must be protected from damage by hardware and support remote update.
For the server to function, one or two Intel Xeon E5-2600 V4 processors (Broadwell-EP) and two hard drives combined in RAID 1 are sufficient. Typically, the main device has two SSD disks for installing the operating system, for example SmartOS, they are added to 24 HDD or SSD drives, as well as network ports and expansion shelf ports. To increase the data storage, in addition to the device, JBOD disk arrays are connected, each of which supports up to 70 HDDs or SSDs.
This model allows you to quickly and cost-effectively expand the volume for data storage, which is especially important when processing traffic statistics and caching content - images, videos and other similar files.
It is best to use the ZFS file system and RAID-Z technology to control disks, maintain their integrity and high speed. To ensure fault tolerance, it is advisable to install at least two interchangeable power supplies.
Often, manufacturers of deep traffic analysis systems offer ready-made sets of hardware and software - such servers differ from standard systems and can be additionally equipped with data storage or statistics collection systems in accordance with legal requirements.
On the other hand, it can be much more convenient if the proposed server is a device on a standard platform that can be easily upgraded. Such systems are more common among Russian companies - Proteus, Vas Experts, Peter-Service, Napa Labs.
Application Categories
Heavy Reading conducted a survey among telecom companies and collected data on the main areas of application of DPI. The most popular area was quality of service (QoS) - this includes monitoring the status of the network and solving problems with equipment.
Previously, home subscriber traffic (HSI - High Speed Internet) was almost not controlled - BitTorrent could take the entire free bandwidth, but now DPI allows operators to distribute the channel between different applications.
The second most important category for operators was network subscriber policy management (PCEF). According to research by analysts, this is the largest category of DPI applications in terms of volume and value, and it will only continue to grow. The third category of providers was called network gateways - today DPI is often used on routers in 4G (P-GW) and 3G (GGSN) networks.
The fourth category in the survey is the use of DPI to analyze user information. Initially, DPI was used to analyze network traffic and trends, but more and more often companies are connecting this technology to analyze subscriber behavior in real time in order to develop more suitable sets of services and at the same time select the right network load.
Properties and comparison of "iron"
According to experts from the University of California at Berkeley, for fast and reliable DPI operation, equipment must have certain properties. DPI is used to check network packets by thousands of identifiers, therefore, for high speed operation, it is necessary to divide this process into parallel flows, that is, the equipment must support parallelism in data processing.
Iron must process network packets at high speed to match the bandwidth of the gigabit network channel. In addition, the equipment should not consume a lot of energy - overheating means slowing down, breakdown, or a large load on the cooling system.
Also, the hardware should be flexibly customizable and provide the possibility of future expansion of functions and quick updates, for example, to protect against new viruses.
At Vas Experts, we compared the devices of primary and middle class, both domestic and foreign manufacturers, which almost all operators can afford. Based on the data we collected, we can conclude that the most expensive junior systems were the Cisco SCE1000 and Huawei SIG9800-X3, and the latter has the highest performance.
As for the middle segment, here we observed almost complete equality in performance and equipment of devices. The only difference is that Russian systems are based on standard components, and this allows you to increase productivity at a much lower cost, because additional extensions for devices from foreign companies such as Sandvine, Allot or Procera are much more expensive and require a license extension (you can find the full analysis in our blog ).
Based on this, it can be concluded that Russian manufacturers can compete with international companies, but their solutions should be used in networks with a low traffic volume and a small number of subscribers. Otherwise, you should pay attention to the closed platforms of foreign developers, which have slightly greater reliability, stability and optimization.
Practical benefit
Unfortunately, a comparison of the characteristics specified in the specification will not give a complete understanding of the practical benefits, so it is important to check the equipment on test copies from manufacturers. However, it is not always possible to get devices for testing, in this case you need to focus on the experience of people who already work with specific systems and technologies, look for information on forums and technical support specialists.
Also, when choosing a system, integration costs are of great importance (in addition to performance and functionality) - for many, this becomes a decisive factor for installing a DPI system. Here is an example of calculating the effectiveness of implementing such a system (ROI).
Suppose that the total volume of traffic consumed by subscribers is 20 Gbit per month, the cost of 1 Gbit is $ 3,000, and the cost of a DPI system is $ 75,000. Then the use of the DPI system with the declared efficiency of reducing the uplink by 35% will give 7 Gb / s bandwidth savings per month - this is $ 21,000 per month. In this case, the DPI system will pay off in 3.6 months (75000/21000 = about 3.6).
If you take into account that you don’t have to pay for adding additional functions (blocking prohibited sites, protection against viruses and attacks, CGNAT), then DPI implementation will pay off even faster, and customers will receive higher quality services for the same price. It is worth noting that Russian companies sell equipment at prices in rubles - this is a serious advantage for operators who receive their income in the same currency. In this case, buying and maintaining the system will be even more profitable.
