August 31, 2018 at 12:37Confessions of a tester: how I rummaged in a competitor IDS
No self-respecting UTM network security gateway can do without an Intrusion Detection System (IDS). Another thing is that the option is often indicated by the manufacturer for a tick in order to keep up with competitors. I know from my experience as a tester how this happens - there seems to be IDS, but in fact it’s no good. That is why, when I was offered to test a relatively inexpensive UTM-piece of hardware, I suggested first of all to “drive out” its OWL.
UTM-piece of Traffic Inspector Next Generation S100 and Cisco 2960G Switch
If you have experience working with IDS, it is interesting to read about it in the comments to the article. It’s advisable - not about expensive hardware (it’s clear that some Cisco NGFW for a million is doing fine), but about solutions that are more affordable. I think lokalka administrators and potential users of network gateways will be curious to discuss who works with IDS and whether it is worth buying at a high price, when you can, after dancing with a tambourine, achieve the same for less money.
In this test, we are talking about the S100 model - the most affordable price in the Traffic Inspector Next Generation line (numbers mean that it is designed for 100 subscribers). Here is her description on the official website >>> . In short, the hardware contains, for example, a network filter, a VPN, a resource blocker, and, of course, IDS.
I propose a simple testing methodology - we check the throughput, and build the dependence of the number of dropped packets on the traffic intensity in increments of 50, 100, 150 and 200 Mb / s. Why do we take such intervals? We start from 100 Mb / s - the most popular client request, and put off plus and minus from it to see what happens in more or less loaded modes, as well as in extreme mode 200 Mb / s.
Life experience tells me that with all the active rules, the S100 may not pull, so I suggest the described procedure to be carried out in three modes: first, when all the rules are active, then with the part of the rules turned off (let's call it Group No. 1) and, finally, only when it’s active just a few rules (let's call it Group No. 2). We form the following rule groups:
- Group No. 0: well, that’s understandable, all the rules without exception.
- Group No. 1: Emerging Threads rule group (I have known the rules since testing IDS Snort, with the exception of emerging-games rules).
- Group No. 2: a group of rules that I consider simply binding on IDS (see below), including the addition of p2p rules, because from my own experience I know how large companies negatively relate to the fact that their employees actively use corporate network for downloading your favorite TV shows.
Testing is possible using the tcpreplay utility . This utility allows you to play back pre-recorded network traffic at a specific speed. Command: tcpreplay –i <interface> -l 0 testTI.pcap . File testTI.pcapit contains 1,146,313 packets (we choose such a volume so that, on the one hand, there are good statistics, on the other hand, the time to “run” would not be too long, in our case, no more than 15 minutes). In addition to this, as I said, we launch a torrent tracker.
Test bench layout:
If someone has questions about the testing methodology, I’m ready to answer in the comments.
So, the results.
Group No. 0
Testing on a full set implies downloading all the rules, and there are 30 305 of them.
When testing, we use the IDS default settings:
We start testing with 100 Mb / s and we understand that the piece of iron can barely pull: out of 114 thousand packets, 109 thousand are discarded! So there is no point in testing at 150 and even more so at 200 Mb / s. On the contrary, I propose to give the device a chance and conduct an additional test at 10 Mb / s. The result is in the table:
Note:
kernel_packets - sent packets;
kernel_drops - dropped packets. As you can see, with default settings and with a complete set of rules, large packet losses occur (> 30% relative to kernel_packets). Let's hope that optimizing the rules for settings will change the situation;
decoder_packets - the number of packages that the system correctly processed;
detect_alerts - number of detected attacks. The bulk of the attacks are of the Fragmented Package type, but the Torrent Tracker Detection attacks have also been identified.
Group No. 1
Obviously, the piece of hardware is not optimized to work as a full-fledged powerful IDS, but there is room for maneuver, namely the ability to change the route search mechanism, disable the loading of package contents into reports (payload), and also disable rule groups and certain package groups. A little experimentation with the settings - and we come to the option that I present in the following figure.
List of active rules that we leave:
Test result:
As you can see, the picture improved significantly with this group of rules (the percentage of dropped packets decreased several times). An attack such as “Detecting a torrent tracker” was successfully detected (“Fragmented packet” attacks no longer appeared).
Group No. 2
Settings are the same. List of active rules:
The result in the table:
Here the picture of packet processing is even better, which is expected.
Summary
The final results table, expressed as a percentage of kernel_drops relative to kernel_packets, is shown below:
Graphically, the result is as follows:
As you can see, the number of active rules and settings directly affect the efficiency. It does not make sense to turn on the settings to the maximum and all the rules at the same time - the losses immediately go off scale even by 10 Mb / s. In optimized modes, the piece of hardware feels normal up to 100 Mb / s, but losses increase sharply on more intensive traffic. However, for “office” use, 100 Mb / s is enough. If you drive the device at this speed and select the rules, you can achieve quite satisfactory IDS.
Perhaps, to use the full set of rules, improvements are needed to use the pf_ring mechanism (https://www.ntop.org/products/packet-capture/pf_ring/) as a mechanism for transferring a packet to userspace from the network interface buffer. To do this, you will need to use several instances of Suricata, which, of course, will take resources from other processes, but perhaps the game will be worth the candle.
I repeat that, in my opinion, the main purpose of the tested device is a firewall, and the IDS option is auxiliary. Honestly, I was ready for the fact that the hardware will fail. It turned out that with a certain gutta perception of the system administrator, the intrusion detection system in the S100 is fully operational.
PS As I said above, I urge readers to write about their experience in using IDS in relatively inexpensive solutions.
PPS Test is posted in the manufacturer’s account, because I don’t want to “shine” myself. Nevertheless, I am ready to answer all questions and debate in the comments, but, again, not on my own behalf :)
UTM-piece of Traffic Inspector Next Generation S100 and Cisco 2960G Switch
If you have experience working with IDS, it is interesting to read about it in the comments to the article. It’s advisable - not about expensive hardware (it’s clear that some Cisco NGFW for a million is doing fine), but about solutions that are more affordable. I think lokalka administrators and potential users of network gateways will be curious to discuss who works with IDS and whether it is worth buying at a high price, when you can, after dancing with a tambourine, achieve the same for less money.
In this test, we are talking about the S100 model - the most affordable price in the Traffic Inspector Next Generation line (numbers mean that it is designed for 100 subscribers). Here is her description on the official website >>> . In short, the hardware contains, for example, a network filter, a VPN, a resource blocker, and, of course, IDS.
I propose a simple testing methodology - we check the throughput, and build the dependence of the number of dropped packets on the traffic intensity in increments of 50, 100, 150 and 200 Mb / s. Why do we take such intervals? We start from 100 Mb / s - the most popular client request, and put off plus and minus from it to see what happens in more or less loaded modes, as well as in extreme mode 200 Mb / s.
Life experience tells me that with all the active rules, the S100 may not pull, so I suggest the described procedure to be carried out in three modes: first, when all the rules are active, then with the part of the rules turned off (let's call it Group No. 1) and, finally, only when it’s active just a few rules (let's call it Group No. 2). We form the following rule groups:
- Group No. 0: well, that’s understandable, all the rules without exception.
- Group No. 1: Emerging Threads rule group (I have known the rules since testing IDS Snort, with the exception of emerging-games rules).
- Group No. 2: a group of rules that I consider simply binding on IDS (see below), including the addition of p2p rules, because from my own experience I know how large companies negatively relate to the fact that their employees actively use corporate network for downloading your favorite TV shows.
Testing is possible using the tcpreplay utility . This utility allows you to play back pre-recorded network traffic at a specific speed. Command: tcpreplay –i <interface> -l 0 testTI.pcap . File testTI.pcapit contains 1,146,313 packets (we choose such a volume so that, on the one hand, there are good statistics, on the other hand, the time to “run” would not be too long, in our case, no more than 15 minutes). In addition to this, as I said, we launch a torrent tracker.
Test bench layout:
If someone has questions about the testing methodology, I’m ready to answer in the comments.
So, the results.
Group No. 0
Testing on a full set implies downloading all the rules, and there are 30 305 of them.
When testing, we use the IDS default settings:
We start testing with 100 Mb / s and we understand that the piece of iron can barely pull: out of 114 thousand packets, 109 thousand are discarded! So there is no point in testing at 150 and even more so at 200 Mb / s. On the contrary, I propose to give the device a chance and conduct an additional test at 10 Mb / s. The result is in the table:
Note:
kernel_packets - sent packets;
kernel_drops - dropped packets. As you can see, with default settings and with a complete set of rules, large packet losses occur (> 30% relative to kernel_packets). Let's hope that optimizing the rules for settings will change the situation;
decoder_packets - the number of packages that the system correctly processed;
detect_alerts - number of detected attacks. The bulk of the attacks are of the Fragmented Package type, but the Torrent Tracker Detection attacks have also been identified.
Group No. 1
Obviously, the piece of hardware is not optimized to work as a full-fledged powerful IDS, but there is room for maneuver, namely the ability to change the route search mechanism, disable the loading of package contents into reports (payload), and also disable rule groups and certain package groups. A little experimentation with the settings - and we come to the option that I present in the following figure.
List of active rules that we leave:
Test result:
As you can see, the picture improved significantly with this group of rules (the percentage of dropped packets decreased several times). An attack such as “Detecting a torrent tracker” was successfully detected (“Fragmented packet” attacks no longer appeared).
Group No. 2
Settings are the same. List of active rules:
The result in the table:
Here the picture of packet processing is even better, which is expected.
Summary
The final results table, expressed as a percentage of kernel_drops relative to kernel_packets, is shown below:
Graphically, the result is as follows:
As you can see, the number of active rules and settings directly affect the efficiency. It does not make sense to turn on the settings to the maximum and all the rules at the same time - the losses immediately go off scale even by 10 Mb / s. In optimized modes, the piece of hardware feels normal up to 100 Mb / s, but losses increase sharply on more intensive traffic. However, for “office” use, 100 Mb / s is enough. If you drive the device at this speed and select the rules, you can achieve quite satisfactory IDS.
Perhaps, to use the full set of rules, improvements are needed to use the pf_ring mechanism (https://www.ntop.org/products/packet-capture/pf_ring/) as a mechanism for transferring a packet to userspace from the network interface buffer. To do this, you will need to use several instances of Suricata, which, of course, will take resources from other processes, but perhaps the game will be worth the candle.
I repeat that, in my opinion, the main purpose of the tested device is a firewall, and the IDS option is auxiliary. Honestly, I was ready for the fact that the hardware will fail. It turned out that with a certain gutta perception of the system administrator, the intrusion detection system in the S100 is fully operational.
PS As I said above, I urge readers to write about their experience in using IDS in relatively inexpensive solutions.
PPS Test is posted in the manufacturer’s account, because I don’t want to “shine” myself. Nevertheless, I am ready to answer all questions and debate in the comments, but, again, not on my own behalf :)