April 1, 2018 at 18:59Load testing script to verify compliance of the current parameters of communication channels with the declared
Issue
In organizations that use VPN / Internet data channels leased from telecom operators to build a corporate branch network, sooner or later a situation may arise when it suddenly turns out that the channels to some remote units do not meet the stated bandwidth requirements, or with a slight load of channels there are losses that affect the quality of network services.
Moreover, the monitoring system regularly monitors the availability of channels, losses on them, delay, but due to the fact that communication channels are not always loaded with traffic, especially backup channels, it cannot timely detect all deviations of their parameters from those agreed with telecom operators. For such purposes, periodic load testing of channels is required, as a result of which the loss on the channel is checked while it is being loaded by traffic that utilizes the channel to almost the maximum bandwidth, followed by monitoring the amount of received traffic by the remote router. I want to share my best practices regarding the automation of this process.
Idea
The idea is to develop and periodically use a script that runs in my case on a Linux server at a central point on the corporate network. It is understood that the script interacts with routers on which:
- SNMP protocol is configured, its identical versions and the same community;
- the bandwidth parameters in the settings of the channel-forming interfaces correspond to the channel bandwidth values declared by the telecom operators.
At the script input, a file with ip addresses of the channel-forming interfaces of the routers, the communication channels of which we must check, is received as a parameter. The number of lines in a file is not limited to any particular value. After sequential reading from the input file of the corresponding ip address, the script:
- It starts the background process of generating traffic in the direction of the read ip address, which utilizes the tested channel bandwidth by less than 100% so as not to completely load the channel. I limited the process of generating traffic to 2 minutes with a generated traffic bandwidth of 85% of the maximum channel bandwidth.
- A few seconds after the start of traffic generation, it requests via SNMP the current value of received bytes for the interface of the router with the specified ip address. Memorizes the current time t0.
- Launches a series of 60 ICMP packets. Fixes the amount of loss.
- Re-requests via SNMP the value of the received bytes of the same interface. Memorizes the current time t1. Calculates the measurement duration: t1-t0.
- It calculates the number of bytes received by the interface as a result of measurement, based on which it determines the interface loading bandwidth for incoming traffic at the time of testing. I want to note that not all traffic generated by the script can be delivered to the router. This is possible if the channel bandwidth mismatch is declared. The script should reveal similar facts.
- Outputs the following values as a result:
- the ratio of the band of delivered traffic to the band of generated traffic,
- the number of channel losses and
- the result of matching the channel parameters with the declared one, which is formed on the basis of the previous two values.
Implementation Features
Traffic generator
The TCPBLAST / UDPBLAST program is used as a traffic generator, the source codes of which are located at the link: TCPBLAST / UDPBLAST
This program was the best suited for the role of a traffic generator for the specified task, because in addition to the ability to specify the bandwidth of generated traffic in the form of command line parameters, it has the functionality to start generating traffic with a time limit, which avoids the risks of loading communication channels for a long time in case of unforeseen situations, for example, when the script crashes.
Detection of SNMP Router Interface Indices for Test Channels
The script receives at the input only the ip address of the router interface. This is enough to determine the SNMP index of the interface.
The OID used for these purposes: .1.3.6.1.2.1.4.20.1.2
Test Channel Bandwidth Detection
The bandwidth of the interface and other parameters are determined using the corresponding OIDs with the
SNMP index of the interface substituted for them.
Script code
#!/usr/bin/perl
use strict;
use warnings;
use POSIX qw(strftime);
use SNMP;
die "Usage: $0 " if ($#ARGV < 0);
my $community = 'community';
### pingLoss function ###
sub pingLoss {
my ($param) = @_;
my @result = `ping $param`;
foreach my $str (@result) {
if ($str =~ /(\d+)%/ && $1 < 100) {
return $1;
}
}
return 100;
}
### stressTest function for one host ###
sub stressTest {
my $ifIp = shift;
my $ifIndex;
my $ifName = "";
my $ifAlias = "";
my $ifSpeed = "";
my $ifInOctets;
my $ifInOctetsBegin;
my $ifInOctetsEnd;
my $sendBytes;
my $sysName = "";
my $testPeriod;
my $pingLossCount = "";
my $inOutPercent = "";
my $testStatus;
my $nowDateTime = strftime "%Y.%m.%d %H:%M:%S", localtime;
if (&pingLoss("$ifIp -c 3 -i 0.2 -W 2") > 90) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tping error\n";
return 0;
}
my $sess = new SNMP::Session(DestHost => "$ifIp:161",
Community => $community,
Version => "2c",
NonIncreasing => 1,
UseLongNames => 1,);
$sysName = $sess->get('.1.3.6.1.2.1.1.5.0');
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tsnmp sysName error\n";
return 0;
}
if ($sysName =~ m/^([\w_-]+)\./) {
$sysName = $1;
}
$ifIndex = $sess->get('.1.3.6.1.2.1.4.20.1.2.' . $ifIp);
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tsnmp ifIndex error\n";
return 0;
}
$ifName = $sess->get('.1.3.6.1.2.1.31.1.1.1.1.' . $ifIndex);
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t$sysName\t\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tsnmp ifName error\n";
return 0;
}
$ifAlias = $sess->get('.1.3.6.1.2.1.31.1.1.1.18.' . $ifIndex);
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t\t$ifSpeed\t$inOutPercent\t$pingLossCount\tsnmp ifAlias error\n";
return 0;
}
$ifSpeed = $sess->get('.1.3.6.1.2.1.2.2.1.5.' . $ifIndex);
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t\t$inOutPercent\t$pingLossCount\tsnmp ifSpeed error\n";
return 0;
}
if ($ifSpeed > 30000000) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tHigh speed channel\n";
return 0;
}
my $sendBytesSec = $ifSpeed * 0.85 / 8;
system("killall udpblast > /dev/null 2> /dev/null");
system("udpblast -c 1000000 --rate $sendBytesSec,100s $ifIp > /dev/null 2> /dev/null &");
sleep(5);
my $testBegin = strftime "%s", localtime;
$ifInOctetsBegin = $sess->get('.1.3.6.1.2.1.2.2.1.10.' . $ifIndex);
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tsnmp ifInOctetsBegin error\n";
return 0;
}
$pingLossCount = &pingLoss("$ifIp -c 60 -W 1.5");
system("killall udpblast > /dev/null 2> /dev/null");
$ifInOctetsEnd = $sess->get('.1.3.6.1.2.1.2.2.1.10.' . $ifIndex);
if ($sess->{ErrorNum}) {
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\tsnmp ifInOctetsEnd error\n";
return 0;
}
my $testEnd = strftime "%s", localtime;
$testPeriod = $testEnd - $testBegin;
$ifInOctets = $ifInOctetsEnd - $ifInOctetsBegin;
$sendBytes = $sendBytesSec * $testPeriod * 1.04;
$inOutPercent = sprintf("%d", $ifInOctets * 100 / $sendBytes);
$testStatus = $inOutPercent > 89 && $pingLossCount < 6 ? "Good" : "Bad";
print "$nowDateTime\t$ifIp\t$sysName\t$ifName\t$ifAlias\t$ifSpeed\t$inOutPercent\t$pingLossCount\t$testStatus\n";
}
### Main ###
open (inputFile, $ARGV[0])
or die "Failed to open $ARGV[0]: $!\n";
print "date time \tipAddress \tsysName \tifName \tifAlias \tifSpeed \tinOutPercent\tpingLossCount\ttestStatus\n";
foreach my $readString () {
my $ipAddress = "";
if ($readString =~ m/([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})/ && $1 < 256 && $2 < 256 && $3 < 256 && $4 < 256) {
$ipAddress = "$1.$2.$3.$4";
}
else {next};
&stressTest($ipAddress);
}
I would be very happy for constructive criticism in order to optimize the script algorithm.