March 22, 2016 at 17:08Secrets of testing Ethernet channels
Good afternoon dear friends. For several years she worked as a system administrator in a number of corporate and home providers in St. Petersburg and to this day I often encounter the fact that when buying equipment, operators look more at the price and description of functions than at real indicators, suppliers usually do not write anything about them, as a result which instead of having one switch, you have to install more and more, and the quality of communication may not be better. Not all operators are aware of the existence of the concept of SLA (Service Level Agreement), for this reason I have collected reliable information on testing networks and equipment, and I am ready to provide it to your attention.
Ethernet needs to be tested!
Are there any exact definition and recipes for how to conduct tests of channel bandwidth and the quality of the provided communication? I found several articles from which only one thing became clear, today in Russia networks are tested using methods designed for other purposes, and this cannot but surprise, because communication services in large cities of our country are quite developed, there is a high-speed channel in literally every apartment, and some operators already provide gigabit channels to home customers, but not everyone knows about testing methods for the quality of telematic services provided.
What exactly and why you need to test?
Think about how often a pig in a poke is bought today:
This is a tiny handful of examples of what customers and telecom operators are risking today.
Software utilities for testing the "Internet"
Full testing of the channel cannot be echo requests, ping and mtr will never tell you what the channel capacity is. Iperf and other software utilities will not be able to tell about this, since while using the network and testing the software utilities do not know the amount of user data that is currently in the channel, as well as software testing, a number of inaccuracies are possible due to the presence of packet headers, depending headers remain standard length of the frame size, and the body with the data increases or decreases, software utilities determine the bandwidth of the channel without considering the size of the headers, which is different the size of the packages introduces a certain mess into such testing.
You will not be able to evaluate the quality of the leased vlan by looking at the channel loading schedule or downloading bulk files from the Internet. Why speedtest.net is not proof of the speed of the provided channel, probably not worth specifying? Indeed, it is immediately clear that - it is not known which channels and through which networks they go to speedtest servers, it is also unknown how much the channel is loaded during the test, and many other test parameters, and if there are so many unknowns in the test, then its results can in no way be accurate. The result of speedtest is rather a delta from certain indicators, rather than real numbers.
The quality of the provided communication services is a combination of many parameters, and using the right tools you can quickly and efficiently obtain accurate data about the provided service. It is important not only to obtain accurate data, but also to have confidence that the data can be used to prove their case, for example, in court.
Ethernet Techniques and Analyzers
Today, there are two main methods for testing bandwidth: the old one - RFC-2544 and a little younger: Y.1564 . The ITU-T Y.1564 methodology is more relevant today, it has descriptions for testing modern, high-speed communication channels with modern concepts about SLA (Service Layer Agreement).
Since the quality of an ethernet channel is a combination of many factors, therefore, proper testing should cover all of these combinations as much as possible. When testing, it is necessary to take into account many aspects and it would be useful to have advanced features such as BER Test , packet jitter, support for MPLS, QoS, load testing of application-level protocols (http, ftp, etc ...).
For testing channels from 1G to 10G and higher, it is quite difficult to do load tests using non-specialized iron; processors are often not able to generate enough traffic, unlike specialized testers-analyzers. Such devices can be put in a rack, cabinet, even in a drawer in the attic and run tests remotely, or you can make automatic measurements at different time intervals. Any portable analyzer devices will not deteriorate in the harsh conditions of the sewer, as they pass severe tests for strength.
Delivery and acceptance of communication channels.
It is best to have a tester analyzer in the regular arsenal for handing over or accepting constructed lines and highways, for working to high standards, although companies specializing in field testing can be found on the Internet. For some reason, it is believed that buying a tester analyzer is very expensive.
Learn more about the RFC-2544 testing methodology and how it works.
The RFC-2544 methodology recommends measuring different frame sizes: for Ethernet traffic, frames are 64, 128, 256, 512, 1024, 1280, 1518 octets in size, for each frame size a separate start of serial testing is required. If necessary, you can also test for the Jumbo frame (frames size 4096 or 9000 octets). Different frame sizes are needed to simulate different types of traffic.
Initially, the technique was developed directly for testing network devices, for example, when developing switches, but a set of functions was adapted to measure the quality of channels. The technique was approved in 1999 by ISOC .
The technique offers a set of 6 tests, I will describe in more detail how the testing is conducted, for clarity of perception:
Determining the throughput of the device under test (Throughput)
Test description: a small amount of packets specially formed by the tester are sent at a certain speed to the device input port , on the output port, the number is calculated, if more is transmitted than received, the speed decreases and the test starts again.
Determination of frame delay time (Latency)
Description of the test: after determining the throughput (Throughput), for each frame size, at the corresponding maximum speed, a packet stream is sent to a specific address. The stream should have a minimum duration of 120 seconds. After 60 seconds, a label is inserted into 1 packet. The label format is determined by the equipment manufacturer. On the transmitting side, the time by which the packet with the label was completely sent is recorded. On the receiving side, the label is determined and the time of complete reception of the packet with the label is recorded. Latency is the difference between the sending time and the receiving time. This test, according to the methodology, must be repeated at least 20 times. Based on the results of 20 measurements, the average delay is calculated. The test should be carried out by sending the entire test stream to one address and sending each frame to a new address.
Determination of the frame loss rate (Frame loss rate)
Test description: a certain number of frames at a certain speed is sent to the input port of the device and the number of packets received from the output port of the device is counted. The frame loss rate is calculated as follows:
((number of frames transmitted - number of frames received) * 100) / number of frames transmitted
The first sending occurs at the maximum possible speed, then the sending speed is reduced with a maximum step of 10%, according to the method, reducing the% step will give most accurate results. The decrease in speed must be continued until the last two sendings are error-free, namely, we will find out the maximum data transfer rate at which the frame loss rate becomes 0.
Testing the ability to handle back-to-back frames
Description of the test: the test is reduced to sending a certain number of frames with a minimum inter-frame delay to the input port of the device under test and counting frames from the output port of the device. If the number of sent frames and received is equal, then the volume of sent frames increases and the test is repeated, if the received packets are less than the sent frames, the amount of sent frames decreases and the test is repeated. As a result, we should get the maximum number of packets sent and received without loss for each packet size, this will be the value of the back-to-back test. According to the technique, the duration of sending frames to the device port should not be less than two seconds, and the minimum number should be at least 50 times. The final figure is the average result of 50 tests.
Recovery after overload (System recovery), applicable only for testing devices
Description of the test: a stream of frames is sent to the device input for at least 60 seconds at a rate of 110% relative to the measured throughput test. If the throughput test shows perfect results, then the maximum speed of this connection is selected. At the time of congestion, the flow rate decreases by half and the difference between the time of the decrease in the flow rate and the time when the last frame was lost is detected.
Recovery time of the tested device after restart (Reset), applicable only for testing devices
Description of the test: a continuous stream of frames is sent to the input of the device at the speed determined as a result of the throughput test with a minimum frame size. The device is reset. The recovery time after a reset is the difference between the time the last packet was received before the reset and the time the first packet was received after the reset. Both hardware and software device reset types are tested.
What has changed with the latest Y.1564 technique?
New recommendations were reviewed and approved by the ITU in 2011.. To the already stated recommendations, RFC 2544 adds packet jitter (jitter), namely the ability to calculate the time difference when receiving a series of consecutive data packets related to the same stream, in an ideal world it should not exist, but in problem networks, the sequence can be broken, which may affect the speed of data processing. RFC2544 allows you to do checks exclusively at the maximum channel speed at which there will be no packet loss, and this is usually higher than the CIR (Committed Information Rate - guaranteed bandwidth) speed . Y.1564 created specifically for SLA, estimates the speed and quality of the provided channel according to key performance indicators (KPI) and allows you to check the provided channel in accordance with the contract.
Y.1564 allows you to check the guaranteed bandwidth, the maximum allowable, as well as give a load in excess of the band, for example, to check the shaper settings.
There are several differences between the methods, RFC2544 does not verify the correctness of the service settings (KPI compliance with the specified ones, and the speed limit is higher than EIR (Excess Information Rate - maximum unguaranteed bandwidth), in order to avoid network congestion). In the original version of RFC2544, jitter is not measured. According to RFC2544, each test is launched in a separate thread, which does not allow measuring the quality of the provided services in aggregate and increases the testing time, another minus of RFC2544 is that there is no profiling ability to test different types of traffic in one channel, for example, if the network uses QoS , Y.1564 takes into account shortcomings and slightly expanded functionality.
Can I test only new channels or already working ones too?
You need to test new channels, and even more so old ones. You can learn in advance about emerging issues without bringing customers to a support call. With modern testers and analyzers, it is possible to carry out checks on a working network, check channels both at a speed of 10/100 / 1000Mbit and 10/40 / 100G. There is one BUT, it is very important to understand what and how you are doing, it is important not to accidentally put the channel under test.
Test Modes - In / Out of service.
Today, network testing seeks to systematize and constantly monitor channels, earlier versions of the RFC2544 methodology were created for testing channels / equipment in OutOfService mode, and were used mainly for equipment testing, but today all manufacturers of test devices are switching to newer ones testing standards allowing continuous monitoring of the network in InService mode. Such testing allows you to check the bandwidth speed without disconnecting clients, which is important for telecom operators.
Epilogue
Comrades, as one friend of mine says, let's fight together with the “co-workers” and begin to test what we are building and what we are exploiting.
Used Books:
RFC 2544 Methodology: https: //www.ietf.org/rfc/rfc2544.txt Y.1564
Methodology: www.itu.int/rec/T-REC-Y.1564-201103-I/en
André article about the difference in methods: www.mrv.com/blog/why-rcf2544-not-sufficient-anymore
* The opinion of the company may not coincide with the opinion of the author ;-)
Ethernet needs to be tested!
Are there any exact definition and recipes for how to conduct tests of channel bandwidth and the quality of the provided communication? I found several articles from which only one thing became clear, today in Russia networks are tested using methods designed for other purposes, and this cannot but surprise, because communication services in large cities of our country are quite developed, there is a high-speed channel in literally every apartment, and some operators already provide gigabit channels to home customers, but not everyone knows about testing methods for the quality of telematic services provided.
What exactly and why you need to test?
Think about how often a pig in a poke is bought today:
- Communication channels leased by you or leased;
- Acceptance of communication channels built by you or for you;
- Provided communication services, especially if there is a penalty in the contract;
- The equipment you want to buy, but they want to sell it to you, and they say that it is super-cool and inexpensive.
This is a tiny handful of examples of what customers and telecom operators are risking today.
Software utilities for testing the "Internet"
Full testing of the channel cannot be echo requests, ping and mtr will never tell you what the channel capacity is. Iperf and other software utilities will not be able to tell about this, since while using the network and testing the software utilities do not know the amount of user data that is currently in the channel, as well as software testing, a number of inaccuracies are possible due to the presence of packet headers, depending headers remain standard length of the frame size, and the body with the data increases or decreases, software utilities determine the bandwidth of the channel without considering the size of the headers, which is different the size of the packages introduces a certain mess into such testing.
You will not be able to evaluate the quality of the leased vlan by looking at the channel loading schedule or downloading bulk files from the Internet. Why speedtest.net is not proof of the speed of the provided channel, probably not worth specifying? Indeed, it is immediately clear that - it is not known which channels and through which networks they go to speedtest servers, it is also unknown how much the channel is loaded during the test, and many other test parameters, and if there are so many unknowns in the test, then its results can in no way be accurate. The result of speedtest is rather a delta from certain indicators, rather than real numbers.
The quality of the provided communication services is a combination of many parameters, and using the right tools you can quickly and efficiently obtain accurate data about the provided service. It is important not only to obtain accurate data, but also to have confidence that the data can be used to prove their case, for example, in court.
Ethernet Techniques and Analyzers
Today, there are two main methods for testing bandwidth: the old one - RFC-2544 and a little younger: Y.1564 . The ITU-T Y.1564 methodology is more relevant today, it has descriptions for testing modern, high-speed communication channels with modern concepts about SLA (Service Layer Agreement).
Since the quality of an ethernet channel is a combination of many factors, therefore, proper testing should cover all of these combinations as much as possible. When testing, it is necessary to take into account many aspects and it would be useful to have advanced features such as BER Test , packet jitter, support for MPLS, QoS, load testing of application-level protocols (http, ftp, etc ...).
For testing channels from 1G to 10G and higher, it is quite difficult to do load tests using non-specialized iron; processors are often not able to generate enough traffic, unlike specialized testers-analyzers. Such devices can be put in a rack, cabinet, even in a drawer in the attic and run tests remotely, or you can make automatic measurements at different time intervals. Any portable analyzer devices will not deteriorate in the harsh conditions of the sewer, as they pass severe tests for strength.
Delivery and acceptance of communication channels.
It is best to have a tester analyzer in the regular arsenal for handing over or accepting constructed lines and highways, for working to high standards, although companies specializing in field testing can be found on the Internet. For some reason, it is believed that buying a tester analyzer is very expensive.
Learn more about the RFC-2544 testing methodology and how it works.
The RFC-2544 methodology recommends measuring different frame sizes: for Ethernet traffic, frames are 64, 128, 256, 512, 1024, 1280, 1518 octets in size, for each frame size a separate start of serial testing is required. If necessary, you can also test for the Jumbo frame (frames size 4096 or 9000 octets). Different frame sizes are needed to simulate different types of traffic.
Initially, the technique was developed directly for testing network devices, for example, when developing switches, but a set of functions was adapted to measure the quality of channels. The technique was approved in 1999 by ISOC .
The technique offers a set of 6 tests, I will describe in more detail how the testing is conducted, for clarity of perception:
Determining the throughput of the device under test (Throughput)
Test description: a small amount of packets specially formed by the tester are sent at a certain speed to the device input port , on the output port, the number is calculated, if more is transmitted than received, the speed decreases and the test starts again.
Determination of frame delay time (Latency)
Description of the test: after determining the throughput (Throughput), for each frame size, at the corresponding maximum speed, a packet stream is sent to a specific address. The stream should have a minimum duration of 120 seconds. After 60 seconds, a label is inserted into 1 packet. The label format is determined by the equipment manufacturer. On the transmitting side, the time by which the packet with the label was completely sent is recorded. On the receiving side, the label is determined and the time of complete reception of the packet with the label is recorded. Latency is the difference between the sending time and the receiving time. This test, according to the methodology, must be repeated at least 20 times. Based on the results of 20 measurements, the average delay is calculated. The test should be carried out by sending the entire test stream to one address and sending each frame to a new address.
Determination of the frame loss rate (Frame loss rate)
Test description: a certain number of frames at a certain speed is sent to the input port of the device and the number of packets received from the output port of the device is counted. The frame loss rate is calculated as follows:
((number of frames transmitted - number of frames received) * 100) / number of frames transmitted
The first sending occurs at the maximum possible speed, then the sending speed is reduced with a maximum step of 10%, according to the method, reducing the% step will give most accurate results. The decrease in speed must be continued until the last two sendings are error-free, namely, we will find out the maximum data transfer rate at which the frame loss rate becomes 0.
Testing the ability to handle back-to-back frames
Description of the test: the test is reduced to sending a certain number of frames with a minimum inter-frame delay to the input port of the device under test and counting frames from the output port of the device. If the number of sent frames and received is equal, then the volume of sent frames increases and the test is repeated, if the received packets are less than the sent frames, the amount of sent frames decreases and the test is repeated. As a result, we should get the maximum number of packets sent and received without loss for each packet size, this will be the value of the back-to-back test. According to the technique, the duration of sending frames to the device port should not be less than two seconds, and the minimum number should be at least 50 times. The final figure is the average result of 50 tests.
Recovery after overload (System recovery), applicable only for testing devices
Description of the test: a stream of frames is sent to the device input for at least 60 seconds at a rate of 110% relative to the measured throughput test. If the throughput test shows perfect results, then the maximum speed of this connection is selected. At the time of congestion, the flow rate decreases by half and the difference between the time of the decrease in the flow rate and the time when the last frame was lost is detected.
Recovery time of the tested device after restart (Reset), applicable only for testing devices
Description of the test: a continuous stream of frames is sent to the input of the device at the speed determined as a result of the throughput test with a minimum frame size. The device is reset. The recovery time after a reset is the difference between the time the last packet was received before the reset and the time the first packet was received after the reset. Both hardware and software device reset types are tested.
What has changed with the latest Y.1564 technique?
New recommendations were reviewed and approved by the ITU in 2011.. To the already stated recommendations, RFC 2544 adds packet jitter (jitter), namely the ability to calculate the time difference when receiving a series of consecutive data packets related to the same stream, in an ideal world it should not exist, but in problem networks, the sequence can be broken, which may affect the speed of data processing. RFC2544 allows you to do checks exclusively at the maximum channel speed at which there will be no packet loss, and this is usually higher than the CIR (Committed Information Rate - guaranteed bandwidth) speed . Y.1564 created specifically for SLA, estimates the speed and quality of the provided channel according to key performance indicators (KPI) and allows you to check the provided channel in accordance with the contract.
Y.1564 allows you to check the guaranteed bandwidth, the maximum allowable, as well as give a load in excess of the band, for example, to check the shaper settings.
There are several differences between the methods, RFC2544 does not verify the correctness of the service settings (KPI compliance with the specified ones, and the speed limit is higher than EIR (Excess Information Rate - maximum unguaranteed bandwidth), in order to avoid network congestion). In the original version of RFC2544, jitter is not measured. According to RFC2544, each test is launched in a separate thread, which does not allow measuring the quality of the provided services in aggregate and increases the testing time, another minus of RFC2544 is that there is no profiling ability to test different types of traffic in one channel, for example, if the network uses QoS , Y.1564 takes into account shortcomings and slightly expanded functionality.
Can I test only new channels or already working ones too?
You need to test new channels, and even more so old ones. You can learn in advance about emerging issues without bringing customers to a support call. With modern testers and analyzers, it is possible to carry out checks on a working network, check channels both at a speed of 10/100 / 1000Mbit and 10/40 / 100G. There is one BUT, it is very important to understand what and how you are doing, it is important not to accidentally put the channel under test.
Test Modes - In / Out of service.
Today, network testing seeks to systematize and constantly monitor channels, earlier versions of the RFC2544 methodology were created for testing channels / equipment in OutOfService mode, and were used mainly for equipment testing, but today all manufacturers of test devices are switching to newer ones testing standards allowing continuous monitoring of the network in InService mode. Such testing allows you to check the bandwidth speed without disconnecting clients, which is important for telecom operators.
Epilogue
Comrades, as one friend of mine says, let's fight together with the “co-workers” and begin to test what we are building and what we are exploiting.
Used Books:
RFC 2544 Methodology: https: //www.ietf.org/rfc/rfc2544.txt Y.1564
Methodology: www.itu.int/rec/T-REC-Y.1564-201103-I/en
André article about the difference in methods: www.mrv.com/blog/why-rcf2544-not-sufficient-anymore
* The opinion of the company may not coincide with the opinion of the author ;-)
Only registered users can participate in the survey. Please come in.
Would testing equipment or networks be useful to you?
- 79.7% Yes, I want to know what I'm buying 217
- 20.2% None. It works and okay. 55
