Why is it too late to connect under attack to the DDoS neutralization service
“I used to have to think . ” This Russian maxim is applicable to very many situations in life, but these words are doubly true in protecting network resources from denial of service attacks.
Preliminary planning is extremely important if you intend to create a popular and competitive service somewhere on the Internet - not only attackers, but also competitors can pay attention to it. Ultimately, an unexpected and large influx of users is essentially no different from a distributed attack.
If you did not attend to the defense in advance, the conditions under which you will have to return to this topic may be completely suboptimal. We decided to introduce the reader to several key points - the difficulties that any service that is under attack and trying to connect to the system to neutralize denial of service attacks will encounter.
Lyrical digression: if you intend to go on vacation to a tropical country on another continent, then it is useful, in addition to the generally accepted travel insurance, to take care of the issue of vaccinations against the most dangerous and common diseases.
Yes, the chance of a mosquito bite is small if you wear a long sleeve and roll up your pants in socks. In an extreme case, of course, they will save you. But this process can be delayed, you will spend a lot of nerves and, quite possibly, funds for medical specialists and drugs both on the spot and upon return. But you just had to get a vaccine against a dangerous disease and make sure that your insurance covers all cases.
What are the benefits of connecting to a network filtering traffic before an attack?
The first and most important thing is knowledge about user behavior. Observing the traffic of the target resource to stress, we are well aware of what the normal behavior of the users of the resource is and what traffic statistics are natural for the resource. When under attack, a gradual (or fast, it all depends on the attackers) degradation of the service occurs, as a result of which the behavior of normal users of the resource changes significantly. In some cases, the actions of legitimate users begin to resemble an attack and worsen its consequences, as an example - a cyclical update of the same page by the user to obtain the desired page in the browser.
The stronger the degradation of the service under attack, the more difficult it is to distinguish legitimate users from malicious bots. If we have not seen user behavior in anticipation of an attack with normal and proven legitimate behavior, we do not know in advance by what signs it is possible to distinguish a real user from an attacker bot sending garbage requests to the server. In addition, the amount of traffic that accompanies a normal service workflow would also be known.
If at the beginning of the attack the network or filtering equipment was not included in its neutralization, then in the learning process for automation and learning models, all users who click "Refresh page" are almost indistinguishable from attack traffic, respectively, the probability of their blocking is high. Knowing the previous behavior of users of the resource, you can protect them and yourself from the occurrence of such a situation.
In the same way, with the volume of traffic and traffic statistics in a normal situation - in the absence of such information at the time of the attack, it will be very difficult to say with certainty whether the traffic returned to normal after neutralizing the attack due to the fact that real users could be blocked, just having slightly different behavior from median during service degradation under attack.
Of course, such problems are not always found, but they become especially acute in some attack scenarios. Depending on the complexity of the attacking bot and the cunning of the attacker, an attack to the root of the service can become really unpleasant - in this case, when the service degrades, the behavior of the bots looks almost indistinguishable from the behavior of legitimate users who simply have “nothing to open”. In the case of attacks of another type, as a rule, significant differences remain in the legitimate and malicious behavior in terms of data flows.
Imagine a banal everyday situation - a breakthrough of a water pipe, a flood. You were not ready for it, there were no plugs at hand, shut-off taps on the pipe and so on, the worst case scenario. You can quickly close a hole and stop the flow, but furniture and children’s toys are already floating all over the apartment, and guests are knocking on the door. Can you let them go? Hardly.
What are the real cons and possible consequences when connecting under attack?
“Sick to healthy” is about neutralizing DDoS. And strangely enough, but even for a long time companies on the market with professional employees and thousands of consumers across the country or even around the world do not always think about neutralizing DDoS before the onset of the attack. We would like to change this situation by delivering adequate information to the audience, but we will not build any illusions - until the thunder strikes, the man will not cross himself.
Let's look at the main difficulties and difficulties that a resource under attack has to cope with and as a service that neutralizes distributed attacks.
The first thing we can say with confidence in the case of a service that connects to the filtering system while under attack is that it is very likely to experience partial or complete inaccessibility for users.
The second - if you are being attacked, then most likely the attacker knows not only your domain name, but also the IP address. In addition, usually the IP address of the attacked resource belongs to the host, so we a priori believe that the host is also known. That is, the attacker has enough information to modify the attack vector.
It takes time to move quickly to either a different hosting provider (maybe it’s not your attack) or change the IP transit provider (you may be just a side victim of attackers)
Of course, working out false positives is not the only difficulty. Connecting under attack does not eliminate the problem of partial or complete inaccessibility, and if the attack has already begun and has already caused some harm, then in addition to connecting the resource to the protection system, “damage control” is needed, that is, work on the consequences of the attack on the side of the consumer and the customer.
It happens that the attack “went well” and caused significant damage. Literally, this means the following - in addition to clogging the communication channel, through which it will be necessary anyway to arrange connectivity with the cloud-based provider of traffic filtering, while there are no dedicated channels, the border router or the balancer can “die” (refuse to serve). Other infrastructure components may also be bad, for example, a firewall (firewall), which is a delicate device and does not like distributed denial of service attacks, often looks into a disconnect from the outside world. Stopping the normal operation of the firewall threatens a hole in the network perimeter. Yes, this is treated by rebooting the failed equipment, but you still need to get to it and wait for it to return to a consistent state.
Speaking of attacks on the application layer (L7), everything can be fine here with the channel, and there is connectivity, but the web application is still sick, is depressed and can, for example, fall off the base or fall along with it. And returning it to its normal consistent state can take from several hours to several days, depending on the existing architectural difficulties.
There are two most common ways to connect to traffic filtering networks - using DNS or BGP. Let's consider these two scenarios separately.
DNS
Assume that the attack is conducted only by the domain name.
A typical scenario for connecting a site under a DNS attack is as follows: the owner of a network resource represented by a domain name, whose DNS A record contains the current IP address of the attacked web server, turns to us for help. After observing all the formalities, Qrator Labs gives the client a special IP address, to which he replaces his current address.
As a rule, at this moment you have to consider the possible high TTL for changing the DNS record, which can be from a few hours to a day maximum (RFC limit: 2147483647 seconds) - during this time the old A record will exist in the DNS recursors cache. Therefore, if you recognize in advance that an attack is likely, you must have a low TTL for the DNS A record.
But in some cases, even this may not save. After all, attackers check its effectiveness during an attack, and seeing that the attack does not give a result, the smart villain will quickly switch to the attack at the IP address that he “remembers”, bypassing the filtering network.
In this case, a new IP address is required, which is unknown to the attackers and does not compromise the protected domain name in any way, preferably in a different address block, and ideally from another service provider. After all, attackers can always go further and switch the vector of the DDoS attack into the address block (prefix) of the supplier, having sufficient power. Well, to us - you are welcome. We move on.
BGP
In the case of BGP connection, everything looks a little different.
If the attacked service has its own address blocks (prefixes) and wants to transfer the entire infrastructure under protection entirely by announcing its own prefixes through the DDoS neutralization service provider, what does this process look like?
An autonomous system under attack is added to our AS-SET, in order to be able to announce our own prefixes, after which the notorious day (24 hours) begins for all our uplinks to receive this information and update prefix lists. Naturally, in an emergency, we try to meet such a client, forcing this process, but this is not possible in every case and is done manually. In the light of the foregoing, time is a key and basic stress factor, because it is necessary to protect a resource without delay, “anesthetize instantly”.
In the case when the owner of the prefix prepares for the attack in advance by making a number of presets, it can be applied to the Qrator filtering network almost instantly, saving the time and nerves of technical specialists in extracurricular time and in emergency mode. Mutual integration in this case is not only a technical, but also a psychological process.
Lyrical digression: your provider of connectivity services buys IP transit, as a rule, from large and reliable telecom operators, not lower than regional Tier-1 operators. They filter the prefixes coming to them from clients based on some list (prefix-list), which they take from open databases (RIPE, RADB and others). Regularly, some providers of IP-transit service for protection against DDoS update these filters cyclically once a day, while others do this only on request. With a competent DDoS neutralization service provider, points of presence are located all over the world.
The most difficult case of connecting under attack is the appeal of the owner of a complex infrastructure that has a variety of equipment: routers, firewalls, load balancers, and other wonders of modern technology. In such scenarios, even a regular connection to a DDoS neutralization service provider is a complex process, which should be approached in a balanced and planned manner. It’s even a shame to mention that in the event of an attack on such a service, its emergency connection to the filtering network eats up a lot of effort and leaves no time for careful planning, because every minute counts under the attack. As long as these problems are not resolved, the service has problems with user accessibility and degrades.
Often, a DDoS attack is combined in some order with a hacker attack - hacking and penetration, either before a denial of service attack, or after. The risks involved are of a completely different order: the leak of user data, gaining full access to the system by an attacker. These problems need to be solved perpendicularly with the neutralization of DDoS, which in the worst case leads to even longer inaccessibility for users, and at best to an increased delay for those who try to request the necessary page.
Postscript .
Colleagues, we bring to your attention the following important news: www.ietf.org/mail-archive/web/idr/current/msg18258.html
Initiative to implement the mechanism of automatic protection against the occurrence of “route leaks” (direct route leaks) which Qrator Labs engineers accepted, successfully passed the adoption call stage and moved to the Interdomain Routing (IDR) working group.
The next step is the revision of the document as part of the IDR and, subsequently, verification by the steering group ( IESG ). If you successfully complete these steps, the draft will become the new RFC networking standard .
Authors: Alexander Azimov, Evgeny Bogomazov, Randy Bush (Internet Initiative Japan), Kotikalapudi Sriram (US NIST) and Keyur Patel (Arrcus Inc.) realize that there is a strong demand for proposed changes in the industry . However, the rush in this process is also unacceptable, and the authors will make every effort to make the proposed standard convenient for both transnational operators and small home networks.
We thank the technical specialists who expressed their support in the framework of the adoption call. At the same time, special thanks to those who not only expressed their opinion, but also sent their comments. We will try to take them into account in subsequent changes to this draft.
We also want to inform interested engineers that you still have the opportunity to express your own wishes for additions and clarifications through the IETF mailing list (draft-ymbk-idr-bgp-open-policy@ietf.org) or through the Qrator Labs initiative site .
It is also important to note that by the time the final decision is made, the standard should have two working implementations. One of them is already available - this is our development based on the Bird routing daemon, available on Github: github.com/QratorLabs/bird . We invite vendors and the open source community to join this process.