October 28, 2011 at 16:51Digit of the month: 70K. Daily
The next post of Yevgeny Kaspersky asks to please Habr:
Our service is both dangerous and difficult. It’s also very difficult. It is difficult for an ordinary mortal to understand all the intricacies of the work of an antivirus company. And oh, how you want to talk about them! So, as far as we can, we try to translate them into human language. And the tip of this iceberg is a set of figures and facts that illustrate this service.
For example, here is such a curious infographic: There are many more interesting infographics here . But one of the most asked questions is “how many viruses do you find every day?” .
The question is actually non-trivial, but what to do - you need a simple answer to a simple question. And until recently, we had a standard number - 35 thousand. Something like that. Averaged. Without mind-blowing details about malvar families, polymorphism, vector patterns, records, etc.
But as early as several months this answer was often followed by a clarifying question that, supposedly, it’s not enough to converge! In general, we tensed here, counted and were stunned. As a result, an update appeared: 70 thousand . Daily. Yeah.
Again, I will not go into details (if anyone is interested - ask in comments). I’d better tell you how we manage to shovel these daily seventy thousand samples.
Well, many people know that our family talisman is a woodpecker. And for good reason. In the good old days, our virlab worked like that. Like on a conveyor belt, viruses sat and “hollowed”. Incidentally, a very complex, tedious and respected profession! And I, too, has been woodpecking for many years!
However, the times are not the same. And it’s clear that with the current flow, “fucking” is simply unrealistic, economically inefficient, and just plain stupid. We have been on combat duty for many years now ... auto woodpeckers ! The human factor gets only the most intellectual work - to disassemble the most complex samples, to investigate botnets, to ensure that auto-woodpeckers do not falsify, and, of course, to train and develop them in every possible way.
In general, there are several sources through which we obtain samples of malvari for analysis: self-propelled guns (“stick” to special traps),submissions (forwarded by users), exchange of collections with other antivirus companies and our cloud service KSN ( video , details ). Moreover, on the contribution to protecting users, the last source now occupies a leading position. Here is an example of KSN and let's see how our automation of processing malvari works.
KSN participating computers (and now there are more than 50 million of them ) send statistics (non-personalized!) About the work of our products to the cloud. There is information about trapped malware and infected sites, as well as a lot of useful information for detecting a new malware, for example, suspicious program behavior, hashes of downloaded files, and more.
Here, for example, the user launches a previously unknown file. Local antivirus scans it with all available tools - cleanly. We ask the cloud - there is no data. Ok - give the go-ahead for launch. And then it turns out that he somehow strangely registers himself in the registry, tries to access system services, establishes suspicious connections, has a double extension (jpg.exe) or something else. The signal arrives at KSN, where the system automatically calculates the reputation of the file (the weight of all attributes and actions) and makes a decision about the detection. As a result, the “fas” command is sent to the protected computer, the file is blocked, and its actions are rolled back. Of course, the more messages about the same file from different computers, the higher the processing priority and the higher the accuracy and criticality of the verdict. Appear such a file on other computers,
Another example.
Several users downloaded the file at the same link. But each time the file has a different hash. It smells like polymorphism! KSN begins to spin up the case and sees that, for example, the site was registered just a couple of days ago, some iframe was hanging on it or infected files were sent from it before (well, there are a lot of different signs). And again, the cloud calculates the reputation and sends a command to block both the file itself and access to the site.
Important: thanks to this approach, on average, only 40 seconds pass between a detection and a verdict !
But the work of our woodpeckers does not end there.
Another system is already downloading the same suspicious file from the network and transferring it to the automatic processor for analysis. There is generally a whole bunch of all kinds of patented and not yet patented technologies, therefore I will not dig deeper. This handler develops and tests updates familiar to everyone and puts it on servers for downloads.
Something like that. I remember about 8 years ago, competitors, envious, were delighted, how do we manage to carry out such a huge work with such small resources? Automation however! And only competent automation can cope with this crazy stream of malvari! By the way, although the scope of work is constantly expanding and deepening, the number of employees in the virlab has not changed for a year now.
This raises a logical question.But how do small anti-virus companies survive ? It is known that keeping a good virlab is not only a matter of money, but also of brains. Where do they find the resources to stay afloat, with almost no money spent on R&D?
Sensitive subject.
For several years now, the natural theft of the “Detect” has flourished in the antivirus industry . Instead of analyzing malvara, developing their expertise and inventing new technologies, some (and there are about a dozen of these “some”!) Companies simply spy on the results of others and stupidly add hash detection to their databases. Incompetent tests help themthat do not reflect the level of protection in real conditions. As a result, not the best software creeps up, the number of its installations is growing, and the overall level of security is falling. Honest companies lose their motivation for research, investment in R&D is falling, but sales of thieves are growing, and cyber villains are rejoicing.
Well, this is a topic for a separate story.
Our service is both dangerous and difficult. It’s also very difficult. It is difficult for an ordinary mortal to understand all the intricacies of the work of an antivirus company. And oh, how you want to talk about them! So, as far as we can, we try to translate them into human language. And the tip of this iceberg is a set of figures and facts that illustrate this service.
For example, here is such a curious infographic: There are many more interesting infographics here . But one of the most asked questions is “how many viruses do you find every day?” .
The question is actually non-trivial, but what to do - you need a simple answer to a simple question. And until recently, we had a standard number - 35 thousand. Something like that. Averaged. Without mind-blowing details about malvar families, polymorphism, vector patterns, records, etc.
But as early as several months this answer was often followed by a clarifying question that, supposedly, it’s not enough to converge! In general, we tensed here, counted and were stunned. As a result, an update appeared: 70 thousand . Daily. Yeah.
Again, I will not go into details (if anyone is interested - ask in comments). I’d better tell you how we manage to shovel these daily seventy thousand samples.
Well, many people know that our family talisman is a woodpecker. And for good reason. In the good old days, our virlab worked like that. Like on a conveyor belt, viruses sat and “hollowed”. Incidentally, a very complex, tedious and respected profession! And I, too, has been woodpecking for many years! However, the times are not the same. And it’s clear that with the current flow, “fucking” is simply unrealistic, economically inefficient, and just plain stupid. We have been on combat duty for many years now ... auto woodpeckers ! The human factor gets only the most intellectual work - to disassemble the most complex samples, to investigate botnets, to ensure that auto-woodpeckers do not falsify, and, of course, to train and develop them in every possible way.
In general, there are several sources through which we obtain samples of malvari for analysis: self-propelled guns (“stick” to special traps),submissions (forwarded by users), exchange of collections with other antivirus companies and our cloud service KSN ( video , details ). Moreover, on the contribution to protecting users, the last source now occupies a leading position. Here is an example of KSN and let's see how our automation of processing malvari works.
KSN participating computers (and now there are more than 50 million of them ) send statistics (non-personalized!) About the work of our products to the cloud. There is information about trapped malware and infected sites, as well as a lot of useful information for detecting a new malware, for example, suspicious program behavior, hashes of downloaded files, and more.
Here, for example, the user launches a previously unknown file. Local antivirus scans it with all available tools - cleanly. We ask the cloud - there is no data. Ok - give the go-ahead for launch. And then it turns out that he somehow strangely registers himself in the registry, tries to access system services, establishes suspicious connections, has a double extension (jpg.exe) or something else. The signal arrives at KSN, where the system automatically calculates the reputation of the file (the weight of all attributes and actions) and makes a decision about the detection. As a result, the “fas” command is sent to the protected computer, the file is blocked, and its actions are rolled back. Of course, the more messages about the same file from different computers, the higher the processing priority and the higher the accuracy and criticality of the verdict. Appear such a file on other computers,
Another example.
Several users downloaded the file at the same link. But each time the file has a different hash. It smells like polymorphism! KSN begins to spin up the case and sees that, for example, the site was registered just a couple of days ago, some iframe was hanging on it or infected files were sent from it before (well, there are a lot of different signs). And again, the cloud calculates the reputation and sends a command to block both the file itself and access to the site.
Important: thanks to this approach, on average, only 40 seconds pass between a detection and a verdict !
But the work of our woodpeckers does not end there.Another system is already downloading the same suspicious file from the network and transferring it to the automatic processor for analysis. There is generally a whole bunch of all kinds of patented and not yet patented technologies, therefore I will not dig deeper. This handler develops and tests updates familiar to everyone and puts it on servers for downloads.
Something like that. I remember about 8 years ago, competitors, envious, were delighted, how do we manage to carry out such a huge work with such small resources? Automation however! And only competent automation can cope with this crazy stream of malvari! By the way, although the scope of work is constantly expanding and deepening, the number of employees in the virlab has not changed for a year now.
This raises a logical question.But how do small anti-virus companies survive ? It is known that keeping a good virlab is not only a matter of money, but also of brains. Where do they find the resources to stay afloat, with almost no money spent on R&D?
Sensitive subject.
For several years now, the natural theft of the “Detect” has flourished in the antivirus industry . Instead of analyzing malvara, developing their expertise and inventing new technologies, some (and there are about a dozen of these “some”!) Companies simply spy on the results of others and stupidly add hash detection to their databases. Incompetent tests help themthat do not reflect the level of protection in real conditions. As a result, not the best software creeps up, the number of its installations is growing, and the overall level of security is falling. Honest companies lose their motivation for research, investment in R&D is falling, but sales of thieves are growing, and cyber villains are rejoicing.
Well, this is a topic for a separate story.