SIEM systems: are there any prospects for OpenSource?
I want to welcome everyone who reads this article.
Initially, I wanted to comment on the article “OSSIM Security Management System” in the “sandbox” ( habrahabr.ru/post/157183 ), but the comment was very large. The main question is what type of SIEM does OSSIM belong to?
Now many people are interested in SIEM, but the idea of them is formed mainly by beautiful marketing leaflets. In most cases, these leaflets answer rather poorly to the question “Why do we need SIEM?”, Paying more attention to advertising a specific “modern” SIEM product.
And what can now be considered a “modern” SIEM product? Let's see what SIEM systems generally are. At the moment, there are systems of 2 generations. 2nd generation in most cases is called "Next-Generation SIEM".
Let's see what the first generation SIEM consists of:
But here's what Next-Generation SIEM thinks:
As you can see, the second generation of SIEMs is characterized by a larger number of sources and they were able to respond to "abnormal" situations: for example, if the user suddenly changed activity (he used to simply browse pages using HTTP, but now he’s actively driving traffic out through other protocols, for example) - this is an occasion to generate an “event”. In addition, Next-Generation SIEMs are able to analyze traffic passing through the network without using additional hardware or software (by transferring a free SIEM server network card to “inaudible” mode), monitor application activity - and this, in addition to the remaining main function, “collecting logs” from sources ”and“ analysis of events ”. Next-Generation SIEMs can also track virtual infrastructures, which the first generation SIEMs do not always do well.
And the analysis itself has become more intelligent - the number of "crimes" that need to be examined by a specialist, when using the second generation SIEM, decreases by about 25-30%, this is achieved due to the fact that some statistics of routine operations have already been accumulated, some trigger threshold for “atypical” operations (say, user1 usually edits and sends documents via smtp, but if he suddenly starts to use a different protocol for sending, an event that needs attention will be generated immediately And only when overcoming a certain frequency threshold (or amount) of such events. Of course, recorded data network activity, etc.).
But! Many managers claim that if you have SIEM% siem_name%, then there is no need to install DLP, IDS, vulnerability scanners, etc. In fact, this is not so. Next-Generation SIEM can track any anomalies in the network stream, but it cannot conduct a normal analysis. SIEM, in fact, is useless without other security systems. The main advantage of SIEM - the collection, storage and analysis of logs - will be reduced to 0 without the sources of these logs.
A frequent phrase in the leaflets “SIEM% siem_name% - ease and speed of implementation, a minimum of false positives, does not require reconfiguration of existing security features ...”. This is not true. You can, of course, get by with a minimum of reconfigurations - just redirect the entire stream of events from devices and security systems to SIEM. On a properly configured SIEM, this will not cause a significant increase in the number of false positives, but it will seriously load the database server (and cause the database to grow). Which in the end will result in an increase in DB maintenance time and, possibly, in missing some really important security incidents. Therefore, it will be necessary to think about what exactly to send to SIEM from existing devices, and what to leave at the mercy of the existing security system. Example - you can redirect all the logs of the antivirus installed on the user's PC, directly on SIEM - including database update events - but do you need it? Especially when you consider that some sources can generate the same type or recurring events. Of course, in most cases, SIEM provides for the possibility of combining them, but this again causes unnecessary load on the SIEM server.
Particular attention should be paid to the ability to collect flow-streams (NetFlow, sFlow, etc.) - a useful thing for cutting off unnecessary information about traffic on the network and, at the same time, obtaining additional useful information about the status of the network received directly from the network devices.
Just like there is no easy implementation. Before starting, you need to carry out a lot of analytical work, determine which events are important, which are not, etc.
Ultimately - the question is - who needs systems of this class? These systems, in the opinion of the author, are necessary for those who have a large network infrastructure and who want to at least somehow organize events and be aware of incidents. But at the same time, it’s ready for high costs, the payback of such systems is not instantaneous, the benefits, at first glance, are not obvious. In addition, these systems are demanding of hardware. Although there was a successful phrase in one leaflet: "SIEM allows CIO to explain IT problems in business language." In addition, these systems are demanding on the hardware.
The phrase is not without reason: reports created by modern SIEM, not only that in various formats, but also customizable to the needs of a particular organization, often allow you to get all the necessary data on two or three sheets of A4 format, presented in the form of clear and visual graphs or digits.
In conclusion, I want to summarize: SIEM (or products that call themselves such) - a lot. But, since they are affordable only for large customers, they will pay attention to the leaders. There are only 3 absolute leaders today: ArcSight ESM, QRadar SIEM (from IBM Q1 Labs), McAfee ESM (formerly NitroView ESM). They can also include a rather interesting development of LogRhythm and NetIQ - according to Gartner.
In light of this, it is not clear what prospects for open-source SIEM systems, which, for the most part, belong to the first generation, in my opinion, this is not a product that can easily be replaced by open-source without the risk of losing benefits in the form of regular updates and qualified support. On the other hand, there is a positive example of OpenBSD ...
And what do you think, are there any prospects for open source SIEMs now?
PS> Until recently, I thought that there were no simple and understandable articles on SIEM, it was gratifying to see that this situation was gradually being corrected. On Securitylab.ru, in the “Analytics” section there is a series of articles on SIEM, in my opinion, articles from the category of mandatory reading before work. They well explain the theory of the operation of systems of this class.
Initially, I wanted to comment on the article “OSSIM Security Management System” in the “sandbox” ( habrahabr.ru/post/157183 ), but the comment was very large. The main question is what type of SIEM does OSSIM belong to?
Now many people are interested in SIEM, but the idea of them is formed mainly by beautiful marketing leaflets. In most cases, these leaflets answer rather poorly to the question “Why do we need SIEM?”, Paying more attention to advertising a specific “modern” SIEM product.
And what can now be considered a “modern” SIEM product? Let's see what SIEM systems generally are. At the moment, there are systems of 2 generations. 2nd generation in most cases is called "Next-Generation SIEM".
Let's see what the first generation SIEM consists of:
But here's what Next-Generation SIEM thinks:
As you can see, the second generation of SIEMs is characterized by a larger number of sources and they were able to respond to "abnormal" situations: for example, if the user suddenly changed activity (he used to simply browse pages using HTTP, but now he’s actively driving traffic out through other protocols, for example) - this is an occasion to generate an “event”. In addition, Next-Generation SIEMs are able to analyze traffic passing through the network without using additional hardware or software (by transferring a free SIEM server network card to “inaudible” mode), monitor application activity - and this, in addition to the remaining main function, “collecting logs” from sources ”and“ analysis of events ”. Next-Generation SIEMs can also track virtual infrastructures, which the first generation SIEMs do not always do well.
And the analysis itself has become more intelligent - the number of "crimes" that need to be examined by a specialist, when using the second generation SIEM, decreases by about 25-30%, this is achieved due to the fact that some statistics of routine operations have already been accumulated, some trigger threshold for “atypical” operations (say, user1 usually edits and sends documents via smtp, but if he suddenly starts to use a different protocol for sending, an event that needs attention will be generated immediately And only when overcoming a certain frequency threshold (or amount) of such events. Of course, recorded data network activity, etc.).
But! Many managers claim that if you have SIEM% siem_name%, then there is no need to install DLP, IDS, vulnerability scanners, etc. In fact, this is not so. Next-Generation SIEM can track any anomalies in the network stream, but it cannot conduct a normal analysis. SIEM, in fact, is useless without other security systems. The main advantage of SIEM - the collection, storage and analysis of logs - will be reduced to 0 without the sources of these logs.
A frequent phrase in the leaflets “SIEM% siem_name% - ease and speed of implementation, a minimum of false positives, does not require reconfiguration of existing security features ...”. This is not true. You can, of course, get by with a minimum of reconfigurations - just redirect the entire stream of events from devices and security systems to SIEM. On a properly configured SIEM, this will not cause a significant increase in the number of false positives, but it will seriously load the database server (and cause the database to grow). Which in the end will result in an increase in DB maintenance time and, possibly, in missing some really important security incidents. Therefore, it will be necessary to think about what exactly to send to SIEM from existing devices, and what to leave at the mercy of the existing security system. Example - you can redirect all the logs of the antivirus installed on the user's PC, directly on SIEM - including database update events - but do you need it? Especially when you consider that some sources can generate the same type or recurring events. Of course, in most cases, SIEM provides for the possibility of combining them, but this again causes unnecessary load on the SIEM server.
Particular attention should be paid to the ability to collect flow-streams (NetFlow, sFlow, etc.) - a useful thing for cutting off unnecessary information about traffic on the network and, at the same time, obtaining additional useful information about the status of the network received directly from the network devices.
Just like there is no easy implementation. Before starting, you need to carry out a lot of analytical work, determine which events are important, which are not, etc.
Ultimately - the question is - who needs systems of this class? These systems, in the opinion of the author, are necessary for those who have a large network infrastructure and who want to at least somehow organize events and be aware of incidents. But at the same time, it’s ready for high costs, the payback of such systems is not instantaneous, the benefits, at first glance, are not obvious. In addition, these systems are demanding of hardware. Although there was a successful phrase in one leaflet: "SIEM allows CIO to explain IT problems in business language." In addition, these systems are demanding on the hardware.
The phrase is not without reason: reports created by modern SIEM, not only that in various formats, but also customizable to the needs of a particular organization, often allow you to get all the necessary data on two or three sheets of A4 format, presented in the form of clear and visual graphs or digits.
In conclusion, I want to summarize: SIEM (or products that call themselves such) - a lot. But, since they are affordable only for large customers, they will pay attention to the leaders. There are only 3 absolute leaders today: ArcSight ESM, QRadar SIEM (from IBM Q1 Labs), McAfee ESM (formerly NitroView ESM). They can also include a rather interesting development of LogRhythm and NetIQ - according to Gartner.
In light of this, it is not clear what prospects for open-source SIEM systems, which, for the most part, belong to the first generation, in my opinion, this is not a product that can easily be replaced by open-source without the risk of losing benefits in the form of regular updates and qualified support. On the other hand, there is a positive example of OpenBSD ...
And what do you think, are there any prospects for open source SIEMs now?
PS> Until recently, I thought that there were no simple and understandable articles on SIEM, it was gratifying to see that this situation was gradually being corrected. On Securitylab.ru, in the “Analytics” section there is a series of articles on SIEM, in my opinion, articles from the category of mandatory reading before work. They well explain the theory of the operation of systems of this class.