OSSIM - We use a comprehensive Open Source security management system
This article is inspired by the article " OSSIM - Deploying an Integrated Open Source Security Management System ". I will not repeat myself and describe the process of installing the system. I just want to make some clarifications and clarifications related to the practical experience of using OSSIM.
In any instruction, you will read that "definitely on a virtual machine." My answer is: "definitely for iron." Both answers are correct. Why?
The first answer is correct, since OSSIM is installed “bundled” with the operating system and the disk with the image may not contain the drivers needed for your hardware. If you come across this, then this will be your problem, which you will have to solve on your own. Installing on a virtual machine provides some versatility and no problems. In addition, there is no guarantee that after the update the problems do not have to be solved again. There are official commercial "hardware implementations" that are sold as appliance and, accordingly, are supported, but we look at Open Source OSSIM and here all the hardware problems are your problems.
The second answer is correct, since any hypervisor is a brake. Advocates of universal virtualization can blow trumpets, drumming, waving flags, but hypervisors drastically reduce the performance of the guest system. This can usually be neglected, but OSSIM loves speed. As a result, a bare-core mini-computer with a 4-core atom, 16GB DDR3 and 128GB SATA3 SSD makes a virtual machine on the HP dl380 easy. You will have to give much more resources to the virtual machine than the physical one would need. And it will cost more. As for the drivers, during installation you will be prompted to insert a USB flash drive with drivers, if necessary.
Important.On an iron machine, the installer will only start from a bootable CD. If you upload the image to a USB flash drive, OSSIM will not install, but you can install Debian.
Very important. Never attempt to install OSSIM as a “second system” on a working computer. The installer believes that he is put on a bare machine and does not ask idiotic questions "how to partition a disk?" First of all, without asking, he will overwrite the partition table and format the disk.
Installing OSSIM does not take “15 minutes” at all. Much longer. At the last stage, you will even think that the installer is frozen and everything is gone. Wait. He works.
The speed of the disk subsystem. Use SSD correctly. Volumes are not important. 100GB is enough. But speed is very important. There are two reasons for this. Firstly, text logs that are sent to syslog and then read by OSSIM plugins. Secondly, a database that runs on the same machine. SSD dramatically improves system performance.
The number of processor cores. The performance of each core is not as important as their number. OSSIM performs very simple operations, but there are many at once and they can be performed in parallel. It is also important for its Network IDS (Suricata), as you know, it is multi-threaded.
The amount of RAM. Less critical, but it’s better if nothing is pushed into the page file.
A suitable configuration for a virtual machine based on a daily stream of 2 million events in the logs and 10 terabytes of traffic on a "illegible" interface: 8 cores, 16GB. But it is back to back. Resources will be eaten to the eyeballs.
And yes, of course, the illegible interface on which you intend to drop traffic collected for the analysis of the meerkat from all the wilans of your network from the span port of your switch should be 10Gbs. Otherwise, it will simply choke.
If you intend to manage the OSSIM server from a Windows machine, you will need putty and WinSCP. Well, or FAR if you love him. It's nicer to drive from a machine with Ubuntu. At least, you don’t need to check every time to see if you accidentally wrote line endings as a CRLF in some script.
After installation, you will not be able to connect to the server via SSH from a remote machine. In Debian 8, the default option in the sshd configuration is `PermitRootLogin without-password`, it must be changed in` / etc / ssh / ssh_config` to `PermitRootLogin yes`.
Another important thing is the correct configuration of the time zone of the plugins. The fact is that even when all log sources are in the same time zone, they do not necessarily use the same local time. For example, System Center Configuration Manager believes that it is reasonable to store time in the database in UTC. And if you have a plugin that reads new events from its database (and I have one), then you need to consider that they are not recorded in local time.
The time zone for plugins is set in two places: firstly, the default time zone for all plugins is set in `/ etc / ossim / agent / config.cfg`, and secondly, it can be redefined in the config file of each individual plugin. Defining a time zone means an instruction for the plugin generator to “assume that the data comes from such a time zone and convert the time to ours.” In this case, “ours” is the local time of the server. In fact, the time will be written to the database in UTC, but there is a separate field in which the offset of the "local system" will be written.
Interesting things begin when you have two sources of the same type in different time zones. For example, you take cisco-asa router logs from different branches. In this case, you will need to process them with different plugins, in the config files of which you specify a different parameter `tzone =`. This parameter is set in the `[default]` section. I am writing about this because you will not find this in the documentation, I do not know why. POSIX format: `tzone = Europe / Moscow`.
My answer: the "native" tool AlienVault HIDS, aka OSSEC. Of course I will explain this briefly.
Theoretically, you can use any of the many agents to send a Windows event log to syslog or use WMI. OSSIM has a WMI client and a standard plugin for reading Windows logs. There is also a standard plugin for the SNARE agent, but if you are dealing with Russian Windows, then it will not be useful to you. The thing is that SNARE sends data from Russian Windows in cp1251 encoding, and the standard parser for SNARE is written under cp1252. Have to edit the regular season.
But the most interesting thing will begin when you collect logs from "multilingual" systems. For example, you have Russian Windows on workstations, and English on your servers. And how to analyze it now? This is actually a common problem for all SIEMs. They solve it in different ways. For example, ArcSight uses a rather sophisticated system for collecting Windows logs, which allows you to collect logs only in English, regardless of the localization of the system. OSSEC uses a very simple technique. When installing the agent on Windows, a csv file with a table of string messages of the Windows log in English is written to its working directory. Therefore, the “essential part” of the message, which is needed for analysis, always comes in one language, regardless of the localization of the system. But the "data" comes "in the original language." It is quite convenient.
In addition, the standard plugin for the OSSEC agent is very well written. He carefully parses events by type. If you use another way to collect logs, you will have to sweat a lot on arranging events on the shelves. Well, and finally, OSSEC is not just a “log forwarding” - it is really a Host IDS and not even a bad one. Not for nothing TrendMicro decided to use this engine in its "advanced" antivirus. And yes, OSSEC is reliable as a control shot. You can safely install the agent on any critical system.
And further. Rabbits are not only valuable fur, but also 3-4 kilograms of dietary meat. OSSEC is not a log collector, it is SIEM in itself. In OSSIM, it does not transmit the Windows log at all, but its own alert.log, which is formed on the basis of preliminary processing of events received from agents. Here you will find, for example: events of change of controlled files, or such aggregate events as “multiple error” or “multiple change of the checksum of the key in the registry”. It is more useful than just a collector. OSSEC is very widespread on the Internet, it is very often used to protect web servers, so the community is large and active.
Of course, you can play around with other ways to collect Windows logs. It is interesting.
On a Windows machine, the agent is installed with some default configuration file. This file is here: `/ usr / share / ossec-generator / installer / ossec.conf`. OSSEC supports downloading configurations from the server. The file `/ var / ossec / etc / shared / agent.conf` is used for this purpose. By default, it is absent. This file can be created from the web interface of the OSSIM console (Environment - Detection - Agents - agent.conf). Or just write in a text editor.
It should contain configuration directives in XML format that will be merged with the local agent configuration file. Directive blocks for different agents can be marked to apply only to the right agents. Marking by OS type, agent name, profile name is allowed (in the local agent configuration, in this case, the name of his profile must be indicated).
In this way, agent configurations can be changed centrally and pointwise. The term "merged" requires clarification. What happens if the local section of the configuration file is overridden? The theory says that the local file is read first, then the server file, the rule read last last overrides all the previous ones. In practice, I have never tried such an intersection of configuration parameters. There was simply no need. I hope, when needed, it will work as I think.
Another important point. The template for the local agent configuration shows the method of connecting to the server: As you can see here the ip address. I don’t know why the authors preferred such a template. You can specify the FQDN then this section should look different:
However, this has not been done. This means that when changing the ip address of the server, all agents will fall off. Not a good idea. Of course, SIEM is not such a device on which the ip address is changed, but it is somehow unpleasant. Of course, you can change this section in the template, but most likely, the next time you update the template will be regenerated. It will be necessary to constantly monitor this. This problem does not bother me, because I do not have the habit of changing addresses.
On Windows, it is most convenient to install an agent from the OSSIM web console using the Automatic deployment button. But you will need the credentials of the local administrator of the target host (or domain). Installing from Group Policy or using SCCM is very difficult. The fact is that each installation file is individual for a particular host, because it contains a unique key for encrypting communication with the server. Such secrecy at the PCI DSS level. Sadness.
For Linux, agentless is possible, but this requires configuring SSH host connections. In my opinion, this is a bad idea. I prefer to put agents. In this case, a completely manual installation with the compilation of the agent on the target machine from the "raw". Version 2.8.2 is officially supported, but version 2.8.3 also works without problems. Actually there are packages for different systems, for example, for Debian. See here for more details.
Click the support button in the web console menu and get links. Required reading:
USM 5.x Plugins Management Guide
Customizing Correlation Directives or Cross Correlation Rules
Intrusion Detection in AlienVault
Policy Management Fundamentals
Using USM and OSSIM 5.1 with OTX - AlienVault
Assets, Groups & Networks
System Errors, Warnings and Suggestions
It seems to everyone to understand, but as it comes down to the matter, it turns out that people, for the most part, do not correctly understand the purpose of SIEM. First of all, this is not a remedy. And not even a “security management system,” excuse me the title of the article, but not the first one I started. In fact, it is a means of detecting successful security breaches. The standard scheme for building protection begins with the identification of possible attacks (actual threats) and how to carry them out. Then, technical and organizational defense measures are invented and implemented that are placed on possible “attack vectors”. And after all this, “controls” are set up, the purpose of which is to see that all your protective measures do not work. And SIEM belongs to this class.
This is what you need to understand. SIEM is designed to detect violations that have already occurred or bypass protective measures, both technical and organizational. This is something like a video surveillance system. First of all, you must determine by what signs you will detect these violations, how SIEM will identify these signs, where it will get the necessary information from. Do you have any “dead zones” that SIEM does not see. Situations may be the most unexpected.
OSSIM saves the suspicious packet in pcap format for analysis. Here it is:
What was that?
But it was like that. In one office, IT experts tested some kind of thing that was spinning in the tomcat application server environment. They installed this thing on the first server that came across, of course leaving the console entry with an empty password. By tested and forgotten. The first server turned out to be a Terminal Services server. And, as you know, tomcat worked under the system account. An angry hacker phishing slipped a trojan onto the computer of one of the employees and looked at what was useful on the network. Found this forgotten web server and was delighted. I poured a library on it to support console commands. That moment of testing this library is shown in the picture. The net user command was run, and suricata booed when it saw the contents of the output in http-responce. While sorting out what the garbage,
The situation is very vital. People always make mistakes. Not only IT specialists, but also hackers.
Or here's a simpler story. Here is the alarm page completely fresh.
This is the so-called OTX Pulse. Here are its details:
What was it?
And this is the browser of one of the employees, when he looked at one of the forum threads on bankir.ru, rushed to the URL owqkq.ne1t3v8.top, and this is one of the parking pages using the Angler Exploit Toolkit. Also an interesting story.
To avoid “dead zones”, you must first place the HIDS agents (OSSEC) on ALL hosts on the network. Then, configure the transmission to the “illegible” OSSIM interface of ALL traffic on your internal network so that it is processed by NIDS (Suricata). It is imperative to configure and run periodic vulnerability scanning with the OpenVAS scanner of at least all DMZ nodes. It is important. Observations show that even for a very small company, 20-50 enemy scanners pass every day. It will be better if you discover a vulnerability earlier than they. I am not exaggerating, just downplaying. Here is a real snippet of the daily report:
And this is only the part that relates to the villains known to the community, and there is also another part of the report with a list of unknown villains. Of course, most of them do not push much. For example: like this (this is a clipping from the same report): But there are also very intrusive personalities. You will also need firewall logs. This is the minimum that is in any office. And then there are internal things: access logs to the DBMS and individual business applications, web server logs, etc. etc. You can collect anything you want. If there is no standard plugin for parsing logs, it is very easy to write your own. The truth is easy. Developing your first plugin will not take you more than a couple of days. And the development of the second will take a couple of hours.
As for the control of violations of organizational protective measures, for example, it is forbidden in your office to make changes to the configuration of the production environment without the approval of the information security service. OSSEC will tell you about any configuration change and you can check if it was agreed or not. The most interesting part begins when it turns out that no one has made this change.
I will not give advice on exactly which control rules should be applied. Each village has its own traditions. If you have questions, ask.
Important.AlienVault OSSIM is not only SIEM in the classical sense. This is an entire bookcase that includes: Host IDS, Network IDS, Wireless IDS, Volnurability Scaner, NetFlow Collector. That is, this is a complete classic set of “controls” for organizing video surveillance of a company’s network.
If it seems to you that something is working wrong, you should start looking at the logs. Where?
OSSEC logs and errors:
`/var/ossec/logs/ossec.log`- here you will see OSSEC errors. The most common mistake is a loss of communication with the agent, it happens when there are some fatal communication failures over the network, or after reinstalling the agent. Rare, but with a large number of agents it may not be so rare. If you see on the console that an agent is not active, and you know that the computer is turned on, then you are here. In the log, the error looks like `ERROR: Duplicated counter for 'agent-name'`. It is eliminated simply. In the web console (environment - detection - agents) or in the file `/ var / ossec / etc / client.keys` we look for an agent with this name and look at its number (the number in the leftmost column). Then we climb into the `/ var / ossec / queue / rids` directory and delete the file with the name - agent number in it. We go SSH to the server console, exit the command line and do `/etc/init.d/ossec restart`. All things. Honestly, I have not seen other errors with OSSEC.
`/var/ossec/logs/alerts/alerts.log` is the log in which OSSEC collects chewed events from agents, this is where OSSIM reads events and processes them with an OSSEC plugin. Here you can see what is falling there and how it looks.
The rest of the logs in the traditional `/ var / log` of them are useful` / var / log / alienvault / agent / agent.log` and in the same place `agent_error.log`. They can come in handy when debugging your own plugins. Note that on the working system the size of `agent.log` is in gigabytes.
I have never seen any serious errors in the operation of OSSIM. Once after the next system update, the regular backup stopped working. It turned out they incorrectly assigned the rights to one of the files. This was fixed literally the next day. Information on such issues takes place on the community forum. One more time, after the next update, the following updates stopped working. This time a notification came that the update that was posted four hours ago contains a bug and that it needs to be fixed in this way, but the corrected update was posted at such and such a time. By the way, if the update by standard means (from the web or ssh console) crashes and does not work, it can be performed by commands from the server console.
Perhaps this is all I wanted to say. And most importantly. This is open source. Whatever you want, Python is with you. And if you do not want him to be with you, look at the commercial version - Python with them. And here we come to the last question.
My answer is: if you need SIEM for the needs of the company, then Enterprise. You will never have the human and technical resources to bring the Open Source version to the Enterprise level, of course, if your company does not set as its goal the development of its own SIEM. But, if this is not your business, take care of your own. Unfortunately, any Enterprise SIEM is very expensive. But they are so expensive not because they have some unusual correlation server engines, but because they have invested huge expenses in creating event correlation rule libraries, analyzing the “signatures” of complex attacks and how to detect them, and debugging these methods. This is a job that you cannot do on your own. I don’t want to talk about the advantages of commercial products here; to me these advantages seem obvious and not worth discussing.
So why am I wasting time on this Open Source? First of all, I like him. Of all such Open Source projects, this one is successful. Secondly, AlienVault’s website says something like “we want to make security accessible to everyone.” Not bad idea. I support. Too many companies simply can not afford to buy a commercial product of this class. I know that there is a whole army of freelance system administrators, let's call them serving small companies. It makes sense for them to look this way. It's not for nothing that at the very beginning I mentioned a mini-computer. Little SIEM is a very real thing.
Enterprise - enterprise, and community - community. Land - to the peasants, factories - to workers, money - to bank employees. “Happiness to all, and that enough for everyone” (A. and B. Strugatsky. “Picnic on the sidelines”).
Sources used:
https://www.alienvault.com/documentation
https://alienvault.ru/open-threat-exchange/
http://ossec.github.io/docs/
http://suricata-ids.org/ docs /
What to bet on? Iron or virtual?
In any instruction, you will read that "definitely on a virtual machine." My answer is: "definitely for iron." Both answers are correct. Why?
The first answer is correct, since OSSIM is installed “bundled” with the operating system and the disk with the image may not contain the drivers needed for your hardware. If you come across this, then this will be your problem, which you will have to solve on your own. Installing on a virtual machine provides some versatility and no problems. In addition, there is no guarantee that after the update the problems do not have to be solved again. There are official commercial "hardware implementations" that are sold as appliance and, accordingly, are supported, but we look at Open Source OSSIM and here all the hardware problems are your problems.
The second answer is correct, since any hypervisor is a brake. Advocates of universal virtualization can blow trumpets, drumming, waving flags, but hypervisors drastically reduce the performance of the guest system. This can usually be neglected, but OSSIM loves speed. As a result, a bare-core mini-computer with a 4-core atom, 16GB DDR3 and 128GB SATA3 SSD makes a virtual machine on the HP dl380 easy. You will have to give much more resources to the virtual machine than the physical one would need. And it will cost more. As for the drivers, during installation you will be prompted to insert a USB flash drive with drivers, if necessary.
Important.On an iron machine, the installer will only start from a bootable CD. If you upload the image to a USB flash drive, OSSIM will not install, but you can install Debian.
Very important. Never attempt to install OSSIM as a “second system” on a working computer. The installer believes that he is put on a bare machine and does not ask idiotic questions "how to partition a disk?" First of all, without asking, he will overwrite the partition table and format the disk.
Installing OSSIM does not take “15 minutes” at all. Much longer. At the last stage, you will even think that the installer is frozen and everything is gone. Wait. He works.
The most important resources for OSSIM in decreasing order of importance
The speed of the disk subsystem. Use SSD correctly. Volumes are not important. 100GB is enough. But speed is very important. There are two reasons for this. Firstly, text logs that are sent to syslog and then read by OSSIM plugins. Secondly, a database that runs on the same machine. SSD dramatically improves system performance.
The number of processor cores. The performance of each core is not as important as their number. OSSIM performs very simple operations, but there are many at once and they can be performed in parallel. It is also important for its Network IDS (Suricata), as you know, it is multi-threaded.
The amount of RAM. Less critical, but it’s better if nothing is pushed into the page file.
A suitable configuration for a virtual machine based on a daily stream of 2 million events in the logs and 10 terabytes of traffic on a "illegible" interface: 8 cores, 16GB. But it is back to back. Resources will be eaten to the eyeballs.
And yes, of course, the illegible interface on which you intend to drop traffic collected for the analysis of the meerkat from all the wilans of your network from the span port of your switch should be 10Gbs. Otherwise, it will simply choke.
If you intend to manage the OSSIM server from a Windows machine, you will need putty and WinSCP. Well, or FAR if you love him. It's nicer to drive from a machine with Ubuntu. At least, you don’t need to check every time to see if you accidentally wrote line endings as a CRLF in some script.
After installation, you will not be able to connect to the server via SSH from a remote machine. In Debian 8, the default option in the sshd configuration is `PermitRootLogin without-password`, it must be changed in` / etc / ssh / ssh_config` to `PermitRootLogin yes`.
Time zone
Another important thing is the correct configuration of the time zone of the plugins. The fact is that even when all log sources are in the same time zone, they do not necessarily use the same local time. For example, System Center Configuration Manager believes that it is reasonable to store time in the database in UTC. And if you have a plugin that reads new events from its database (and I have one), then you need to consider that they are not recorded in local time.
The time zone for plugins is set in two places: firstly, the default time zone for all plugins is set in `/ etc / ossim / agent / config.cfg`, and secondly, it can be redefined in the config file of each individual plugin. Defining a time zone means an instruction for the plugin generator to “assume that the data comes from such a time zone and convert the time to ours.” In this case, “ours” is the local time of the server. In fact, the time will be written to the database in UTC, but there is a separate field in which the offset of the "local system" will be written.
Interesting things begin when you have two sources of the same type in different time zones. For example, you take cisco-asa router logs from different branches. In this case, you will need to process them with different plugins, in the config files of which you specify a different parameter `tzone =`. This parameter is set in the `[default]` section. I am writing about this because you will not find this in the documentation, I do not know why. POSIX format: `tzone = Europe / Moscow`.
What is the best way to collect Windows logs?
My answer: the "native" tool AlienVault HIDS, aka OSSEC. Of course I will explain this briefly.
Theoretically, you can use any of the many agents to send a Windows event log to syslog or use WMI. OSSIM has a WMI client and a standard plugin for reading Windows logs. There is also a standard plugin for the SNARE agent, but if you are dealing with Russian Windows, then it will not be useful to you. The thing is that SNARE sends data from Russian Windows in cp1251 encoding, and the standard parser for SNARE is written under cp1252. Have to edit the regular season.
But the most interesting thing will begin when you collect logs from "multilingual" systems. For example, you have Russian Windows on workstations, and English on your servers. And how to analyze it now? This is actually a common problem for all SIEMs. They solve it in different ways. For example, ArcSight uses a rather sophisticated system for collecting Windows logs, which allows you to collect logs only in English, regardless of the localization of the system. OSSEC uses a very simple technique. When installing the agent on Windows, a csv file with a table of string messages of the Windows log in English is written to its working directory. Therefore, the “essential part” of the message, which is needed for analysis, always comes in one language, regardless of the localization of the system. But the "data" comes "in the original language." It is quite convenient.
In addition, the standard plugin for the OSSEC agent is very well written. He carefully parses events by type. If you use another way to collect logs, you will have to sweat a lot on arranging events on the shelves. Well, and finally, OSSEC is not just a “log forwarding” - it is really a Host IDS and not even a bad one. Not for nothing TrendMicro decided to use this engine in its "advanced" antivirus. And yes, OSSEC is reliable as a control shot. You can safely install the agent on any critical system.
And further. Rabbits are not only valuable fur, but also 3-4 kilograms of dietary meat. OSSEC is not a log collector, it is SIEM in itself. In OSSIM, it does not transmit the Windows log at all, but its own alert.log, which is formed on the basis of preliminary processing of events received from agents. Here you will find, for example: events of change of controlled files, or such aggregate events as “multiple error” or “multiple change of the checksum of the key in the registry”. It is more useful than just a collector. OSSEC is very widespread on the Internet, it is very often used to protect web servers, so the community is large and active.
Of course, you can play around with other ways to collect Windows logs. It is interesting.
Something About OSSEC Agent Configuration
On a Windows machine, the agent is installed with some default configuration file. This file is here: `/ usr / share / ossec-generator / installer / ossec.conf`. OSSEC supports downloading configurations from the server. The file `/ var / ossec / etc / shared / agent.conf` is used for this purpose. By default, it is absent. This file can be created from the web interface of the OSSIM console (Environment - Detection - Agents - agent.conf). Or just write in a text editor.
It should contain configuration directives in XML format that will be merged with the local agent configuration file. Directive blocks for different agents can be marked to apply only to the right agents. Marking by OS type, agent name, profile name is allowed (in the local agent configuration, in this case, the name of his profile must be indicated).
настройки
настройки
настройки
настройки
In this way, agent configurations can be changed centrally and pointwise. The term "merged" requires clarification. What happens if the local section of the configuration file is overridden? The theory says that the local file is read first, then the server file, the rule read last last overrides all the previous ones. In practice, I have never tried such an intersection of configuration parameters. There was simply no need. I hope, when needed, it will work as I think.
Another important point. The template for the local agent configuration shows the method of connecting to the server: As you can see here the ip address. I don’t know why the authors preferred such a template. You can specify the FQDN then this section should look different:
172.17.2.10
120
240
fqdn
...
However, this has not been done. This means that when changing the ip address of the server, all agents will fall off. Not a good idea. Of course, SIEM is not such a device on which the ip address is changed, but it is somehow unpleasant. Of course, you can change this section in the template, but most likely, the next time you update the template will be regenerated. It will be necessary to constantly monitor this. This problem does not bother me, because I do not have the habit of changing addresses.
Host Agent Installation Considerations
On Windows, it is most convenient to install an agent from the OSSIM web console using the Automatic deployment button. But you will need the credentials of the local administrator of the target host (or domain). Installing from Group Policy or using SCCM is very difficult. The fact is that each installation file is individual for a particular host, because it contains a unique key for encrypting communication with the server. Such secrecy at the PCI DSS level. Sadness.
For Linux, agentless is possible, but this requires configuring SSH host connections. In my opinion, this is a bad idea. I prefer to put agents. In this case, a completely manual installation with the compilation of the agent on the target machine from the "raw". Version 2.8.2 is officially supported, but version 2.8.3 also works without problems. Actually there are packages for different systems, for example, for Debian. See here for more details.
Where to look for documentation and what is better to read
Click the support button in the web console menu and get links. Required reading:
USM 5.x Plugins Management Guide
Customizing Correlation Directives or Cross Correlation Rules
Intrusion Detection in AlienVault
Policy Management Fundamentals
Using USM and OSSIM 5.1 with OTX - AlienVault
Assets, Groups & Networks
System Errors, Warnings and Suggestions
Why do we need SIEM?
It seems to everyone to understand, but as it comes down to the matter, it turns out that people, for the most part, do not correctly understand the purpose of SIEM. First of all, this is not a remedy. And not even a “security management system,” excuse me the title of the article, but not the first one I started. In fact, it is a means of detecting successful security breaches. The standard scheme for building protection begins with the identification of possible attacks (actual threats) and how to carry them out. Then, technical and organizational defense measures are invented and implemented that are placed on possible “attack vectors”. And after all this, “controls” are set up, the purpose of which is to see that all your protective measures do not work. And SIEM belongs to this class.
This is what you need to understand. SIEM is designed to detect violations that have already occurred or bypass protective measures, both technical and organizational. This is something like a video surveillance system. First of all, you must determine by what signs you will detect these violations, how SIEM will identify these signs, where it will get the necessary information from. Do you have any “dead zones” that SIEM does not see. Situations may be the most unexpected.
OSSIM saves the suspicious packet in pcap format for analysis. Here it is:
What was that?
But it was like that. In one office, IT experts tested some kind of thing that was spinning in the tomcat application server environment. They installed this thing on the first server that came across, of course leaving the console entry with an empty password. By tested and forgotten. The first server turned out to be a Terminal Services server. And, as you know, tomcat worked under the system account. An angry hacker phishing slipped a trojan onto the computer of one of the employees and looked at what was useful on the network. Found this forgotten web server and was delighted. I poured a library on it to support console commands. That moment of testing this library is shown in the picture. The net user command was run, and suricata booed when it saw the contents of the output in http-responce. While sorting out what the garbage,
The situation is very vital. People always make mistakes. Not only IT specialists, but also hackers.
Or here's a simpler story. Here is the alarm page completely fresh.
This is the so-called OTX Pulse. Here are its details:
What was it?
And this is the browser of one of the employees, when he looked at one of the forum threads on bankir.ru, rushed to the URL owqkq.ne1t3v8.top, and this is one of the parking pages using the Angler Exploit Toolkit. Also an interesting story.
To avoid “dead zones”, you must first place the HIDS agents (OSSEC) on ALL hosts on the network. Then, configure the transmission to the “illegible” OSSIM interface of ALL traffic on your internal network so that it is processed by NIDS (Suricata). It is imperative to configure and run periodic vulnerability scanning with the OpenVAS scanner of at least all DMZ nodes. It is important. Observations show that even for a very small company, 20-50 enemy scanners pass every day. It will be better if you discover a vulnerability earlier than they. I am not exaggerating, just downplaying. Here is a real snippet of the daily report:
Коммуникации с известными вредоносными хостами за период 2016:02:18 - 2016:02:19
Время Внешний IP Место Репутация хоста
2016-02-18 09:22:46 180.97.106.37 Nanjing Malicious Host
2016-02-18 09:38:37 216.218.206.123 Fremont Malicious Host
2016-02-18 09:52:57 85.25.214.226 Germany Scanning Host
2016-02-18 10:08:11 146.185.250.105 Saint Petersburg Malicious Host
2016-02-18 10:22:54 178.62.14.193 London Malicious Host
2016-02-18 10:23:24 94.102.49.79 Netherlands Malicious Host
2016-02-18 10:47:52 195.88.209.6 Moscow Malicious Host
2016-02-18 10:53:29 222.186.34.177 Nanjing Malicious Host
2016-02-18 11:07:48 71.6.135.131 San Diego Malicious Host
2016-02-18 11:58:17 193.105.134.220 Sweden Malicious Host
2016-02-18 11:58:51 62.210.206.219 France Malicious Host
2016-02-18 12:28:13 193.109.69.150 Russia Malicious Host
2016-02-18 12:43:40 216.218.206.96 Fremont Malicious Host
2016-02-18 13:08:50 209.126.124.67 St Louis Malicious Host
2016-02-18 13:53:19 178.33.17.241 France Malicious Host
2016-02-18 14:23:52 198.20.70.114 Chicago Malicious Host
2016-02-18 14:32:49 104.219.238.10 Rye Malicious Host Scanning Host
2016-02-18 14:38:38 198.23.112.119 Dallas Scanning Host
2016-02-18 15:02:58 198.20.69.98 Chicago Malicious Host
2016-02-18 15:03:29 64.125.239.136 United States Malicious Host
2016-02-18 15:28:35 162.248.74.2 Clarks Summit Malicious Host
2016-02-18 15:43:36 222.174.5.28 Jinan Malicious Host
2016-02-18 15:57:42 66.240.236.119 San Diego Malicious Host
2016-02-18 16:13:09 74.82.47.45 Fremont Malicious Host
2016-02-18 16:13:44 64.125.239.92 United States Malicious Host
2016-02-18 17:07:57 142.54.162.74 Kansas City Malicious Host
2016-02-18 17:22:41 64.125.239.107 United States Malicious Host
2016-02-18 17:58:54 23.239.66.99 United States Malicious Host
2016-02-18 18:07:50 61.216.2.14 Taiwan Malicious Host
2016-02-18 18:08:03 198.20.69.74 Chicago Malicious Host
2016-02-18 18:08:18 141.212.122.84 Ann Arbor Malicious Host
2016-02-18 18:08:18 141.212.122.81 Ann Arbor Malicious Host
2016-02-18 19:52:53 185.94.111.1 Russia Malicious Host
2016-02-18 19:58:27 162.244.35.24 United States Malicious Host
2016-02-18 20:23:00 162.244.35.22 United States Malicious Host
2016-02-18 20:23:37 89.248.160.192 Netherlands Malicious Host
2016-02-18 20:43:55 222.174.5.17 Jinan Malicious Host
2016-02-18 21:23:55 185.130.5.201 Republic of Lithuania Malicious Host
2016-02-18 21:47:39 92.60.184.34 Ukraine Scanning Host
2016-02-18 22:33:48 209.126.102.181 St Louis Malicious Host
2016-02-18 22:57:37 71.6.167.142 San Diego Malicious Host
2016-02-18 23:13:37 212.83.148.78 France Malicious Host
2016-02-19 00:07:54 185.130.5.240 Republic of Lithuania Scanning Host
2016-02-19 00:48:22 64.125.239.224 United States Malicious Host
2016-02-19 01:13:11 66.240.192.138 San Diego Malicious Host Scanning Host
2016-02-19 02:33:05 198.204.234.74 Kansas City Scanning Host Malicious Host
2016-02-19 02:57:03 104.243.223.8 Tampa Malicious Host
2016-02-19 02:58:02 198.20.99.130 Netherlands Malicious Host
2016-02-19 03:27:43 162.244.35.25 United States Malicious Host
2016-02-19 03:28:13 89.163.251.200 Germany Malicious Host
2016-02-19 04:28:25 71.6.165.200 San Diego Malicious Host
2016-02-19 04:52:08 93.174.93.181 Netherlands Malicious Host
2016-02-19 04:58:24 184.105.247.238 Fremont Malicious Host
2016-02-19 05:23:07 192.162.101.79 Russia Malicious Host
2016-02-19 05:23:20 64.125.239.112 United States Malicious Host
2016-02-19 06:12:43 188.138.1.218 Germany Malicious Host Scanning Host
2016-02-19 06:12:44 74.82.47.55 Fremont Malicious Host
2016-02-19 06:43:55 209.239.123.106 St Louis Malicious Host
2016-02-19 07:13:45 185.56.28.67 Netherlands Malicious Host
2016-02-19 08:13:09 184.105.247.228 Fremont Malicious Host
2016-02-19 08:28:51 184.105.139.72 Fremont Malicious Host
And this is only the part that relates to the villains known to the community, and there is also another part of the report with a list of unknown villains. Of course, most of them do not push much. For example: like this (this is a clipping from the same report): But there are also very intrusive personalities. You will also need firewall logs. This is the minimum that is in any office. And then there are internal things: access logs to the DBMS and individual business applications, web server logs, etc. etc. You can collect anything you want. If there is no standard plugin for parsing logs, it is very easy to write your own. The truth is easy. Developing your first plugin will not take you more than a couple of days. And the development of the second will take a couple of hours.
Информация Netflow для 180.97.106.37 : Nanjing : Malicious Host
Время Период Протокол Источник Получатель Пакетов Байт Потоков
2016-02-18 09:22:46.585 0.000 ICMP 91.111.111.9:0 180.97.106.37:3.0 1 56 1
2016-02-18 09:22:46.586 0.000 TCP 180.97.106.37:46024 91.111.111.9:3128 1 40 1
2016-02-18 20:32:23.247 0.000 TCP 180.97.106.37:37254 91.111.111.101:22 1 40 1
2016-02-18 20:43:38.783 0.000 TCP 180.97.106.37:45840 91.111.111.25:22 1 40 1
2016-02-18 20:43:38.783 0.000 ICMP 91.111.111.25:0 180.97.106.37:3.0 3 204 1
2016-02-18 22:07:25.502 0.000 TCP 180.97.106.37:54895 91.111.111.36:22 2 80 1
2016-02-18 22:41:06.739 0.000 TCP 91.111.111.13:22 180.97.106.37:48365 1 40 1
2016-02-18 23:16:01.974 0.996 TCP 180.97.106.37:13302 91.111.111.32:80 10 539 1
2016-02-18 23:16:01.975 0.679 TCP 91.111.111.32:80 180.97.106.37:13302 7 3048 1
2016-02-18 23:20:07.473 0.000 TCP 180.97.106.37:43667 91.111.111.9:22 2 80 1
2016-02-18 23:20:07.473 0.000 ICMP 91.111.111.9:0 180.97.106.37:3.0 1 56 1
2016-02-19 05:50:52.757 12.217 TCP 91.111.111.44:80 180.97.106.37:53461 5 260 1
As for the control of violations of organizational protective measures, for example, it is forbidden in your office to make changes to the configuration of the production environment without the approval of the information security service. OSSEC will tell you about any configuration change and you can check if it was agreed or not. The most interesting part begins when it turns out that no one has made this change.
I will not give advice on exactly which control rules should be applied. Each village has its own traditions. If you have questions, ask.
Important.AlienVault OSSIM is not only SIEM in the classical sense. This is an entire bookcase that includes: Host IDS, Network IDS, Wireless IDS, Volnurability Scaner, NetFlow Collector. That is, this is a complete classic set of “controls” for organizing video surveillance of a company’s network.
Trouble and Shooting
If it seems to you that something is working wrong, you should start looking at the logs. Where?
OSSEC logs and errors:
`/var/ossec/logs/ossec.log`- here you will see OSSEC errors. The most common mistake is a loss of communication with the agent, it happens when there are some fatal communication failures over the network, or after reinstalling the agent. Rare, but with a large number of agents it may not be so rare. If you see on the console that an agent is not active, and you know that the computer is turned on, then you are here. In the log, the error looks like `ERROR: Duplicated counter for 'agent-name'`. It is eliminated simply. In the web console (environment - detection - agents) or in the file `/ var / ossec / etc / client.keys` we look for an agent with this name and look at its number (the number in the leftmost column). Then we climb into the `/ var / ossec / queue / rids` directory and delete the file with the name - agent number in it. We go SSH to the server console, exit the command line and do `/etc/init.d/ossec restart`. All things. Honestly, I have not seen other errors with OSSEC.
`/var/ossec/logs/alerts/alerts.log` is the log in which OSSEC collects chewed events from agents, this is where OSSIM reads events and processes them with an OSSEC plugin. Here you can see what is falling there and how it looks.
The rest of the logs in the traditional `/ var / log` of them are useful` / var / log / alienvault / agent / agent.log` and in the same place `agent_error.log`. They can come in handy when debugging your own plugins. Note that on the working system the size of `agent.log` is in gigabytes.
I have never seen any serious errors in the operation of OSSIM. Once after the next system update, the regular backup stopped working. It turned out they incorrectly assigned the rights to one of the files. This was fixed literally the next day. Information on such issues takes place on the community forum. One more time, after the next update, the following updates stopped working. This time a notification came that the update that was posted four hours ago contains a bug and that it needs to be fixed in this way, but the corrected update was posted at such and such a time. By the way, if the update by standard means (from the web or ssh console) crashes and does not work, it can be performed by commands from the server console.
apt-get update
apt-get upgrade
Perhaps this is all I wanted to say. And most importantly. This is open source. Whatever you want, Python is with you. And if you do not want him to be with you, look at the commercial version - Python with them. And here we come to the last question.
Open Source or Enterprise?
My answer is: if you need SIEM for the needs of the company, then Enterprise. You will never have the human and technical resources to bring the Open Source version to the Enterprise level, of course, if your company does not set as its goal the development of its own SIEM. But, if this is not your business, take care of your own. Unfortunately, any Enterprise SIEM is very expensive. But they are so expensive not because they have some unusual correlation server engines, but because they have invested huge expenses in creating event correlation rule libraries, analyzing the “signatures” of complex attacks and how to detect them, and debugging these methods. This is a job that you cannot do on your own. I don’t want to talk about the advantages of commercial products here; to me these advantages seem obvious and not worth discussing.
So why am I wasting time on this Open Source? First of all, I like him. Of all such Open Source projects, this one is successful. Secondly, AlienVault’s website says something like “we want to make security accessible to everyone.” Not bad idea. I support. Too many companies simply can not afford to buy a commercial product of this class. I know that there is a whole army of freelance system administrators, let's call them serving small companies. It makes sense for them to look this way. It's not for nothing that at the very beginning I mentioned a mini-computer. Little SIEM is a very real thing.
Enterprise - enterprise, and community - community. Land - to the peasants, factories - to workers, money - to bank employees. “Happiness to all, and that enough for everyone” (A. and B. Strugatsky. “Picnic on the sidelines”).
Sources used:
https://www.alienvault.com/documentation
https://alienvault.ru/open-threat-exchange/
http://ossec.github.io/docs/
http://suricata-ids.org/ docs /