March 7, 2017 at 21:32OSSEC parsers configuration note (decoders)
Greetings to the distinguished community. In this article I want to describe several important (in my opinion) points that you need to keep in mind when setting up OSSEC software (HIDS, SIEM system). The official documentation for the system is presented in a fairly large amount on the vast expanses of the network, but some important points are not described anywhere. As "travel notes" I will give them below. I will make a reservation right away that I will not describe the installation of the system, the deployment of agents, the initial configuration. Those. I assume that the reader already knows what decoder and rule are in the context of OSSEC. All of the following has been tested on version 2.8.1, it may be fixed in future versions. So to the battle.
When developing parsers, or as OSSEC calls them, “decoder” there is the possibility of using several levels, i.e. parent decoder (parent) and children.
Parent describes a general view under which all events of a certain class fall (for example, all events of the event log of a Windows OS). Depending on certain parameters of the original message (namely, prematch parameter), child parsers can be applied to it or not, and thus different information can be received (parsed) from different types of messages. Here you need to keep in mind the following points that are not described in the documentation:
1 - child decoders completely erase all the data that was received from the original "raw" message (using its parsing);
2 - a consequence of paragraph 1 - in the parent decoder you do not need to perform parsing, i.e. use the regex directive (anyone who knows the OSSEC context will understand). All that should be done in it is to determine ALL events from a given source according to a given pattern. But do not specify fields for parsing. All parsing will be performed by child parsers;
3 - regex used in OSSEC is not able to process records of the form ". *" Inside expressions. Example. If you need from a line of the form:
Get, for example, only a timestamp and Logon type, then using a regular expression:
this will not succeed. OSSEC just won't handle it. No error messages. What to do? Read about it below. Oh, by the way, constructions like \ d {3} embedded in the OSSEC regexp also do not understand, i.e. if you need exactly three digits in a row - please write \ d \ d \ d. Horror-horror-horror.
4 - The possibility of using several decoders simultaneously.
So, the question of obtaining data from "raw" events of different formats can be solved as follows (I will consider the example of the Windws OS log, with which I worked). This solution was drawn from here.. The fact is that OSSEC can apply several decoders to one raw message at the same time and not in a hierarchical order (because if you build a hierarchy, then the last in the decoder hierarchy will overwrite all the data received by the parent, as we remember from point 1 this article).
In order to force OSSEC to use several decoders at once, they need to be given the same name.
In the example below, a two-level hierarchy of decoders is built. The first level simply looks for all Windows Event Log events, and the child level of decoders parses. In a visual form, it is shown below in Figure 1.
Figure 1 - The hierarchy of decoders is clearly shown. The
parent decoder looks as follows:
At the second level of the hierarchy, we immediately have 3 decoders, which have the same name - “windows-generic”.
The first decoder receives data from the "raw" message of the windows log, which it places in the following fields of the scheme (describing the event inside OSSEC): status, id, extra_data, srcuser, system_name
Two other decoders are used to extract the source IP address from the Windows log. At first I tried to solve this problem by writing regexp with ". *" In the middle. However, as I wrote earlier, OSSEC does not process such regexp. Therefore, I decided to create separate decoders for obtaining the IP source from the "raw" windows log.
Because The source IP address can appear in the Windows log in a different form (for example, after the expression "Source IP address:" or after the expression "Client Address"), then I created two decoders, each for its own design. They are listed below.
decoder for obtaining IP from a design of the form “Source IP Address:":
decoder for obtaining IP from a design of the form “Client Address:":
Please note that in both decoders that receive the source IP in the regex directive, offset = after_regex is set . This is not a typo. The fact is that this type of offset only works if there is already another decoder with the same name, but which has one of the standard values in the regex offset parameter (after_prematch, after_parent or offset is not there). And, accordingly, the decoder with after_regex is triggered after the decoder with the standard value.
As you understand, similar constructions can be created for obtaining other data from the windows log.
That's all for me, thanks for reading. And good luck in knowing not known!
When developing parsers, or as OSSEC calls them, “decoder” there is the possibility of using several levels, i.e. parent decoder (parent) and children.
Parent describes a general view under which all events of a certain class fall (for example, all events of the event log of a Windows OS). Depending on certain parameters of the original message (namely, prematch parameter), child parsers can be applied to it or not, and thus different information can be received (parsed) from different types of messages. Here you need to keep in mind the following points that are not described in the documentation:
1 - child decoders completely erase all the data that was received from the original "raw" message (using its parsing);
2 - a consequence of paragraph 1 - in the parent decoder you do not need to perform parsing, i.e. use the regex directive (anyone who knows the OSSEC context will understand). All that should be done in it is to determine ALL events from a given source according to a given pattern. But do not specify fields for parsing. All parsing will be performed by child parsers;
3 - regex used in OSSEC is not able to process records of the form ". *" Inside expressions. Example. If you need from a line of the form:
2017 Mar 05 19:45:26 WinEvtLog: Security: AUDIT_SUCCESS(4634): Microsoft-Windows-Security-Auditing: DOM_WINSRV-DC$: DOMAIN: nsrv_WinSRV-DC.domain.test: An account was logged off. Subject: Security ID: S-1-5-18 Account Name: DOM_WINSRV-DC$ Account Domain: DOMAIN Logon ID: 0x6dd15b4 Logon Type: 3 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer." 4646,1Get, for example, only a timestamp and Logon type, then using a regular expression:
(\d\d\d\d\s+\w\w\w\s+\d\d\s+\d\d:\d\d:\d\d)\s+.*?Logon\s+Type:\s+(\d+)this will not succeed. OSSEC just won't handle it. No error messages. What to do? Read about it below. Oh, by the way, constructions like \ d {3} embedded in the OSSEC regexp also do not understand, i.e. if you need exactly three digits in a row - please write \ d \ d \ d. Horror-horror-horror.
4 - The possibility of using several decoders simultaneously.
So, the question of obtaining data from "raw" events of different formats can be solved as follows (I will consider the example of the Windws OS log, with which I worked). This solution was drawn from here.. The fact is that OSSEC can apply several decoders to one raw message at the same time and not in a hierarchical order (because if you build a hierarchy, then the last in the decoder hierarchy will overwrite all the data received by the parent, as we remember from point 1 this article).
In order to force OSSEC to use several decoders at once, they need to be given the same name.
In the example below, a two-level hierarchy of decoders is built. The first level simply looks for all Windows Event Log events, and the child level of decoders parses. In a visual form, it is shown below in Figure 1.
Figure 1 - The hierarchy of decoders is clearly shown. The
parent decoder looks as follows:
windows ^\d\d\d\d \w\w\w \d\d \d\d:\d\d:\d\d WinEvtLog: At the second level of the hierarchy, we immediately have 3 decoders, which have the same name - “windows-generic”.
The first decoder receives data from the "raw" message of the windows log, which it places in the following fields of the scheme (describing the event inside OSSEC): status, id, extra_data, srcuser, system_name
windows windows Security \S+:\s+(\w+)\((\d+)\):\s+(\S+):\s+ (\.+):\s+\.+:\s+(\S+):\s+ status, id, extra_data, srcuser, system_name name, location, user, system_name Two other decoders are used to extract the source IP address from the Windows log. At first I tried to solve this problem by writing regexp with ". *" In the middle. However, as I wrote earlier, OSSEC does not process such regexp. Therefore, I decided to create separate decoders for obtaining the IP source from the "raw" windows log.
Because The source IP address can appear in the Windows log in a different form (for example, after the expression "Source IP address:" or after the expression "Client Address"), then I created two decoders, each for its own design. They are listed below.
decoder for obtaining IP from a design of the form “Source IP Address:
windows windows Source\s+IP\s+Address:\s+(\S+) srcip decoder for obtaining IP from a design of the form “Client Address:
windows windows Client\s+Address:\s+(\S+) srcip Please note that in both decoders that receive the source IP in the regex directive, offset = after_regex is set . This is not a typo. The fact is that this type of offset only works if there is already another decoder with the same name, but which has one of the standard values in the regex offset parameter (after_prematch, after_parent or offset is not there). And, accordingly, the decoder with after_regex is triggered after the decoder with the standard value.
As you understand, similar constructions can be created for obtaining other data from the windows log.
That's all for me, thanks for reading. And good luck in knowing not known!