April 11, 2017 at 17:57OSSIM WMI plugins with crisp. Recipe
Good day, dear readers.
I want to share with you my experiments on collecting Windows event logs in OSSIM without using agents, i.e. using WMI plugins.
These plugins use the WMI interface and, accordingly, the wmi client to collect event logs. Using these plugins is not recommended, or rather, it is NOT recommended by the vendor (Alienvault) by the method of collecting event logs of target systems. They argue that this method uses much more system resources than collecting event logs using the OSSEC agent. This is true, but in addition, when using WMI, events can simply disappear, or be delayed for 20-30 minutes, which, of course, is not acceptable.
In addition, the standard WMI plugins are made in such a way that they receive practically no useful information from Windows logs. They receive only the Windows event identifier (numeric and its translation into human language, for example: 4624 - An account was successfully logged on), log name (Security, System, Application ...), timestamp and a huge message in the message field (which contains all the useful information - IP addresses, usernames, UAC settings, etc.). Therefore, in order to get some useful information, the message field must be subjected to additional parsing, i.e. regular WMI plugins need to be additionally configured. Without this setting, a standard WMI plugin is almost useless.
If, nevertheless, the situation forces you to use not the OSSEC agent, but WMI plugins, I will tell you how to reduce the number and reduce the size of the rake that you encounter along the way, namely:
I will consider this task using the example of obtaining a username. The tuning algorithm for receiving any other data will be similar.
To obtain additional data, the “custom functions” functional is used, which allows applying a self-written function to any field received via WMI. This function will parse the message field of the Windows log and put the received value (username) in the username field of the event displayed in OSSIM.
First, take a look at the standard wmi-security-logger plugin. As can be seen from its configuration file, located in /etc/ossim/agent/plugins/wmi-security-logger.cfg, it performs two WQL (SQL for WMI) queries, the first of which (the cmd field in the start_cmd section) gets the number events starting from which the log is read, and the second (cmd field in the cmd section) already reads the log. Here is an example of a second query reading a log:
The plugin reads these fields from the Windows log:
ComputerName, EventCode, Logfile, Message, RecordNumber, SourceName, TimeWritten, User
You should not build hopes about the “User” field in Windows 2008R2 and the “null” value gets fresh in it.
Thus, the most informative field is “message”. From it using the function we will get the username.
If you look at the examples of the Windows log, we see that there are two constructions containing the username:
1) Logon Account: <username>
An example of a "raw" event, as OSSIM sees it:
2) Account Name: <username>
An example of a "raw" event, as OSSIM sees it:
Thus, we need a function that can get the username from both of these constructs. The functions used in the plugins must be written in python 2.7 and stored in a separate file, the path to which will be written in a special variable in the plugin configuration.
Please note that in the "raw" log \ r, \ t, \ n are all special characters, i.e. carriage return, new line and tab character.
For files in which functions are stored there is a special daddy - / etc / ossim / agent / plugins / custom_functions. A file with functions for obtaining additional information from the WMI log is called “wmi_funcs.cfg”.
Here are its contents:
The body of the function must be enclosed in the tags “Start Function” and “End Function”. Also, if you need to import additional python modules, as in my case, import must also be written before the function body itself.
It is especially worth paying attention to the moment that to designate special characters from the "raw" log "\ r", "\ n", "\ t" you need to use the construction [^ \ w] r, [^ \ w] n and [ ^ \ w] t respectively. Those. the symbol "\" from the "raw" message in our regular expression (namely, with the help of regular expressions we get data by this function) is represented as "[^ \ w]". This solution was found by trial and error. No other constructions, including “. *”, “\ S +”, escaping slashes, and others could not cope with “\” in the raw log.
As can be seen from the function, if there are several “Account Name” constructions in one message (and this happens, for example, in events of changing one user KM to another KM), it (function) takes the second value as the user name. Because the first one is just the name of the user who changed the target KM.
Next, at the end of the [config] section of the plugin configuration file “/etc/ossim/agent/plugins/wmi-security-logger.cfg”, add a parameter that refers to the function file:
custom_functions_file = / etc / ossim / agent / plugins / custom_functions / wmi_funcs .cfg
And in the [cmd] section we add a parameter that uses a custom function to obtain data about the user's ultrasound. In general, it looks like this:
And in my particular case it looks like this:
As a result, the plugin configuration file will look like this:
To diagnose the plug-in, you need to perform the following steps:
1) Check the WMI operation by running the wmic client from the OSSIM server console.
The arguments with which wmic should be started can be taken from the same plugin configuration file.
For example, like this:
The result of successful execution of this command should be a list of numeric event identifiers from the Windows log.
If errors appear, then it is necessary to verify that this KM is configured correctly and has all the necessary rights to receive event logs of the target system. If there is a firewall between the OSSIM server and the target system, you must also make sure that it passes OSSIM → Windows traffic and in the opposite direction.
2) Restart the OSSIM plugin.
First stop the plugin:
Then kill all the wmi client processes that are still hanging:
And in turn:
After that, run the agent:
3) Clean the log of the target system.
On highly loaded servers, such as domain controllers with a large number of users, the event log can grow very much, occupying hundreds of megabytes. In my practice, I came across the fact that on such volumes of WMI event logs, the plugin starts to receive events with significant time delays. The only recommendation here is to set the log size limit on the target system. It is advisable if it is no more than 30 MB.
In the laboratory, the WMI plugin often only started working after the complete emptying of the large security log.
I want to share with you my experiments on collecting Windows event logs in OSSIM without using agents, i.e. using WMI plugins.
These plugins use the WMI interface and, accordingly, the wmi client to collect event logs. Using these plugins is not recommended, or rather, it is NOT recommended by the vendor (Alienvault) by the method of collecting event logs of target systems. They argue that this method uses much more system resources than collecting event logs using the OSSEC agent. This is true, but in addition, when using WMI, events can simply disappear, or be delayed for 20-30 minutes, which, of course, is not acceptable.
In addition, the standard WMI plugins are made in such a way that they receive practically no useful information from Windows logs. They receive only the Windows event identifier (numeric and its translation into human language, for example: 4624 - An account was successfully logged on), log name (Security, System, Application ...), timestamp and a huge message in the message field (which contains all the useful information - IP addresses, usernames, UAC settings, etc.). Therefore, in order to get some useful information, the message field must be subjected to additional parsing, i.e. regular WMI plugins need to be additionally configured. Without this setting, a standard WMI plugin is almost useless.
If, nevertheless, the situation forces you to use not the OSSEC agent, but WMI plugins, I will tell you how to reduce the number and reduce the size of the rake that you encounter along the way, namely:
- How to configure receiving additional data from event logs collected via WMI;
- How to diagnose and troubleshoot a WMI plugin.
1. Obtaining additional data from event logs collected via WMI
I will consider this task using the example of obtaining a username. The tuning algorithm for receiving any other data will be similar.
To obtain additional data, the “custom functions” functional is used, which allows applying a self-written function to any field received via WMI. This function will parse the message field of the Windows log and put the received value (username) in the username field of the event displayed in OSSIM.
First, take a look at the standard wmi-security-logger plugin. As can be seen from its configuration file, located in /etc/ossim/agent/plugins/wmi-security-logger.cfg, it performs two WQL (SQL for WMI) queries, the first of which (the cmd field in the start_cmd section) gets the number events starting from which the log is read, and the second (cmd field in the cmd section) already reads the log. Here is an example of a second query reading a log:
Select ComputerName,EventCode,Logfile,Message,RecordNumber,SourceName,TimeWritten,User from Win32_NTLogEvent Where Logfile = 'Security' and RecordNumber > OSS_COUNTERThe plugin reads these fields from the Windows log:
ComputerName, EventCode, Logfile, Message, RecordNumber, SourceName, TimeWritten, User
You should not build hopes about the “User” field in Windows 2008R2 and the “null” value gets fresh in it.
Thus, the most informative field is “message”. From it using the function we will get the username.
If you look at the examples of the Windows log, we see that there are two constructions containing the username:
1) Logon Account: <username>
An example of a "raw" event, as OSSIM sees it:
nsrv_WinSRV-DC.domain.test|4776|Security|The computer attempted to validate the credentials for an account.\r\n\r\nAuthentication
Package:\tMICROSOFT_AUTHENTICATION_PACKAGE_V1_0\r\nLogon Account:\tAdministrator\r\nSource Workstation:\tALIENVAULT\r\nError
Code:\t0x0|17507786|Microsoft-Windows-Security-Auditing|20170410111356.914996-000|(null)2) Account Name: <username>
An example of a "raw" event, as OSSIM sees it:
nsrv_WinSRV-DC.domain.test|4672|Security|Special privileges assigned to new logon.\r\n\r\nSubject:\r\n\tSecurity
ID:\t\tS-1-5-21-2827692199-2880599349-568663292-500\r\n\tAccount Name:\t\tAdministrator\r\n\tAccount Domain:\t\tDOMAIN\r\n\tLogon
ID:\t\t0x2B9EB90D\r\n\r\nPrivileges:\t\tSeSecurityPrivilege\r\n\t\t\tSeBackupPrivilege\r\n\t\t\tSeRestorePrivilege\r\n\t\t\tSeTakeOwnershipPrivilege\r\n\t\t\tSeDebugPrivilege\r\n\t\t\tSeSystemEnvironmentPrivilege\r\n\t\t\tSeLoadDriverPrivilege\r\n\t\t\tSeImpersonatePrivilege\r\n\t\t\tSeEnableDelegationPrivilege|17507787|Microsoft-Windows-Security-Auditing|20170410111356.914996-000|(null)Thus, we need a function that can get the username from both of these constructs. The functions used in the plugins must be written in python 2.7 and stored in a separate file, the path to which will be written in a special variable in the plugin configuration.
Please note that in the "raw" log \ r, \ t, \ n are all special characters, i.e. carriage return, new line and tab character.
For files in which functions are stored there is a special daddy - / etc / ossim / agent / plugins / custom_functions. A file with functions for obtaining additional information from the WMI log is called “wmi_funcs.cfg”.
Here are its contents:
Start Function win_account
import re
def win_account(self, input=''):
try:
try:
res = re.search( 'Logon Account:[^\w]t(\S+)[^\w]r[^\w]', input ).group(1)
return res
except:
res = re.findall( 'Account Name:[^\w]t[^\w]t(\S+)[^\w]r[^\w]', input )
if len(res) > 1:
return res[1]
else:
return res[0]
except:
return None
End FunctionThe body of the function must be enclosed in the tags “Start Function” and “End Function”. Also, if you need to import additional python modules, as in my case, import must also be written before the function body itself.
It is especially worth paying attention to the moment that to designate special characters from the "raw" log "\ r", "\ n", "\ t" you need to use the construction [^ \ w] r, [^ \ w] n and [ ^ \ w] t respectively. Those. the symbol "\" from the "raw" message in our regular expression (namely, with the help of regular expressions we get data by this function) is represented as "[^ \ w]". This solution was found by trial and error. No other constructions, including “. *”, “\ S +”, escaping slashes, and others could not cope with “\” in the raw log.
As can be seen from the function, if there are several “Account Name” constructions in one message (and this happens, for example, in events of changing one user KM to another KM), it (function) takes the second value as the user name. Because the first one is just the name of the user who changed the target KM.
Next, at the end of the [config] section of the plugin configuration file “/etc/ossim/agent/plugins/wmi-security-logger.cfg”, add a parameter that refers to the function file:
custom_functions_file = / etc / ossim / agent / plugins / custom_functions / wmi_funcs .cfg
And in the [cmd] section we add a parameter that uses a custom function to obtain data about the user's ultrasound. In general, it looks like this:
<имя поля в OSSIM>={:<имя функции>($<номер поля, получаемого запросом WQL>)}And in my particular case it looks like this:
username={:win_account($3)}As a result, the plugin configuration file will look like this:
[DEFAULT]
plugin_id=1518
[config]
type=detector
enable=yes
source=wmi
credentials_file=/etc/ossim/agent/wmi_credentials.csv
sleep=10
process=
start=no
stop=no
custom_functions_file=/etc/ossim/agent/plugins/custom_functions/wmi_funcs.cfg
[start_cmd]
cmd=wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select LogFile,RecordNumber from Win32_NTLogEvent Where Logfile = 'Security'" | head -n 3 | tail -n 1 | cut -f 2 -d \|
regexp=
[cmd]
cmd = wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select ComputerName,EventCode,Logfile,Message,RecordNumber,SourceName,TimeWritten,User from Win32_NTLogEvent Where Logfile = 'Security' and RecordNumber > OSS_COUNTER" | cat
start_regexp=^([^\|]+)\|(\d+)\|([^\|]+)\|
regexp="^(?P[^\|]+)\|(?P\d+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P[^\|]+)\|(?P.*)$"
src_ip={resolv($0)}
plugin_sid={$1}
userdata2={$2}
userdata3={$3}
userdata4={$4}
userdata5={$5}
userdata6={$6}
username={:win_account($3)} 2. Diagnostics and troubleshooting WMI plugin
To diagnose the plug-in, you need to perform the following steps:
1) Check the WMI operation by running the wmic client from the OSSIM server console.
The arguments with which wmic should be started can be taken from the same plugin configuration file.
For example, like this:
wmic -U domain\Administrator%Passw0rd //server.example.com "Select LogFile,RecordNumber from Win32_NTLogEvent Where Logfile = 'Security'"The result of successful execution of this command should be a list of numeric event identifiers from the Windows log.
If errors appear, then it is necessary to verify that this KM is configured correctly and has all the necessary rights to receive event logs of the target system. If there is a firewall between the OSSIM server and the target system, you must also make sure that it passes OSSIM → Windows traffic and in the opposite direction.
2) Restart the OSSIM plugin.
First stop the plugin:
/etc/init.d/ossim-agent stopThen kill all the wmi client processes that are still hanging:
ps –ef |grep wmiAnd in turn:
kill -9 “номер процесса”After that, run the agent:
/etc/init.d/ossim-agent start3) Clean the log of the target system.
On highly loaded servers, such as domain controllers with a large number of users, the event log can grow very much, occupying hundreds of megabytes. In my practice, I came across the fact that on such volumes of WMI event logs, the plugin starts to receive events with significant time delays. The only recommendation here is to set the log size limit on the target system. It is advisable if it is no more than 30 MB.
In the laboratory, the WMI plugin often only started working after the complete emptying of the large security log.