DNS tunnel, PsExec, keylogger: disassemble the scheme and technical attack tools
The article “ One quarter of the life of SOC. Three incidents without cuts ”were very actively pitted, so we decided to talk about another interesting attack, which we recently investigated.
There is an opinion that international corporations and large companies that keep up to date in the provision of services to their customers, also clearly organize processes in all areas of their activities, including information security. Unfortunately, this is not always the case.
Some time ago, a large company with developed infrastructure turned to us for help. The problem was strange events in the company's infrastructure:
To analyze the situation, we connected the main infrastructure sources to the SIEM system located in the Solar JSOC cloud. To do this, we placed a collector server for collecting logs on the customer’s site and built a site-to-site between the sites. In parallel, the company was sent instructions on how to set up the necessary audit level, as well as a detailed description of the preparatory work for connecting the source.
At the first stage, we connected firewalls and proxies, anti-virus, logs of domain controllers and DNS. By the evening of the next day we had the logs of all the necessary systems.
The first thing that was detected was the access from 12 workstations to the Corkow / Metel management servers. It turned out that for more than two years the client parts of one of the Win32 / Corkow virus modifications remained unnoticed by anyone in the company's infrastructure, despite the presence of antivirus software. Malicious people sent telemetry to control servers that had been turned off for a long time (the domain names of the servers were named after two great Russian artists and are widely known to information security analysts). The anti-virus vendor, the software of which was used in the company, did not add known signatures to its databases, therefore it could not detect the virus.
But the point was not in this sensational, but not more dangerous virus. Literally after several hours of monitoring, for the first time in real practice of Solar JSOC, a full-fledged, non-test, DNS tunnel was discovered that sends information from several hosts of the company's infrastructure.
Immediately after the discovery of the company, a request was sent to investigate the discovered sources of DNS tunnels. Several machines were connected at the local log level, and the imaging process was launched for further research.
When connecting the hosts, Solar JSOC specialists faced the first difficulty - the Security Log was empty on all machines. At the same time, the image was examined, and then a second complexity arose - the USN (Update Sequence Number) and MFT (Master File Table) did not contain at least any significant information, the latter due to the frequent scheduled defragmentation of the disks.
The first significant information was found in the logs of domain controllers - access to the hosts under the domain administrator account was revealed there. Login was made with logon type 3 - network input.
Further, analyzing on all potentially compromised System Log hosts that was not cleared, we found the installation of the it_helpdesk service . After analyzing the MD5 sum, it became clear that this is the renamed utility PsExec. The company's IT department has confirmed that this software is not a corporate standard of administration and is not used by employees.
After that, the information security department of the company, using the centralized infrastructure management system, identified all the hosts on which this service was ever launched. The total number of such hosts exceeded 40 units.
Now back to the study of the images of workstations. Analysis of the current state of the file system of one of the machines and reproduction of the infection in the laboratory gave an understandable chronology of the infection:
Stage I
Stage II
Stage III
The general infection scheme is as follows:
Description of the tools used to implement the attack
The following malware components were used to implement the attack within the framework of the studied images of workstations and servers:
Client-server communication
All server communication with the client is encrypted. Encryption key: 25 d9 01 4c 21 c9 ed 89 86 14 8d 05 _________
The virus sends DNS packets to the management server for resolution. The DNS name, starting with the third subdomain, is encoded data.
Packets of the form <27 symbols> .xxxxx.su are sent to the server. The sequence can be more than 27 characters, but the minimum packet is 27 characters. A sequence of 27 characters is the data encoded after encryption, which the client sends to the server. This package does not carry any information except pseudorandom numbers and the hash of the package. Pseudorandom numbers are needed so that after all the transformations the packets are not similar to each other. A packet of 27 characters tells the server that it is ready to accept a processing command. An example of such a package:
In response, a command comes in the form of 6 ipv4 addresses - 24 bytes of data. Address data is written sequentially and sorted by low octet. Discarding the lower octets, we get a sequence of 18 bytes.
The first byte is the amount of unused data (n = 1-3).
The last n bytes are pseudorandom and are not used in the future.
The remaining bytes are encrypted data with the key above.
The first three bytes are a pseudorandom number and a hash of the packet, which makes the same command from the server visually different, which is why the IP addresses seem random. The rest is a team.
Next steps and mitigation
At the first stage of the search for indicators, images of four workstations from which active DNS tunnels were recorded were analyzed.
After identifying the host and network indicators, as well as the general pattern of actions of the attackers, it was necessary to check the entire infrastructure both to identify the source of infection and to search for all compromised nodes.
The overall picture of the search for indicators of compromise was as follows: The
search was carried out both by Solar JSOC specialists and the company's employees. In total, the search took 3 days. The scout of infected systems grew to 64, the number of compromised accounts potentially increased to the total number of company employees, since one of the compromised machines was a domain controller.
At the second stage, several more machines were selected for the study to search for additional indicators of compromise. Images, dumps were removed, and it was possible to begin the process of “reloading” compromised hosts.
If such a massive, prolonged and deep infection is detected, the process of cleaning the “tails” is very difficult and long. The sequence of actions was as follows:
Key recommendations and monitoring measures
Attackers always reserve their access to the infrastructure, so in parallel a full monitoring of incidents and profiling of all activities was carried out.
There was a separate complication with the latter, since having collected a profile in two weeks, you cannot confidently call it legitimate, because the attackers could still be in the infrastructure. Therefore, a procedure was needed to coordinate all the collected activities with the company and in the future - fixing the collected profiles. General recommendations for identifying backup access were as follows:
At the same time, Solar JSOC monitored activity on critical servers and workstations in the following areas:
Key findings and findings of the incident investigation
After taking operational measures to block the threat, time appeared for a complete report on the incident:
As a final thought, I would like to note that the detection of similar incidents is the task of the Security Operations Center, but without it, you can do something if you organize the work with ordinary employees of the company and constantly increase their Security Awareness.
Attackers often try to hide their activity from the information security service and IT administrators, since these categories of employees are competent in the field of information security and understand that various anomalies can be caused by external influences. At the same time, hackers often neglect to hide their actions from ordinary users. The increased load on the computer, strange actions on the system, applications that suddenly open or close, the appearance of new files, icons, installed applications that the user notices, can serve as an indicator of system compromise. Therefore, security guards need to be careful about incoming requests, complaints from company personnel and motivate employees to inform those responsible for the noted anomalies.
There is an opinion that international corporations and large companies that keep up to date in the provision of services to their customers, also clearly organize processes in all areas of their activities, including information security. Unfortunately, this is not always the case.
Some time ago, a large company with developed infrastructure turned to us for help. The problem was strange events in the company's infrastructure:
- Workstations and servers suddenly went to reboot and were removed from the domain.
- Users found that their account was locked.
- The computers of some employees began to "slow down" for no apparent reason.
To analyze the situation, we connected the main infrastructure sources to the SIEM system located in the Solar JSOC cloud. To do this, we placed a collector server for collecting logs on the customer’s site and built a site-to-site between the sites. In parallel, the company was sent instructions on how to set up the necessary audit level, as well as a detailed description of the preparatory work for connecting the source.
At the first stage, we connected firewalls and proxies, anti-virus, logs of domain controllers and DNS. By the evening of the next day we had the logs of all the necessary systems.
The first thing that was detected was the access from 12 workstations to the Corkow / Metel management servers. It turned out that for more than two years the client parts of one of the Win32 / Corkow virus modifications remained unnoticed by anyone in the company's infrastructure, despite the presence of antivirus software. Malicious people sent telemetry to control servers that had been turned off for a long time (the domain names of the servers were named after two great Russian artists and are widely known to information security analysts). The anti-virus vendor, the software of which was used in the company, did not add known signatures to its databases, therefore it could not detect the virus.
But the point was not in this sensational, but not more dangerous virus. Literally after several hours of monitoring, for the first time in real practice of Solar JSOC, a full-fledged, non-test, DNS tunnel was discovered that sends information from several hosts of the company's infrastructure.
DNS tunnels can be called frills in the daily life of a security guard. They are used quite rarely, but recently they have been highlighted in a number of high-profile cases on an international scale as a channel for outputting information outside the company's perimeter.First of all, measures were taken to block external addresses on all border network devices.
But the danger of DNS tunnels lies not only in the fact that with their help you can quietly steal data from the infrastructure. DNS tunnels allow you to build a reverse shell with a final host, which allows you to control its actions remotely.
Despite the fact that DNS tunnels are a very old topic, and all IPS and NGFW class solutions should detect them, in practice this is far from the case. The slightest change in the parameters (for example, transferring payload to the key field or another field of the standard DNS format, or outside the standard fields of the DNS query) allows you to easily bypass standard signatures.
Immediately after the discovery of the company, a request was sent to investigate the discovered sources of DNS tunnels. Several machines were connected at the local log level, and the imaging process was launched for further research.
When connecting the hosts, Solar JSOC specialists faced the first difficulty - the Security Log was empty on all machines. At the same time, the image was examined, and then a second complexity arose - the USN (Update Sequence Number) and MFT (Master File Table) did not contain at least any significant information, the latter due to the frequent scheduled defragmentation of the disks.
The first significant information was found in the logs of domain controllers - access to the hosts under the domain administrator account was revealed there. Login was made with logon type 3 - network input.
Further, analyzing on all potentially compromised System Log hosts that was not cleared, we found the installation of the it_helpdesk service . After analyzing the MD5 sum, it became clear that this is the renamed utility PsExec. The company's IT department has confirmed that this software is not a corporate standard of administration and is not used by employees.
PsExec is part of PsTools, a package of free utilities developed by Sysinternals and then acquired by Microsoft. They are designed to simplify the administration of Microsoft Windows operating systems. The PsExec utility itself allows you to remotely execute processes.
PsExec allows you to redirect the input and output data of a remotely running executable program by using SMB and the $ ADMIN hidden share resource on the remote system. Using this resource, PsExec uses the Windows Service control Manager API programming interface to start the PsExecsvc service on a remote system that creates a named pipe through which PsExec runs.
After that, the information security department of the company, using the centralized infrastructure management system, identified all the hosts on which this service was ever launched. The total number of such hosts exceeded 40 units.
Now back to the study of the images of workstations. Analysis of the current state of the file system of one of the machines and reproduction of the infection in the laboratory gave an understandable chronology of the infection:
Stage I
- Renaming the original system_dll.dll library to system_dll2 and creating the malicious system_dll.dll object. In this case, system_dll.dll calls system_dll2 for functions that are not defined in its code. system_dll.dll is a malicious object of type PE that serves to load the _________. dll library.
- Creating _________. Dll - is a malicious object of type PE, which serves to establish communication with arbitrary servers using the DNS protocol and execute various commands. This library is loaded by the system_dll.dll malicious object.
- The last time it-helpdesk was run on the machine, the C: \ Windows \ system32 \ shutdown.exe object is supposedly launched to initiate a restart of the operating system. This reboot is necessary for the System Service to load the malicious library System_dll.dll into its address space.
Stage II
- After the operating system is rebooted, error resolvings of the random symbols.xxxxx.su domain appear, which may indicate the functioning of a hidden data channel using the DNS protocol (arbitrary data is transmitted in a domain level 3 name).
- Creation of the Windows / System32 / malware_dll.dll library, which is a malicious object of the type PE used to intercept data entered from the keyboard. The interception of data is stored in the file% USER% / AppData / LocalLow / NTUSER.DAT. The data in the file is encoded using byte encoding with the subtraction of the 10H byte.
- Creation of a malicious object jusched.exe on the attacked host, which serves to reload the malware_dll.dll library. In this case, the jusched.exe object is registered at startup (the registry branch HKLM \ Software \ Microsoft \ Windows \ CurrentVersion \ Run), this means that the system will load this object at the start of the session from any user.
Stage III
- The next time a user session is created, his profile is loaded, the keylogger is launched, the LocalLow / NTUSER.DAT file is created and the results of the keylogger’s work are recorded in it within the entire user session.
- Also at this stage, the archiver is launched from the rar.exe command line in order to create the archive C: \ ProgrammData \ 0.0. This archive contains a shadow copy of the SAM file and the registry branch HKLM \ SYSTEM. This bunch of files can be used to extract account hashes from a SAM file.
- In some cases, this stage is accompanied by multiple reboots of the operating system, the execution of the wevtutil, gpscript, nslookup, cmdkey, and other commands, as well as cleaning the Application log.
- On one of the machines under study, the creation and multiple launches of the tvnserver.exe object were recorded with the simultaneous appearance of the malicious users.exe object on the machine and writing the keys 000 and 001 with the malware configuration to the HKLM \ Software \ Corporation registry branch.
The general infection scheme is as follows:
Description of the tools used to implement the attack
The following malware components were used to implement the attack within the framework of the studied images of workstations and servers:
Component name | Component Assignment |
Malware_dll.dll | Keylogger (32-bit) |
Malware_dll64.dll | Keylogger (64-bit) |
Bach.dll | Renamed the original System_dll.dll library, which is a SystemService |
System_dll.dll | A library that, when the System_Service service starts, is pumped into svchost.exe. System_dll.dll supports the same calls as system_dll2.dll, by redirecting all these functions to system_dll2.dll. Tightens _________. Dll. Is a 32-bit version |
System_dll.dll_ | 64-bit version of System_dll.dll |
_________. dll | BackDoor 32-bit |
_________. dll_ver2 | 64-bit _________. Dll |
S64 | Analogue System_dll.dll for x64 architecture |
P64 | Analog _________. Dll for x64 architecture |
It_helpdesk.exe | Renamed PsExesvc.exe (a PSExec component that is created and run on a remote machine to perform specified actions |
Users.exe | BackDoor The functionality is similar to _________. Dll, but disguised as jusched.exe - “Java Update Scheduler” |
Malicious component activity
- Malware_dll.dll:
- Creating the file "\% APPDATA% \ LocalLow \ NTUSER.DAT".
- Creation of the mbowefvncwiomcowermg32 mutex.
- Sets the capture of keystrokes on the keyboard.
- Encryption of the received data and subsequent recording to the file from the first paragraph.
- System_dll.dll:
- Autostart through the System Service.
- Download _________. Dll.
- _________. dll and users.exe:
- Sending to the resolving DNS addresses:
- www.gf8ealht9d22________________.com
- 832v1hda31sqfcl5bh81lmqk74z.xxxxxxxxx.com
- 13bmvqdr1ju64dqm6n8877hbo0z.xxxxxxxxx.com - Sending DNS packets to the management server (xxxxx.su).
- Execution of commands received from the management server.
- It is possible to create registry keys for storing data between process restarts in the HKLM or HKCU branches:
- \ Software \ Corporation \ 000
- \ Software \ Corporation \ 001
- \ Software \ Corporation \ 002
- Sending to the resolving DNS addresses:
Client-server communication
All server communication with the client is encrypted. Encryption key: 25 d9 01 4c 21 c9 ed 89 86 14 8d 05 _________
The virus sends DNS packets to the management server for resolution. The DNS name, starting with the third subdomain, is encoded data.
Packets of the form <27 symbols> .xxxxx.su are sent to the server. The sequence can be more than 27 characters, but the minimum packet is 27 characters. A sequence of 27 characters is the data encoded after encryption, which the client sends to the server. This package does not carry any information except pseudorandom numbers and the hash of the package. Pseudorandom numbers are needed so that after all the transformations the packets are not similar to each other. A packet of 27 characters tells the server that it is ready to accept a processing command. An example of such a package:
In response, a command comes in the form of 6 ipv4 addresses - 24 bytes of data. Address data is written sequentially and sorted by low octet. Discarding the lower octets, we get a sequence of 18 bytes.
The first byte is the amount of unused data (n = 1-3).
The last n bytes are pseudorandom and are not used in the future.
The remaining bytes are encrypted data with the key above.
The first three bytes are a pseudorandom number and a hash of the packet, which makes the same command from the server visually different, which is why the IP addresses seem random. The rest is a team.
Next steps and mitigation
At the first stage of the search for indicators, images of four workstations from which active DNS tunnels were recorded were analyzed.
After identifying the host and network indicators, as well as the general pattern of actions of the attackers, it was necessary to check the entire infrastructure both to identify the source of infection and to search for all compromised nodes.
The overall picture of the search for indicators of compromise was as follows: The
search was carried out both by Solar JSOC specialists and the company's employees. In total, the search took 3 days. The scout of infected systems grew to 64, the number of compromised accounts potentially increased to the total number of company employees, since one of the compromised machines was a domain controller.
At the second stage, several more machines were selected for the study to search for additional indicators of compromise. Images, dumps were removed, and it was possible to begin the process of “reloading” compromised hosts.
If such a massive, prolonged and deep infection is detected, the process of cleaning the “tails” is very difficult and long. The sequence of actions was as follows:
- Work with accounts:
- Change passwords for all specified compromised accounts, including accounts from business applications.
- The privileges of service accounts were limited; a ban was introduced on the use of accounts with domain administrator rights for the operation of services.
- During the work with accounts, a moratorium was introduced on the use of remote access for employees with the exception of IT administrators. In parallel, a second authentication factor was launched for them.
- Closed typical gaps of organizations with developed infrastructure:
- Direct Internet access bypassing the proxy.
- Removed software classified as not-a-virus and actively using the Internet.
- A profile of open ports on the perimeter was assembled, unnecessary “hot” ones were verified and closed, since the incident allowed this to be done.
- Application administrators have tightened control over actions and transactions carried out in critical business applications, especially those related to financial transactions, transactions within bonus and loyalty programs, access to client and partner base, etc.
- For IT administrators, a complete ban was introduced on working with critical business applications from under local accounts on their workstations. Everything was transferred to domain accounts with limited privileges under the control of the monitoring system.
Key recommendations and monitoring measures
Attackers always reserve their access to the infrastructure, so in parallel a full monitoring of incidents and profiling of all activities was carried out.
There was a separate complication with the latter, since having collected a profile in two weeks, you cannot confidently call it legitimate, because the attackers could still be in the infrastructure. Therefore, a procedure was needed to coordinate all the collected activities with the company and in the future - fixing the collected profiles. General recommendations for identifying backup access were as follows:
- To audit the software installed on workstations and servers in order to identify remote control tools and illegitimate software.
- Provide control over the launch of programs on critical computers and servers of the company, introduce “white lists” of allowed software, especially in relation to remote control tools.
At the same time, Solar JSOC monitored activity on critical servers and workstations in the following areas:
- Network queries to known dangerous and malicious resources, as well as attempts to DNS queries to malicious domains.
- Privileged account activity - reports were sent daily to responsible employees and account owners for verification of actions.
- Changes in privileged user groups.
- Launched processes on critical servers and workstations.
- Changes to system directories and critical registry branches for illegitimate executable files, libraries, and parameters.
- Using remote control systems.
- Anomalies in DNS traffic.
- Viral activity on critical hosts.
- Malicious mailing lists.
- Anomalies in profiles connecting to critical servers.
- Misuse of service accounts.
Key findings and findings of the incident investigation
After taking operational measures to block the threat, time appeared for a complete report on the incident:
- The channel for malware to reach infected machines was an account with domain administrator rights that was already compromised at the time of infection and was used from several computers in the company's LAN.
- The PsExec tool was used to transfer malicious files to the machine, execute remote control commands, and complete the infection of the machines with a keylogger.
- Traces of other remote control software found there several years ago were found on infected infected machines. Among the RATs were TIghtVNC, WinVNC, Pointdev.
- As a result of the keylogger’s work, as a rule, user credentials from the OS and a number of business applications, his mail correspondence, critical files with password information from key business application servers, and passport data of employees were compromised.
- The subsequent remote control and information transfer channel for attackers was a DNS tunnel.
As a final thought, I would like to note that the detection of similar incidents is the task of the Security Operations Center, but without it, you can do something if you organize the work with ordinary employees of the company and constantly increase their Security Awareness.
Attackers often try to hide their activity from the information security service and IT administrators, since these categories of employees are competent in the field of information security and understand that various anomalies can be caused by external influences. At the same time, hackers often neglect to hide their actions from ordinary users. The increased load on the computer, strange actions on the system, applications that suddenly open or close, the appearance of new files, icons, installed applications that the user notices, can serve as an indicator of system compromise. Therefore, security guards need to be careful about incoming requests, complaints from company personnel and motivate employees to inform those responsible for the noted anomalies.