Threat Hunting, or How to Defend Against 5% Threats
95% of information security threats are known, and you can protect yourself from them by traditional means such as antiviruses, firewalls, IDS, WAF. The remaining 5% of the threats are unknown and the most dangerous. They represent 70% of the risk for the company due to the fact that it is very difficult to detect them and even more so to protect themselves from them. Examples of “black swans” are the WannaCry, NotPetya / ExPetr ransomware epidemics, cryptominers, the Stuxnet cyber weapons (which hit Iran’s nuclear facilities) and many (who else remembers Kido / Conficker?) Other attacks that are not very good at defending against classic means of protection. We want to talk about how to counter these 5% of threats using Threat Hunting technology.
The continuous development of cyber attacks requires constant detection and counteraction, which ultimately leads us to the idea of an endless arms race between attackers and defenders. Classic protection systems are no longer able to provide an acceptable level of security at which the level of risk does not affect the key indicators of the company (economic, political, reputation) without finalizing them for a specific infrastructure, but in general they cover some of the risks. Already in the process of implementation and configuration, modern protection systems become catch-up and must respond to the challenges of modern times.
Source
One of the answers to the challenges of our time for an information security specialist may be Threat Hunting technology. The term Threat Hunting (hereinafter referred to as TH) appeared several years ago. The technology itself is quite interesting, but does not yet have any generally accepted standards and rules. The heterogeneity of information sources and the small number of Russian-language sources of information on this topic also complicates the matter. In this regard, we at LANIT-Integration decided to write some review of this technology.
TH technology relies on infrastructure monitoring processes. There are two main scenarios of internal monitoring - Alerting and Hunting . Alerting (by type of MSSP service) is a traditional method, searching for signatures and signs of attacks developed earlier and reacting to them. Traditional signature security features successfully complete this scenario. Hunting (a service of the MDR type) is a monitoring method that answers the question “Where do the signatures and rules come from?” This is the process of creating correlation rules by analyzing hidden or previously unknown indicators and signs of an attack. It is to this type of monitoring that Threat Hunting belongs.
Only by combining both types of monitoring, we get protection that is close to ideal, but there always remains some level of residual risk.
Protection using two types of monitoring
And this is why TH (and hunting entirely!) Will be more and more relevant:
Threats, remedies, risks. The source of
95% of all threats is already well understood . These include species such as spam, DDoS, viruses, rootkits and other classic malware. You can protect yourself from these threats with the same classic defenses.
During the implementation of any project, 20% of the time is 80% of the work.and the remaining 20% of the work takes 80% of the time. Similarly, among the entire threat landscape, 5% of the new type of threat will constitute 70% of the risk to the company. In a company where information security management processes are organized, we can manage 30% of the risk of known threats in one way or another by avoiding (abandoning wireless networks in principle), accepting (introducing the necessary security measures) or shifting (for example, to the shoulders of an integrator) this risk. Protecting yourself from zero-day vulnerabilities , APT attacks, phishing, attacks through the supply chain , cyber spyware and national operations, as well as from a large number of other attacks is already much more difficult. The consequences of these 5% threats will be much more serious (the average amount of bank losses from the buhtrap group is 143 million) than the consequences of spam or viruses from which antivirus software rescues.
Almost everyone has to deal with 5% of threats. Recently, we had to install one open-source solution that uses an application from the PEAR (PHP Extension and Application Repository) repository. An attempt to install this application through pear install failed, because the site was unavailable (now there is a stub on it), I had to install it from GitHub. And just recently it became clear that PEAR was the victim of an attack through the supply chain .
You can also recall the attack using CCleaner , an epidemic of the NePetya ransomware through the update module of the program for tax reporting MEDoc . Threats are becoming more sophisticated, and the logical question arises - "How do you still withstand these 5% of threats?"
So, Threat Hunting is a process of proactive and iterative search and detection of advanced threats that cannot be detected by traditional means of protection. Advanced threats include, for example, attacks such as APT, attacks on 0-day vulnerabilities, Living off the Land, and so on.
It can also be rephrased that TH is a hypothesis testing process. This is mainly a manual process with automation elements, in which the analyst, relying on his knowledge and qualifications, sift through large amounts of information in search of signs of compromise that correspond to the initially defined hypothesis about the presence of a certain threat. A distinctive feature of it is the variety of sources of information.
It should be noted that Threat Hunting is not some kind of software or hardware product. These are not alerts that can be seen in any solution. This is not a process for finding IOC (compromise identifiers). And this is not some kind of passive activity that goes without the participation of information security analysts. Threat Hunting is, first and foremost, a process.
Three main components of Threat Hunting: data, technology, people.
Data (what?) , Including Big Data. All kinds of traffic flows, information on previously conducted APTs, analytics, user activity data, network data, information from employees, information on the darknet and much more.
Technologies (how?) For processing this data are all possible ways of processing this data, including Machine Learning.
People (who?) Are those who have extensive experience in analyzing a variety of attacks, developed intuition and the ability to detect an attack. Usually these are information security analysts who must have the ability to generate hypotheses and find evidence for them. They are the main link in the process.
Adam Bateman describes the PARIS model for the ideal TH process. The name as it alludes to the famous landmark of France. This model can be considered in two directions - above and below.
In the process of hunting for threats, moving down the model, we will deal with a lot of evidence of malicious activity. Each evidence has a measure of confidence - a characteristic that reflects the weight of this evidence. There is “iron”, direct evidence of malicious activity, by which we can immediately reach the top of the pyramid and create an actual notification of a known infection. And there is indirect evidence, the sum of which can also lead us to the top of the pyramid. As always, there is much more indirect evidence than direct evidence, which means that they need to be sorted and analyzed, additional research should be carried out, and it is advisable to automate this.
Model PARIS. Source The
upper part of the model (1 and 2) is based on automation technologies and diverse analytics, and the lower part (3 and 4) is based on people with certain qualifications who control the process. You can consider the model, moving from top to bottom, where in the upper part of blue we have notifications from traditional means of protection (antivirus, EDR, firewall, signatures) with a high degree of confidence and trust, and below are indicators (IOC, URL, MD5 and others), which have less confidence and require further study. And the lowest and thickest level (4) is the generation of hypotheses, the creation of new scenarios of the work of traditional remedies. This level is not limited to the indicated sources of hypotheses. The lower the level, the more requirements are placed on the qualifications of the analyst.
It is very important that analysts do not just test a finite set of predefined hypotheses, but constantly work to generate new hypotheses and options for testing them.
In an ideal world, TH is an ongoing process. But, since there is no ideal world, we will analyze the maturity model and methods in the context of people, processes and used technologies. Consider a model of ideal spherical TH. There are 5 levels of using this technology. Consider them on the example of the evolution of a single team of analysts.
TH maturity levels by people, processes, and technology
Level 0: traditional, without TH. Conventional analysts work with a standard set of alerts in passive monitoring mode using standard tools and technologies: IDS, AV, sandboxes, signature analysis tools.
Level 1: experimental using TH. The same analysts with basic knowledge of forensics and good knowledge of networks and the application can implement one-time Threat Hunting by searching for indicators of compromise. EDRs with partial coverage of data from network devices are added to the tools. Tools are partially applied.
Level 2:intermittent, temporary TH. The same analysts who have already pumped their knowledge of forensics, networks and the application part are charged with the obligation to regularly engage in (sprint) Threat Hunting, say, a week a month. The tools are supplemented with a complete study of data from network devices, automation of data analysis from EDR and partial use of advanced EDR features.
Level 3:preventative, frequent cases of th. Our analysts organized themselves in a dedicated team, began to have excellent knowledge of forensics and malware, as well as knowledge of the methods and tactics of the attacking side. The process is already ongoing 24/7. The team is able to partially test the TH hypotheses, making full use of the advanced EDR capabilities with full coverage of data from network devices. Also, analysts are able to configure tools to fit their needs.
Level 4: high-end, the use of TH. The same team acquired the ability to research, the ability to generate and automate the process of testing hypotheses TH. Now, tight integration of data sources, software development for needs and non-standard use of APIs have been added to the tools.
Threat Hunting Basic Techniques
The TH techniques in order of maturity of the technology used include: basic search, statistical analysis, visualization techniques, simple aggregations, machine learning, and Bayesian methods.
The simplest method is a basic search, which is used to narrow the scope of research using specific queries. Statistical analysis is used, for example, to build a typical user or network activity in the form of a statistical model. Visualization techniques are used to visualize and simplify data analysis in the form of graphs and charts, which make it much easier to catch patterns in the sample. The simple aggregation technique for key fields is used to optimize search and analysis. The higher the level of maturity in an organization is achieved by the TH process, the more relevant is the use of machine learning algorithms. They are also widely used, including in filtering spam, detecting malicious traffic and detecting fraudulent activities.
Sergio Caltagiron, Andrew Pendegast and Christopher Betz in their work “ The Diamond Model of Intrusion Analysis ” showed the main key components of any malicious activity and the basic connection between them.
Diamond Model for Malicious Activity
According to this model, there are 4 Threat Hunting strategies that rely on the relevant key components.
1. A victim-oriented strategy. We assume that the victim has opponents, and they will deliver “opportunities” via email. We are looking for enemy data in the mail. Search for links, attachments, etc. We are looking for confirmation of this hypothesis for a certain period (month, two weeks), if not found, then the hypothesis did not play.
2. Infrastructure-oriented strategy. There are several ways to use this strategy. Depending on access and visibility, some are easier than others. For example, we monitor domain name servers known for hosting malicious domains. Or we are conducting a process of tracking all new domain name registrations for a known pattern used by the adversary.
3. Opportunity oriented strategy. In addition to the victim-oriented strategy used by most network advocates, there is an opportunity-oriented strategy. It is the second most popular and focuses on detecting opportunities from the adversary, namely “malware” and the ability of the adversary to use such legitimate tools as psexec, powershell, certutil and others.
4. Opponent-oriented strategy. The enemy-oriented approach focuses on the enemy. This includes the use of open information from public sources (OSINT), collecting data about the enemy, his techniques and methods (TTP), analysis of past incidents, threat intelligence data, etc.
Some sources of information for Threat Hunting There
can be many sources of information. The ideal analyst should be able to extract information from everything that is around. Typical sources in almost any infrastructure will be data from security features: DLP, SIEM, IDS / IPS, WAF / FW, EDR. Also, typical indicators of information will be all kinds of indicators of compromise, Threat Intelligence services, CERT and OSINT data. Additionally, you can use information from the darknet (for example, suddenly there is an order to hack the mailbox of the head of the organization, or the candidate for the position of network engineer has appeared on his activity), information received from HR (feedback about the candidate from his previous job), information from the security service ( e.g. counterparty verification results).
But before using all available sources, you must have at least one hypothesis.
Source
In order to test hypotheses, they must first be put forward. And in order to put forward many qualitative hypotheses, it is necessary to apply a systematic approach. The hypothesis generation process is described in more detail in the article ; it is very convenient to take this scheme as the basis for the hypothesis process.
The main source of hypotheses will be the ATT & CK matrix.(Adversarial Tactics, Techniques and Common Knowledge). It, in fact, is a knowledge base and model for assessing the behavior of attackers realizing their activities in the last steps of an attack, usually described using the concept of Kill Chain. That is, at the stages after the intruder penetrates the internal network of the enterprise or to a mobile device. Initially, the knowledge base included a description of 121 tactics and techniques used in the attack, each of which is described in detail in the Wiki format. A variety of Threat Intelligence analytics is well suited as a source for generating hypotheses. Of particular note are the results of infrastructure analysis and penetration tests - these are the most valuable data that iron hypotheses can give us because they rely on a specific infrastructure with its specific shortcomings.
Sergey Soldatov gave a good diagram with a detailed description of the process; it illustrates the process of testing the TH hypotheses in a single system. I will indicate the main stages with a brief description.
Source
Step 1: TI Farm
At this stage, you need to select objects (by analyzing them together with all the threat data) with assigning labels of their characteristics to them. This is a file, URL, MD5, process, utility, event. Passing them through Threat Intelligence systems needs to be tagged. That is, this site was spotted at CNC in such and such a year, this MD5 was associated with such and such a malware, this MD5 was downloaded from the website that distributed the malvars.
Step 2: Cases
In a second step we look at the interaction between the objects and to identify the relationship between all these objects. We get labeled systems that do something bad.
Stage 3: Analyst
At the third stage, the case is transferred to an experienced analyst who has vast experience in analysis, and he makes a verdict. It parses to bytes what, where, how, why and why this code does. This body was a malware, this computer was infected. Discloses connections between objects, checks the results of a run through the sandbox.
The results of the analyst’s work are passed on. Digital Forensics examines the images, Malware Analysis examines the found “bodies”, and the Incident Response team can go to the site and explore something already there. The result of the work will be a confirmed hypothesis, an identified attack and ways to counter it.
A source
Threat Hunting is a fairly young technology capable of effectively resisting customized, new and non-standard threats, which has great prospects given the growing number of such threats and the complexity of corporate infrastructure. It needs three components — data, tools, and analytics. The benefits of Threat Hunting are not limited to proactively implementing threats. Do not forget that in the search process we plunge into our infrastructure and its weaknesses through the eyes of a security analyst and can further strengthen these places.
The first steps that, in our opinion, need to be taken to initiate the TH process in your organization.
Ready to discuss the organization of the TH process in the comments.
The continuous development of cyber attacks requires constant detection and counteraction, which ultimately leads us to the idea of an endless arms race between attackers and defenders. Classic protection systems are no longer able to provide an acceptable level of security at which the level of risk does not affect the key indicators of the company (economic, political, reputation) without finalizing them for a specific infrastructure, but in general they cover some of the risks. Already in the process of implementation and configuration, modern protection systems become catch-up and must respond to the challenges of modern times.
One of the answers to the challenges of our time for an information security specialist may be Threat Hunting technology. The term Threat Hunting (hereinafter referred to as TH) appeared several years ago. The technology itself is quite interesting, but does not yet have any generally accepted standards and rules. The heterogeneity of information sources and the small number of Russian-language sources of information on this topic also complicates the matter. In this regard, we at LANIT-Integration decided to write some review of this technology.
Relevance
TH technology relies on infrastructure monitoring processes. There are two main scenarios of internal monitoring - Alerting and Hunting . Alerting (by type of MSSP service) is a traditional method, searching for signatures and signs of attacks developed earlier and reacting to them. Traditional signature security features successfully complete this scenario. Hunting (a service of the MDR type) is a monitoring method that answers the question “Where do the signatures and rules come from?” This is the process of creating correlation rules by analyzing hidden or previously unknown indicators and signs of an attack. It is to this type of monitoring that Threat Hunting belongs.
Only by combining both types of monitoring, we get protection that is close to ideal, but there always remains some level of residual risk.
And this is why TH (and hunting entirely!) Will be more and more relevant:
95% of all threats is already well understood . These include species such as spam, DDoS, viruses, rootkits and other classic malware. You can protect yourself from these threats with the same classic defenses.
During the implementation of any project, 20% of the time is 80% of the work.and the remaining 20% of the work takes 80% of the time. Similarly, among the entire threat landscape, 5% of the new type of threat will constitute 70% of the risk to the company. In a company where information security management processes are organized, we can manage 30% of the risk of known threats in one way or another by avoiding (abandoning wireless networks in principle), accepting (introducing the necessary security measures) or shifting (for example, to the shoulders of an integrator) this risk. Protecting yourself from zero-day vulnerabilities , APT attacks, phishing, attacks through the supply chain , cyber spyware and national operations, as well as from a large number of other attacks is already much more difficult. The consequences of these 5% threats will be much more serious (the average amount of bank losses from the buhtrap group is 143 million) than the consequences of spam or viruses from which antivirus software rescues.
Almost everyone has to deal with 5% of threats. Recently, we had to install one open-source solution that uses an application from the PEAR (PHP Extension and Application Repository) repository. An attempt to install this application through pear install failed, because the site was unavailable (now there is a stub on it), I had to install it from GitHub. And just recently it became clear that PEAR was the victim of an attack through the supply chain .
You can also recall the attack using CCleaner , an epidemic of the NePetya ransomware through the update module of the program for tax reporting MEDoc . Threats are becoming more sophisticated, and the logical question arises - "How do you still withstand these 5% of threats?"
Threat Hunting Definition
So, Threat Hunting is a process of proactive and iterative search and detection of advanced threats that cannot be detected by traditional means of protection. Advanced threats include, for example, attacks such as APT, attacks on 0-day vulnerabilities, Living off the Land, and so on.
It can also be rephrased that TH is a hypothesis testing process. This is mainly a manual process with automation elements, in which the analyst, relying on his knowledge and qualifications, sift through large amounts of information in search of signs of compromise that correspond to the initially defined hypothesis about the presence of a certain threat. A distinctive feature of it is the variety of sources of information.
It should be noted that Threat Hunting is not some kind of software or hardware product. These are not alerts that can be seen in any solution. This is not a process for finding IOC (compromise identifiers). And this is not some kind of passive activity that goes without the participation of information security analysts. Threat Hunting is, first and foremost, a process.
Threat Hunting Components
Three main components of Threat Hunting: data, technology, people.
Data (what?) , Including Big Data. All kinds of traffic flows, information on previously conducted APTs, analytics, user activity data, network data, information from employees, information on the darknet and much more.
Technologies (how?) For processing this data are all possible ways of processing this data, including Machine Learning.
People (who?) Are those who have extensive experience in analyzing a variety of attacks, developed intuition and the ability to detect an attack. Usually these are information security analysts who must have the ability to generate hypotheses and find evidence for them. They are the main link in the process.
Model PARIS
Adam Bateman describes the PARIS model for the ideal TH process. The name as it alludes to the famous landmark of France. This model can be considered in two directions - above and below.
In the process of hunting for threats, moving down the model, we will deal with a lot of evidence of malicious activity. Each evidence has a measure of confidence - a characteristic that reflects the weight of this evidence. There is “iron”, direct evidence of malicious activity, by which we can immediately reach the top of the pyramid and create an actual notification of a known infection. And there is indirect evidence, the sum of which can also lead us to the top of the pyramid. As always, there is much more indirect evidence than direct evidence, which means that they need to be sorted and analyzed, additional research should be carried out, and it is advisable to automate this.
upper part of the model (1 and 2) is based on automation technologies and diverse analytics, and the lower part (3 and 4) is based on people with certain qualifications who control the process. You can consider the model, moving from top to bottom, where in the upper part of blue we have notifications from traditional means of protection (antivirus, EDR, firewall, signatures) with a high degree of confidence and trust, and below are indicators (IOC, URL, MD5 and others), which have less confidence and require further study. And the lowest and thickest level (4) is the generation of hypotheses, the creation of new scenarios of the work of traditional remedies. This level is not limited to the indicated sources of hypotheses. The lower the level, the more requirements are placed on the qualifications of the analyst.
It is very important that analysts do not just test a finite set of predefined hypotheses, but constantly work to generate new hypotheses and options for testing them.
TH use maturity model
In an ideal world, TH is an ongoing process. But, since there is no ideal world, we will analyze the maturity model and methods in the context of people, processes and used technologies. Consider a model of ideal spherical TH. There are 5 levels of using this technology. Consider them on the example of the evolution of a single team of analysts.
| Maturity levels | People | The processes | Technology |
| Level 0 | SOC Analysts | 24/7 | Traditional instruments: |
| Traditional | Alert Set | Passive monitoring | IDS, AV, Sandboxing, |
| Without TH | Work with alerts | signature analysis tools, Threat Intelligence data. | |
| Level 1 | SOC Analysts | One-time TH | EDR |
| Experimental | Basic knowledge of forensics | IOC Search | Partial coverage of data from network devices |
| Experiments with TH | Good knowledge of networks and application | Partial application | |
| Level 2 | Temporary occupation | Sprints | EDR |
| Periodic | The average knowledge of forensics | Week per month | Full application |
| Temporary TH | Excellent knowledge of networks and application | Regular th | Full automation of EDR data usage |
| Partial use of advanced EDR features | |||
| Level 3 | Dedicated TH Team | 24/7 | Partial ability to test hypotheses TH |
| Preventive | Excellent knowledge of forensics and malware | Proactive TH | Full use of advanced EDR features |
| Special cases TH | Excellent knowledge of the attacker | Special cases TH | Full coverage of data from network devices |
| Custom configuration | |||
| Level 4 | Dedicated TH Team | 24/7 | Full ability to test TH hypotheses |
| Leading | Excellent knowledge of forensics and malware | Proactive TH | Level 3, plus: |
| Using TH | Excellent knowledge of the attacker | Testing, Automating, and Verifying TH Hypotheses | tight integration of data sources; |
| Research ability | custom development and custom API use. |
Level 0: traditional, without TH. Conventional analysts work with a standard set of alerts in passive monitoring mode using standard tools and technologies: IDS, AV, sandboxes, signature analysis tools.
Level 1: experimental using TH. The same analysts with basic knowledge of forensics and good knowledge of networks and the application can implement one-time Threat Hunting by searching for indicators of compromise. EDRs with partial coverage of data from network devices are added to the tools. Tools are partially applied.
Level 2:intermittent, temporary TH. The same analysts who have already pumped their knowledge of forensics, networks and the application part are charged with the obligation to regularly engage in (sprint) Threat Hunting, say, a week a month. The tools are supplemented with a complete study of data from network devices, automation of data analysis from EDR and partial use of advanced EDR features.
Level 3:preventative, frequent cases of th. Our analysts organized themselves in a dedicated team, began to have excellent knowledge of forensics and malware, as well as knowledge of the methods and tactics of the attacking side. The process is already ongoing 24/7. The team is able to partially test the TH hypotheses, making full use of the advanced EDR capabilities with full coverage of data from network devices. Also, analysts are able to configure tools to fit their needs.
Level 4: high-end, the use of TH. The same team acquired the ability to research, the ability to generate and automate the process of testing hypotheses TH. Now, tight integration of data sources, software development for needs and non-standard use of APIs have been added to the tools.
Threat Hunting Techniques
The TH techniques in order of maturity of the technology used include: basic search, statistical analysis, visualization techniques, simple aggregations, machine learning, and Bayesian methods.
The simplest method is a basic search, which is used to narrow the scope of research using specific queries. Statistical analysis is used, for example, to build a typical user or network activity in the form of a statistical model. Visualization techniques are used to visualize and simplify data analysis in the form of graphs and charts, which make it much easier to catch patterns in the sample. The simple aggregation technique for key fields is used to optimize search and analysis. The higher the level of maturity in an organization is achieved by the TH process, the more relevant is the use of machine learning algorithms. They are also widely used, including in filtering spam, detecting malicious traffic and detecting fraudulent activities.
Diamond Model and TH Strategy
Sergio Caltagiron, Andrew Pendegast and Christopher Betz in their work “ The Diamond Model of Intrusion Analysis ” showed the main key components of any malicious activity and the basic connection between them.
According to this model, there are 4 Threat Hunting strategies that rely on the relevant key components.
1. A victim-oriented strategy. We assume that the victim has opponents, and they will deliver “opportunities” via email. We are looking for enemy data in the mail. Search for links, attachments, etc. We are looking for confirmation of this hypothesis for a certain period (month, two weeks), if not found, then the hypothesis did not play.
2. Infrastructure-oriented strategy. There are several ways to use this strategy. Depending on access and visibility, some are easier than others. For example, we monitor domain name servers known for hosting malicious domains. Or we are conducting a process of tracking all new domain name registrations for a known pattern used by the adversary.
3. Opportunity oriented strategy. In addition to the victim-oriented strategy used by most network advocates, there is an opportunity-oriented strategy. It is the second most popular and focuses on detecting opportunities from the adversary, namely “malware” and the ability of the adversary to use such legitimate tools as psexec, powershell, certutil and others.
4. Opponent-oriented strategy. The enemy-oriented approach focuses on the enemy. This includes the use of open information from public sources (OSINT), collecting data about the enemy, his techniques and methods (TTP), analysis of past incidents, threat intelligence data, etc.
Sources of information and hypotheses in TH
can be many sources of information. The ideal analyst should be able to extract information from everything that is around. Typical sources in almost any infrastructure will be data from security features: DLP, SIEM, IDS / IPS, WAF / FW, EDR. Also, typical indicators of information will be all kinds of indicators of compromise, Threat Intelligence services, CERT and OSINT data. Additionally, you can use information from the darknet (for example, suddenly there is an order to hack the mailbox of the head of the organization, or the candidate for the position of network engineer has appeared on his activity), information received from HR (feedback about the candidate from his previous job), information from the security service ( e.g. counterparty verification results).
But before using all available sources, you must have at least one hypothesis.
In order to test hypotheses, they must first be put forward. And in order to put forward many qualitative hypotheses, it is necessary to apply a systematic approach. The hypothesis generation process is described in more detail in the article ; it is very convenient to take this scheme as the basis for the hypothesis process.
The main source of hypotheses will be the ATT & CK matrix.(Adversarial Tactics, Techniques and Common Knowledge). It, in fact, is a knowledge base and model for assessing the behavior of attackers realizing their activities in the last steps of an attack, usually described using the concept of Kill Chain. That is, at the stages after the intruder penetrates the internal network of the enterprise or to a mobile device. Initially, the knowledge base included a description of 121 tactics and techniques used in the attack, each of which is described in detail in the Wiki format. A variety of Threat Intelligence analytics is well suited as a source for generating hypotheses. Of particular note are the results of infrastructure analysis and penetration tests - these are the most valuable data that iron hypotheses can give us because they rely on a specific infrastructure with its specific shortcomings.
Hypothesis Testing Process
Sergey Soldatov gave a good diagram with a detailed description of the process; it illustrates the process of testing the TH hypotheses in a single system. I will indicate the main stages with a brief description.
Step 1: TI Farm
At this stage, you need to select objects (by analyzing them together with all the threat data) with assigning labels of their characteristics to them. This is a file, URL, MD5, process, utility, event. Passing them through Threat Intelligence systems needs to be tagged. That is, this site was spotted at CNC in such and such a year, this MD5 was associated with such and such a malware, this MD5 was downloaded from the website that distributed the malvars.
Step 2: Cases
In a second step we look at the interaction between the objects and to identify the relationship between all these objects. We get labeled systems that do something bad.
Stage 3: Analyst
At the third stage, the case is transferred to an experienced analyst who has vast experience in analysis, and he makes a verdict. It parses to bytes what, where, how, why and why this code does. This body was a malware, this computer was infected. Discloses connections between objects, checks the results of a run through the sandbox.
The results of the analyst’s work are passed on. Digital Forensics examines the images, Malware Analysis examines the found “bodies”, and the Incident Response team can go to the site and explore something already there. The result of the work will be a confirmed hypothesis, an identified attack and ways to counter it.
Summary
Threat Hunting is a fairly young technology capable of effectively resisting customized, new and non-standard threats, which has great prospects given the growing number of such threats and the complexity of corporate infrastructure. It needs three components — data, tools, and analytics. The benefits of Threat Hunting are not limited to proactively implementing threats. Do not forget that in the search process we plunge into our infrastructure and its weaknesses through the eyes of a security analyst and can further strengthen these places.
The first steps that, in our opinion, need to be taken to initiate the TH process in your organization.
- Take care of endpoint security and network infrastructure. Take care of visibility (NetFlow) and control (firewall, IDS, IPS, DLP) of all processes in your network. Know your network from the edge router to the very last host.
- Explore MITER ATT & CK .
- Conduct a regular pentest of at least key external resources, analyze its results, identify the main objectives for the attack and close their vulnerabilities.
- Implement an open source Threat Intelligence system (for example, MISP, Yeti) and analyze logs with it.
- Implement an incident response platform (IRP): R-Vision IRP, The Hive, a sandbox for analyzing suspicious files (FortiSandbox, Cuckoo).
- Automate routine processes. Log analysis, incident management, staff informing is a huge field for automation.
- Learn how to effectively interact with engineers, developers, technical support for collaborating on incidents.
- Document the whole process, key points, achieved results, to return to them later or share this data with colleagues;
- Remember the social side: be aware of what is happening with your employees, whom you hire and who give access to the organization’s information resources.
- Keep abreast of trends in the field of new threats and protection methods, increase your level of technical literacy (including the work of IT services and subsystems), attend conferences and communicate with colleagues.
Ready to discuss the organization of the TH process in the comments.
Or come to us to work!