Where did the need for Threat Intelligence come from?

    According to statistics, daily the number of threats is growing rapidly. For example, according to the analysis conducted by the Pandalabs anti-virus laboratory at Panda Security for the 1st quarter of 2017, the number of new types of threats increases daily by 350,000.
    Attacks and compromises can be carried out in minutes, and the attackcompromisedata leakincident detectionResponse and Remedytakes days, weeks, and even months. And most often this happens after the attacker has compromised the data. At the same time, according to the annual report on Cisco information security, security specialists are able to process only 56% of incoming threat messages during their working day, and only every second one is recognized as justified among these threat alerts (i.e. 28%). Thus, 44% of incidents are ignored!

    At the same time, the market critically lacks not only the resources that can handle all incidents, but also a common system, thanks to which it would be possible to respond to them in the early stages of cyberattacks - ideally before exploitation, as well as to accumulate distributed knowledge about threats and share the received data , investigate the causes of threats and respond immediately to them. For faster accumulation of information about possible threats, one should strive to share useful data from a wide range of sources. At the same time, it is important that this information is standardized, that is, the standards and protocols for the transmission and provision of data are predefined.

    Threat tracking is one of the most important functions for effective business protection. TI - a system that allows you to learn about threats, attacks before they can harm you. In the event that an incident does occur, TI will allow you to respond, analyze and investigate, while expanding your knowledge base with context, mechanisms, indicators of compromise and analytics about existing or potential threats.

    Threat Intelligence Tasks




    Figure 1. Tasks of Threat Intelligence.

    Exploration and collection of data on vulnerabilities and threats.
    TI should be integrated into the protection system and should provide the ability to centrally collect information from public and private sources about vulnerabilities and threats. TI

    analytics
    should analyze and accumulate a knowledge base on the detection, disclosure, development and issuance of recommendations for responding to threats.

    Data exchange
    TI should also provide the ability to exchange received data in real time. Analytical information should be instantly disseminated in a standardized format to both internal and external safeguards.

    Quick alert
    TI must promptly notify about attacks and threats at any endpoint, using a single standardized database with classified data.

    Threat Intelligence Data Types


    TI involves working with three types of data: tactical, operational and strategic.

    Tactical
    Information about attackers: tools, tactics, techniques and procedures (TTP) that violators use, data about compromise indicators (IoC) - discrete data to detect signs of malicious activity in the infrastructure.

    Operational
    Data on current and predicted attacks, obtained by tracking new threat vectors, kill chain, methods of compromising information processes, etc.

    Strategic
    Analytical data on global threat trends with the further goal of developing a strategy for the development of information security systems.

    What you need to build Threat Intelligence processes


    To build processes for detecting threats and responding to them using measures based on information received from all available external and internal sources, it is necessary:

    • to determine the data sources (feeds) - where to get the initial data for indicators of compromise (both internal and external) ;

    • conduct internal analytics - within the organization, there may be a large number of specialists and experts in related departments who can identify and consolidate useful information in the direction;

    • introduce open standards and protocols for the transfer and provision of datafor effective communication between different data sources. At this stage, it is important that all threats are described, combined into compromise classes and successfully transferred.

    • Introduce a platform for data processing and analysis. It is necessary to analyze whether it will be its own or ready-made open-source solution, which may already include feeds, APIs, standards and protocols, be able to integrate with various systems, etc.

    Data sources


    Internal data sources
    First of all, it is possible to conduct analytics within the company regarding the possible implementation of data collection using, for example, SIEM or LM from internal information protection tools. Thus, useful data can be obtained:

    • anomalies in network traffic (Netflow / jFlow / sFlow);

    • activity from unusual IP addresses;

    • DNS queries;

    • URL and URI;

    • SMTP headers;

    • email addresses;

    • samples of malicious code;

    • user activity;

    • unsuccessful login attempts;

    • administrative access;

    • operations with the DBMS;

    • connections at atypical ports;

    • the emergence of atypical protocols;

    • mismatch of packet sizes for service protocols to standards;

    • addresses of anonymizers;

    • User Agent in HTTP;

    • malicious IP;

    • reputation of users, sites and files, etc.

    External data sources
    To expand the capabilities of security tools, it is necessary to identify external resources that can be accessed for information about IoC and other threats. At the same time, it is necessary to consider key factors when choosing feed sources depending on the needs of the company:

    • type of source;

    • support for various data formats (JSON, XML, CyBOX, STiX, CSV, etc.);

    • frequency of provision of information;

    • amount of data provided;

    • trust in the source that provides the data;

    • compliance with company infrastructure;

    • price.

    The most popular external feed sources


    Figure 2. The most popular external feed sources

    Internal analytics


    In addition to internal and external feeds, the company may conduct independent independent analytics in the presence of:

    • incident investigation specialists;

    • malware analytics experts;

    • specialists who track the “hot” news in the field of information security, for example:

    o compromised and infected sites;
    o phishing resources;
    o hashes of malicious files;
    o processes in which malicious code is detected;
    o registry keys, etc.

    • information that came to light in real time.

    All information found at this stage should be checked and tested by relevant experts and only after that should be entered into a single source. In addition to verification and testing, it is also important at this stage to formulate recommendations for mitigation of possible risks.

    TI Standards


    All identified threats should be described, standardized. It is also necessary to provide the ability to transmit information about them. Currently, there are a large number of open standards and protocols for solving these problems of providing and transmitting data. Consider the most popular:

    STIX (Structured Threat Information eXpression) - a standard used to provide unified information on cyberthreats (CTI). Allows you to share a description of various threats and related parameters in various areas. STIX provides unified incident information, including:

    o information objects (for example, creating a registry key, network traffic to specific IP addresses, sending email from a specific address, etc.);
    o indicators;
    o incidents;
    o tactics, methods, procedures of the attacked (attack patterns, malware, exploits, etc.);
    o operating objects (for example, vulnerabilities, security errors or incorrect configurations);
    o countermeasures (incident response or fixing security vulnerabilities / errors);
    o groups of cyber attacks (sets of incidents, TTP);
    o participants in cyberthreats (identification, enemy characteristics).

    The following is the STIX architecture:


    Figure 3. STIX architecture

    CybOX(Cyber ​​Observable eXpression) is a standard that provides a common structure for describing and presenting indicators of observed security events. Currently, over 70 different monitored objects have already been submitted: file, network connection, HTTP session, network traffic, X.509 certificate, etc.

    TLP (Traffic Light Protocol) - a protocol that allows you to “color” information in four colors, affecting who can receive the received information about threats:

    - information is not for distribution;

    - available only within the organization;

    - available only within a society or community;

    - available to everyone.

    IODEF(Incident Object Description and Exchange Format) (RFC 5070) - a standard that contains in XML format over 30 classes and subclasses of incidents, including information on contacts, financial damage, time, affected operating systems and applications, etc. IODEF is a well-developed standard and is already used in many places. IODEF-SCI (IODEF for Structured Cyber ​​Security Information) - an extension for IODEF that allows you to add additional data to IODEF: attack patterns, information about platforms, vulnerabilities, neutralization instructions, danger level, etc.

    OpenIOC(Indicator of Compromise) - an open standard for describing indicators of compromise. Built on the basis of XML and contains over 500 different indicators, mainly nodal (host) - file, driver, disk, process, registry, system, hash, etc.

    MISP - an open format for a structured description of indicators, information about threats, actors , a financial fraud, is based on JSON.

    VERIS (Vocabulary for Event Recording and Incident Sharing) - a standard for describing threats and incidents. The VERIS scheme consists of five parts:

    - Incident Tracking;



    - Victim Demographics;



    - Incident Description;



    - Discovery & Response;

    - Impact Assessment.

    TAXII(Trusted Automated Exchange of Intelligence Information) is a standard used to unify the methods for exchanging cyberthreat information (CTI) over the HTTPS protocol described using STIX.

    There are several ways to exchange data:

    - Hub and Spoke. The architecture assumes that one organization acts as a center for the exchange of information - a hub for all other participants in the interaction - spokes. Spoke shares information with a hub, which re-shares this information with the rest of the spokes.



    - Source / Subscriber. The architecture assumes that one organization acts as a source of information for all other companies.



    - Peer to Peer.Architecture assumes that the organization can act as both a producer and a consumer of information.



    TAXII may include the following services, which can be used together or separately:

    - Inbox: services for receiving received content.

    - Poll: a service for requesting content.

    - Collection Management: a service for working with data collections.

    - Discovery: Information about new supported services.

    VEDEF (Vulnerability and Exploit Description and Exchange Forma) - a standard for exchanging information about vulnerabilities and exploits.


    Figure 4. Vulnerability and Exploit Description and Exchange Forma


    Figure 5. Vulnerability and Exploit Description and Exchange Forma

    CAIF (COMMON ANNOUNCEMENT INTERCHANGE FORMAT) - XML-based standard for storing and exchanging security objects. It provides a basic set of elements designed to describe the main security issues. It is important that the set of elements can be expanded. It allows you to group information for more than one target group of readers, and also provides multilingual text descriptions in one document.

    MMDEF (The Malware Metadata Exchange Format) - a standard metadata exchange for malware.

    RID (Real-time Inter-network Defense) - a protocol that allows the interaction of various information security systems, built on the basis of HTTP / HTTPS.

    Vulnerability Management Standards


    MITER (not-for-profit organization that operates research and development centers sponsored by the federal government, operate FFRDCs):

    CVE (Common Vulnerabilities and Exposures) - a standard that defines a common naming of vulnerabilities.

    OVAL (Open Vulnerability and Assessment Language) - an open language for describing vulnerabilities in scanners and security analysis systems.

    CCE (Common Configuration Enumeration) - a standard for describing configurations that can be further verified in scanners and security analysis systems.

    CEE (Common Event Expression) - a standard for the description, storage and exchange of alarms between dissimilar means of protection.

    CME (Common Malware Enumeration) is a standard similar to CVE, but focused on malware.

    CWE (Common Weakness Enumeration) - a standardized set of weaknesses in software.

    CPE (Common Platform Enumeration) - a standard for describing and naming elements of IT infrastructure.

    CAPEC (Common Attack Pattern Enumeration and Classification) - standard classification of attack patterns.

    CRF (Common Result Format) - a standard for describing test results or evaluating security.

    SCAP(Security Content Automation Protocol) is a security data management automation protocol. It is a set of open standards defining technical specifications for the presentation and exchange of safety data.

    CVSS (Common Vulnerability Scoring System) - a standard for prioritizing vulnerabilities.

    Platform for data processing and analysis


    TI platforms are designed primarily to collect indicators of compromise from various sources. It is also necessary to classify and carry out further relevant actions.

    The choice of platform should directly depend on the scale of the TI system planned for implementation. For powerful TIs, you can consider such platforms, for example, as: Miter CRITs, Maltego, ThreatConnect, IBM i2, etc., for easier implementation, you can use open-source solutions, in this case giving preference to price rather than scale, functionality and support . Consider the most popular platforms:

    Anomali ThreatStream

    • A wide variety of feeds.


    Figure 6. A wide variety of feeds

    • Integration with many information security products and SIEM systems.


    Figure 7. Integration with many IS products and SIEM systems

    • Providing detailed information about the threat investigation.

    • Availability of API.

    MANTIS (Model-based Analysis of Threat Intelligence Sources)
    A threat management platform that allows you to import threat information obtained using the OpenIOC, IODEF, CybOX, STIX, TAXII standards. Example of imported STIX data:


    Figure 8. Example of imported STIX data


    Figure 9. Example of imported STIX

    CIF data (Collective Intelligence Framework)

    • Ability to collect and combine threat information from various sources that support CIF.

    • Using the information received to identify incidents.

    • Detection and neutralization of threats by generating rules for Snort, iptables and other means of protection.

    • Mostly works with IP addresses, domain names and URLs associated with malicious activity.

    • Uses IODEF as an information storage format.

    • Open-source platform.

    • Availability of feeds and API.


    Figure 10.

    IBM X-Force Exchange Collective Intelligence Framework

    • Threat analysis with dynamic updates every minute.

    • Track threats from more than 25 billion web pages and images.

    • It is supported by a database containing information on more than 96,000 threats.

    • Analysis of more than 8 million attacks using spam and phishing.

    • Tracking reputation data of 860,000 malicious IP addresses.

    • Integrates between IBM Security products and X-Force Exchange analytics.

    • Implemented the ability to associate threats with protection products.

    • Ability to integrate with other security solutions based on STIX and TAXII standards using RESTful.

    • Ability for subscribers to integrate threat analysis data from X-Force Exchange into their own operations, including the Corporate Security Center (SOC) or the development environment (DevOps).

    MISP

    • Opensource platform for creating, processing, sharing and collaboration with threat information.

    • Flexible automation capabilities, working with API.

    • Support for both native STIX format and STIX, OpenIOC, import text and csv data.

    • Support for the automatic secure exchange of threat information between different actors.

    • Automatic generation of rules for IDS, SIEM, Bro, Snort, Suricata, etc.

    Conclusion


    Cybercriminals have long been using someone else's experience to launch new, more sophisticated attacks. The information security industry has long developed as a closed one; nobody shared useful knowledge and experience to detect threats and prevent attacks.

    TI is a large knowledge base about threats and intruders, which accumulates information about the methods used by attackers to inflict harm, and how to counter them. TI works with dynamic information about the sources of threats and signs of compromise.

    Of course, with the help of TI it will not be possible to prevent all troubles, but it is the most powerful modern tool in response to cybercrime, which will help to quickly identify the directions in which the attacks are being conducted and to defend against them.

    Posted by Oksana Kotereva, Solution Promotion Manager
    Informzashchita company, o.kotereva@infosec.ru

    Also popular now: