SOC is people. “Hello, we are looking for talents” or where do analysts from the center for monitoring and responding to cyber attacks come from
Today, even a cursory search on hh.ru yields about 90 different job and job functionalities with the magic word “analyst” and fairly decent payment terms. Before the eyes of many candidates, big data and machine learning are at once passing, the salary starts to dance much higher than the market and flirt with zeros. So who are the monitoring center analysts who are “responsible for ensuring that the customer is not hacked”? What do they do and what they need to know and be able to get to this position?
In previous articles, we said that the list of the main tasks of the analyst 3 lines include:
To summarize, the analyst is responsible for the technical aspects of monitoring cyber threats at the Customer. The source did not send logs, the event did not take place, the script did not work or the script failed, the attack was missed - the analyst assigned to the Customer is responsible for everything.
However, this does not mean that all Solar JSOC analysts are gray or bald by the age of 30. Not all. Just this role implies high demands on its performer. Let's try to paint them in a bit more detail. Immediately, we note that in this article we deliberately did not focus on the technical competencies that we expect from the candidate for the role of analyst of Solar JSOC. A lot has been said about the technique, but, as written in the title of the cycle of our articles, SOC is people.
We will not focus attention, but it’s impossible not to say a few words about SIEM :) In the description of a vacancy, they often write: “Experience with a SIEM system”. On the one hand, everything is clear: SIEM is a SOC engine, without it, the service, as they say, "will not go." (Some experts have objections and their own right to life, the theory of building SOC without SIEM, but still this is not the topic of our article.)
However, in fact, these words consist of something more than the ability to look at the logs of a certain IT system.
The analyst should be able to model attack vectors based on the minimum amount of information about the Customer’s infrastructure. Of course, it happens that when the Customer connects, we receive from him full information on the L2-L3 subnets, a list of servers and workstations, indicating their roles, uploading from AD and SCCM, etc. And among the Solar JSOC experts, there is even a legend that there was once a Customer who provided all this information up to date ... But, unfortunately, this is not always the case, and we have to work with what we have. This means that you need to be able to assess the adequacy of the connected sources and the events received to provide quality service for monitoring and identifying information security incidents. Obviously, for this purpose, the specialist must have a strong background on the basic IT technologies used to build the typical infrastructure of the company.
In parallel, the analyst should be able to use old sources to solve new (in this case, read - non-core) tasks. For example, one of our Customer-Bank, which has an extensive ATM network throughout the country, had an acute problem: the anti-virus solution used did not allow us to assess the completeness of coverage of these same ATMs with anti-virus software. However, we had a kernel-level firewall connected, and we knew with which processing service the ATMs interact. Using these logs, the responsible analyst was able to prepare a list of IP addresses of ATMs that are knocking on the processing, and at the same time there is no information in the database of the control center of the anti-virus solution about the presence of the agent. For several months of joint intensive work, we managed to reduce the list of such ATMs from several hundred to a few, and the task of inventory,
Corruption and attention to trifles are very useful for the analyst. Investigating incidents that were not recorded by the Solar JSOC scripting pool that was launched is a very complex, routine work with thousands, if not millions of events from various sources. And here the most difficult thing is to find the thread, pulling which will be able to unravel the whole tangle of the incident.
For example, we had a case when an analyst investigated unauthorized penetration into the infrastructure of the Customer and he could not manage to find the initial point of compromise. To solve the problem, we had to build a monthly report on incoming and outgoing network connections with the participation of IP addresses belonging to the Customer’s external address pool. And only after a long analysis of this report, it was possible to find atypical outgoing connections from a test web server to an IP address from the Netherlands, which eventually turned out to be Reverse Shell activity launched by an attacker on a compromised server.
Some of the tasks of the analyst require direct communication with the customer. Sometimes information has to be pulled out of it literally by ticks, for example, when a request arrives in the form of “what was suspicious on such an automated workplace last week?”. In fact, after a series of leading questions, it turns out that the employee who worked on this automated workplace in the smoking room complained to a colleague from the information security unit that a file was missing on his desktop. And then the Bezopasnik decided to ask the external SOC what this was connected with, but the wording of the question was too vague. And this happens all the time. It is difficult to overestimate the notorious ability to work in a team, namely in conjunction with a service manager. To provide quality service, it is important that both pull the team in one direction, and not like in one famous fable.
Separately, it is worth noting a character trait that has become so familiar to all resumes that no one pays attention to it anymore. It is about stress tolerance. Solar JSOC provides a 24-by-7 service, which means that all analysts are involved in round-the-clock duty, ready to join in an investigation into an important incident at any time. At the same time, as statistics show , a considerable part of critical incidents occurs exactly during off-hours. The ability to wake up several times a night comes to the fore, and the brain should start up and be ready to perceive the most important information almost instantly.
Investigation of all recorded incidents is carried out by engineers of the first line of monitoring. The task of the analyst is to connect during an escalation, as well as to monitor the quality of the investigation of incidents worked out by the first line. Moreover, engineers often turn to the analyst with a request to help interpret the events or assess the criticality of the incident. This means that the analyst should direct his junior colleagues, monitor the progress in the quality of the investigation and give a first-line feedback to the team.
Also, often the customer asks to provide this or that information on the events. The analyst must evaluate the task, correctly interpret it and transfer it to the engineers of the first line for implementation, in whole or in part, depending on the level of complexity of the task. Here it is important not to close all technical activity on yourself and in time delegate autonomous tasks to the first line as a scalable resource. As an example of such tasks, you can cite requests like “it is necessary to upload information about the activity of employee N on certain hosts” or “please provide information about network interaction with address xxxx for the last month”. As you can see, the requests are quite simple, but their implementation in the SIEM takes a certain amount of time, and this is completely accomplished by the forces of the first line.
How does the replenishment of the Solar JSOC analyst ranks? I wish everything was as simple as in the picture, but alas.
If you do not consider hiring people from the side, as well as a horizontal transition, then the most natural way to the analyst is to grow from a response engineer (for more information about this role in the JSOC gang, you can read here and here ). "And only this is logical," as the famous character said.
The response engineer, most likely, grew out of the first line of monitoring, which means it went through a difficult way of investigating the unceasing stream of incidents, maneuvering between False Positive Scylla and False Negative Charibda. In addition, the engineer has already acquired the skills of more complex investigations, in-depth work with SIEM, connection of event sources, as well as solving specific problems of Customers. In general, I have mastered the foundation necessary for further growth.
But is this enough to go to analytics? Complex issue. And usually there is no universal answer to it. At a minimum, the analyst has a new duty in comparison with the response engineer - interaction with the Customer. This will seem trivial to many, but practice has shown that this is far from the case. Many guys, with their head immersed in IT, have to work hard on themselves in order to overcome fear and learn to communicate with the people to whom we provide the service. On some very heavy pressure load of responsibility. It is psychologically difficult for others to accept that, as an analyst, there will no longer be elder comrades who will recheck after you and point out errors. For many, it's just too much stress - when you do atypical tasks, each of which turns out to be a challenge to your skills, when several solutions in a row are a dead end. Many then simply give up. So the human qualities here play an important role.
As translation tasks for the post of analyst, we usually offer two types of tasks. One of them is the task of developing JSOC content, for example, developing a block of scenarios for detecting new attack vectors. From fresh - implementation of detecting attacks on Active Directory, in particular DCShadow.
In addition to working with content, the analyst is assigned responsible for two or three Solar JSOC Customers during the translation process: examines their infrastructure, connected sources and events received from them, verifies the completeness of the connected systems and scopes of running scripts, proceeds to monitor detected incidents and the quality of first line engineers on these incidents. After the end of acceptance, all questions regarding the technical side of the service for this Customer are transferred to the area of responsibility of the new analyst.
The team of analysts have graduation positions. The junior analyst is learning a new role for himself and is engaged in typical tasks. The analyst is the main shock force of JSOC, covering the main task pool. Separately, I would like to say about the role of the senior analyst. As the name implies, the senior analyst does an excellent job with his main tasks, while he has an understanding of managing Solar JSOC services, is able to assess business risks, has a high level of communication, is able to work out a non-standard service architecture, if necessary, etc. Thus, in the person of such an employee, we have an autonomous combat unit, which can replace the service manager for the period of his absence without loss of quality.
But what happens next with an employee who has climbed a step called the Analyst? The Solar JSOC development ladder does not end there.
You can focus on the development and “dig deep”, improving your knowledge and skills of the analyst of the monitoring center, gradually becoming a hardened expert who does not care about the level of complexity of the tasks to be solved.
You can do the optimization of the work of analysts, as well as supervise the guys starting to work in this position. In other words, gradually advance to the role of local timlid.
And you can try to join the ranks of classic managers and take upon yourself the burden of a service manager, engaging in such difficult tasks as monitoring SLA compliance, managing the Solar JSOC service, and interacting with the Customer in terms of the level of service provided.
We are trying to help each employee determine the most appropriate development vector for him and find himself in the Solar JSOC structure.
In previous articles, we said that the list of the main tasks of the analyst 3 lines include:
- Analysis of anomalous activities to identify incidents.
- Response to atypical critical incidents of their customers.
- Participation in the investigation of IS incidents not recorded by monitoring
- Technical survey, connection and adaptation of event sources.
- Development of new incident detection scenarios.
To summarize, the analyst is responsible for the technical aspects of monitoring cyber threats at the Customer. The source did not send logs, the event did not take place, the script did not work or the script failed, the attack was missed - the analyst assigned to the Customer is responsible for everything.
However, this does not mean that all Solar JSOC analysts are gray or bald by the age of 30. Not all. Just this role implies high demands on its performer. Let's try to paint them in a bit more detail. Immediately, we note that in this article we deliberately did not focus on the technical competencies that we expect from the candidate for the role of analyst of Solar JSOC. A lot has been said about the technique, but, as written in the title of the cycle of our articles, SOC is people.
Fight and search
We will not focus attention, but it’s impossible not to say a few words about SIEM :) In the description of a vacancy, they often write: “Experience with a SIEM system”. On the one hand, everything is clear: SIEM is a SOC engine, without it, the service, as they say, "will not go." (Some experts have objections and their own right to life, the theory of building SOC without SIEM, but still this is not the topic of our article.)
However, in fact, these words consist of something more than the ability to look at the logs of a certain IT system.
The analyst should be able to model attack vectors based on the minimum amount of information about the Customer’s infrastructure. Of course, it happens that when the Customer connects, we receive from him full information on the L2-L3 subnets, a list of servers and workstations, indicating their roles, uploading from AD and SCCM, etc. And among the Solar JSOC experts, there is even a legend that there was once a Customer who provided all this information up to date ... But, unfortunately, this is not always the case, and we have to work with what we have. This means that you need to be able to assess the adequacy of the connected sources and the events received to provide quality service for monitoring and identifying information security incidents. Obviously, for this purpose, the specialist must have a strong background on the basic IT technologies used to build the typical infrastructure of the company.
In parallel, the analyst should be able to use old sources to solve new (in this case, read - non-core) tasks. For example, one of our Customer-Bank, which has an extensive ATM network throughout the country, had an acute problem: the anti-virus solution used did not allow us to assess the completeness of coverage of these same ATMs with anti-virus software. However, we had a kernel-level firewall connected, and we knew with which processing service the ATMs interact. Using these logs, the responsible analyst was able to prepare a list of IP addresses of ATMs that are knocking on the processing, and at the same time there is no information in the database of the control center of the anti-virus solution about the presence of the agent. For several months of joint intensive work, we managed to reduce the list of such ATMs from several hundred to a few, and the task of inventory,
Find and do not give up
Corruption and attention to trifles are very useful for the analyst. Investigating incidents that were not recorded by the Solar JSOC scripting pool that was launched is a very complex, routine work with thousands, if not millions of events from various sources. And here the most difficult thing is to find the thread, pulling which will be able to unravel the whole tangle of the incident.
For example, we had a case when an analyst investigated unauthorized penetration into the infrastructure of the Customer and he could not manage to find the initial point of compromise. To solve the problem, we had to build a monthly report on incoming and outgoing network connections with the participation of IP addresses belonging to the Customer’s external address pool. And only after a long analysis of this report, it was possible to find atypical outgoing connections from a test web server to an IP address from the Netherlands, which eventually turned out to be Reverse Shell activity launched by an attacker on a compromised server.
Some of the tasks of the analyst require direct communication with the customer. Sometimes information has to be pulled out of it literally by ticks, for example, when a request arrives in the form of “what was suspicious on such an automated workplace last week?”. In fact, after a series of leading questions, it turns out that the employee who worked on this automated workplace in the smoking room complained to a colleague from the information security unit that a file was missing on his desktop. And then the Bezopasnik decided to ask the external SOC what this was connected with, but the wording of the question was too vague. And this happens all the time. It is difficult to overestimate the notorious ability to work in a team, namely in conjunction with a service manager. To provide quality service, it is important that both pull the team in one direction, and not like in one famous fable.
Character resistant, nordic
Separately, it is worth noting a character trait that has become so familiar to all resumes that no one pays attention to it anymore. It is about stress tolerance. Solar JSOC provides a 24-by-7 service, which means that all analysts are involved in round-the-clock duty, ready to join in an investigation into an important incident at any time. At the same time, as statistics show , a considerable part of critical incidents occurs exactly during off-hours. The ability to wake up several times a night comes to the fore, and the brain should start up and be ready to perceive the most important information almost instantly.
Investigation of all recorded incidents is carried out by engineers of the first line of monitoring. The task of the analyst is to connect during an escalation, as well as to monitor the quality of the investigation of incidents worked out by the first line. Moreover, engineers often turn to the analyst with a request to help interpret the events or assess the criticality of the incident. This means that the analyst should direct his junior colleagues, monitor the progress in the quality of the investigation and give a first-line feedback to the team.
Also, often the customer asks to provide this or that information on the events. The analyst must evaluate the task, correctly interpret it and transfer it to the engineers of the first line for implementation, in whole or in part, depending on the level of complexity of the task. Here it is important not to close all technical activity on yourself and in time delegate autonomous tasks to the first line as a scalable resource. As an example of such tasks, you can cite requests like “it is necessary to upload information about the activity of employee N on certain hosts” or “please provide information about network interaction with address xxxx for the last month”. As you can see, the requests are quite simple, but their implementation in the SIEM takes a certain amount of time, and this is completely accomplished by the forces of the first line.
"... let them teach me"
How does the replenishment of the Solar JSOC analyst ranks? I wish everything was as simple as in the picture, but alas.
If you do not consider hiring people from the side, as well as a horizontal transition, then the most natural way to the analyst is to grow from a response engineer (for more information about this role in the JSOC gang, you can read here and here ). "And only this is logical," as the famous character said.
The response engineer, most likely, grew out of the first line of monitoring, which means it went through a difficult way of investigating the unceasing stream of incidents, maneuvering between False Positive Scylla and False Negative Charibda. In addition, the engineer has already acquired the skills of more complex investigations, in-depth work with SIEM, connection of event sources, as well as solving specific problems of Customers. In general, I have mastered the foundation necessary for further growth.
But is this enough to go to analytics? Complex issue. And usually there is no universal answer to it. At a minimum, the analyst has a new duty in comparison with the response engineer - interaction with the Customer. This will seem trivial to many, but practice has shown that this is far from the case. Many guys, with their head immersed in IT, have to work hard on themselves in order to overcome fear and learn to communicate with the people to whom we provide the service. On some very heavy pressure load of responsibility. It is psychologically difficult for others to accept that, as an analyst, there will no longer be elder comrades who will recheck after you and point out errors. For many, it's just too much stress - when you do atypical tasks, each of which turns out to be a challenge to your skills, when several solutions in a row are a dead end. Many then simply give up. So the human qualities here play an important role.
As translation tasks for the post of analyst, we usually offer two types of tasks. One of them is the task of developing JSOC content, for example, developing a block of scenarios for detecting new attack vectors. From fresh - implementation of detecting attacks on Active Directory, in particular DCShadow.
In addition to working with content, the analyst is assigned responsible for two or three Solar JSOC Customers during the translation process: examines their infrastructure, connected sources and events received from them, verifies the completeness of the connected systems and scopes of running scripts, proceeds to monitor detected incidents and the quality of first line engineers on these incidents. After the end of acceptance, all questions regarding the technical side of the service for this Customer are transferred to the area of responsibility of the new analyst.
The team of analysts have graduation positions. The junior analyst is learning a new role for himself and is engaged in typical tasks. The analyst is the main shock force of JSOC, covering the main task pool. Separately, I would like to say about the role of the senior analyst. As the name implies, the senior analyst does an excellent job with his main tasks, while he has an understanding of managing Solar JSOC services, is able to assess business risks, has a high level of communication, is able to work out a non-standard service architecture, if necessary, etc. Thus, in the person of such an employee, we have an autonomous combat unit, which can replace the service manager for the period of his absence without loss of quality.
But what happens next with an employee who has climbed a step called the Analyst? The Solar JSOC development ladder does not end there.
You can focus on the development and “dig deep”, improving your knowledge and skills of the analyst of the monitoring center, gradually becoming a hardened expert who does not care about the level of complexity of the tasks to be solved.
You can do the optimization of the work of analysts, as well as supervise the guys starting to work in this position. In other words, gradually advance to the role of local timlid.
And you can try to join the ranks of classic managers and take upon yourself the burden of a service manager, engaging in such difficult tasks as monitoring SLA compliance, managing the Solar JSOC service, and interacting with the Customer in terms of the level of service provided.
We are trying to help each employee determine the most appropriate development vector for him and find himself in the Solar JSOC structure.