Big Data analytics - realities and prospects in Russia and the world
Only a person who has no external connections with the outside world has not heard about big data today. On Habré, the Big Data analytics topic and related topics are popular. But non-specialists who would like to devote themselves to the study of Big Data, it is not always clear what prospects this area has, where Big Data analytics can be applied and what a good analyst can count on. Let's try to figure it out.
Every year the volume of information generated by a person increases. By 2020, the volume of stored data will increase to 40-44 zettabytes (1 ST ~ 1 billion GB). By 2025, up to about 400 zettabytes. Accordingly, managing structured and unstructured data using modern technology is an area that is becoming increasingly important. Interested in big data, both individual companies and entire states.
By the way, it was in the process of discussing the information boom and methods of processing human-generated data that the term Big Data arose. It is believed that the editor of Nature magazine, Clifford Lynch, first proposed it in 2008.
Since then, the Big Data market has grown by several tens of percent annually. And this trend, according to experts, will continue. Thus, according to Frost & Sullivan, in 2021 the total global market for big data analytics will increase to $ 67.2 billion. The annual growth will be about 35.9%.
Why Big Data Analytics?
It allows you to identify extremely valuable information from structured or unstructured data sets. Thanks to this, a business, for example, can determine trends, forecast production indicators and optimize its own expenses. It is clear that in order to reduce costs, companies are ready to introduce the latest solutions.
Technologies and analysis methods that are used to analyze Big Data:
- Data mining;
- data mixing and integration;
- machine learning;
- artificial neural networks;
- pattern recognition;
- predictive analytics;
- simulation modeling;
- spatial analysis;
- statistical analysis;
- visualization of analytical data.
Big Data analytics in the world
Big data analytics is now used by more than 50% of companies worldwide. Despite the fact that in 2015 this figure was only 17%. Big Data is most actively used by companies that work in the field of telecommunications and financial services. Then come companies that specialize in healthcare technology. Minimal use of Big Data analytics in educational companies: in most cases, representatives of this field announced their intention to use technology in the near future.
In the USA, Big Data analytics is used most actively: more than 55% of companies from various fields work with this technology. In Europe and Asia, the demand for big data analytics is not much lower - about 53%.
And what about Russia?
According to IDC analysts, Russia is the largest regional Big Data analytics market . The growth of the market for such solutions in Central and Eastern Europe is quite active, each year this indicator increases by 11%. By 2022, it will reach quantitatively $ 5.4 billion.
In many ways, such a rapid development of the market is determined by the growth of this sphere in Russia. In 2018, revenue from the sale of relevant solutions in the Russian Federation amounted to 40% of the total investment in Big Data processing technologies in the entire region.
In the Russian Federation, companies from the banking and public sectors, the telecommunications industry and industry spend the most on Big Data processing.
What does Big Data Analyst do and how much does it receive in Russia?
The Big Data Analyst is responsible for exploring vast amounts of information, both partially structured and unstructured. For banking organizations, these are transactions, for operators, calls and traffic, and in retail, customer visits and purchases. As mentioned above, Big Data analysis reveals the connections between various factors of a “raw information history”, for example, a production process or a chemical reaction. Based on the analysis data, new approaches and solutions are developed in a variety of areas - from production to medicine.
Skills needed by Big Data analytics:
- The ability to quickly understand the features in the area for which the analysis is carried out, to immerse yourself in the aspects of the desired sphere. This may be retail, the oil and gas industry, medicine, etc.
- Knowledge of methods of statistical data analysis, construction of mathematical models (neural networks, Bayesian networks, clustering, regression, factorial, variance and correlation analyzes, etc.).
- To be able to extract data from different sources, convert them for analysis, upload to the analytical database.
- SQL mastery.
- Knowledge of English at a level sufficient to seamlessly read technical documentation.
- Knowledge of Python (at least the basics), Bash (it’s very difficult to do without it in the process of work), plus it is desirable to know the basics of Java and Scala (needed to actively use Spark, one of the most popular frameworks for working with big data).
- Ability to work with Hadoop.
Well, how much does Big Data analyst get?
Big Data specialists are in short supply, demand exceeds supply. This is because business comes to an understanding: for development, new technologies are needed, and for the development of technologies, specialists are required.
So, Data Scientist and Data Analyst in the USA entered the top 3 best professions of 2017 according to the recruitment agency Glassdoor. The average salary of these professionals in America starts at $ 100 thousand per year.
In Russia, machine learning specialists receive from 130 to 300 thousand rubles per month, big data analysts - from 73 to 200 thousand rubles per month. It all depends on experience and qualifications. Of course, there are vacancies with a lower salary, and there are more vacancies. Maximum demand for big data analysts in Moscow and St. Petersburg. Not surprisingly, Moscow accounts for about 50% of active vacancies (according to hh.ru). Much less demand is in Minsk and Kiev. It is worth noting that some vacancies offer a flexible schedule and remote work. But in general, companies require specialists who work in the office.
Over time, we can expect an increase in demand for Big Data analysts and representatives of related specialties. As mentioned above, no one has canceled the staff shortage in the field of technology. But, of course, in order to become a Big Data analyst you need to learn and work, improving both the skills mentioned above and additional ones. One of the opportunities to start the path of Big Data analytics is to sign up for a course from Geekbrains and try your hand at working with big data.