Graphic processors in solving modern IT problems



    Graphics processors (GPUs) are a prime example of how technology designed for graphics processing has spread to an unrelated area of ​​high performance computing. Modern GPUs are the heart of many complex projects in the field of machine learning and data analysis. In our review article, we will describe how Selectel customers use equipment with a GPU, and think about the future of data science and computing devices, along with teachers from the Yandex Data Analysis School.

    Graphics processors over the past ten years have changed dramatically. In addition to the colossal increase in productivity, there was a separation of devices by type of use. Thus, video cards for home gaming systems and virtual reality installations stand out in a separate direction. Powerful, highly specialized devices are emerging: for server systems, one of the leading accelerators is the NVIDIA Tesla P100 , designed specifically for industrial use in data centers. In addition to the GPU, research is being actively conducted in the field of creating a new type of processors simulating the work of the brain. An example is the Kirin 970 single-chip platform with its own neuromorphic processor for tasks related to neural networks and pattern recognition.

    This situation makes you think about the following questions:

    • Why has the field of data analysis and machine learning become so popular?
    • How did graphics processors become dominant in the data hardware market?
    • What research in data analysis will be the most promising in the near future?

    Let's try to deal with these issues in order, starting with the first simple video processors and ending with modern high-performance devices.

    GPU Era


    First, remember what a GPU is. The Graphics Processing Unit is a graphics processor widely used in desktop and server systems. A distinctive feature of this device is its focus on massively parallel computing. Unlike GPUs, the architecture of another CPU (Central Processor Unit) computing module is designed for serial data processing. If the number of cores in a conventional CPU is measured in dozens, then in the GPU they are counted in thousands, which imposes restrictions on the types of commands executed, but provides high computational performance in tasks involving parallelism.

    The first steps


    The development of video processors in the early stages was closely associated with the growing need for a separate computing device for processing two and three-dimensional graphics. Before the appearance of separate video controller circuits in the 70s, image output was carried out through the use of discrete logic, which affected the increased power consumption and larger sizes of printed circuit boards. Specialized microcircuits made it possible to isolate the development of devices designed for working with graphics in a separate direction.

    The next revolutionary event was the emergence of a new class of more complex and multifunctional devices - video processors. In 1996, 3dfx Interactive released the Voodoo Graphics chipset, which quickly occupied 85% of the market for specialized video devices and became the leader in 3D graphics at the time. After a series of unsuccessful decisions of the company's management, among which was the purchase of a manufacturer of video cards STB, 3dfx gave way to NVIDIA and ATI (later AMD), and in 2002 declared bankruptcy.

    General GPU Computing


    In 2006, NVIDIA announced the launch of the GeForce 8 series product line, which marked the beginning of a new class of devices designed for general computing on graphics processors (GPGPU). During development, NVIDIA realized that a larger number of cores operating at a lower frequency are more efficient for parallel loads than a small number of more productive cores. Video processors of the new generation provided support for parallel computing, not only for processing video streams, but also for problems related to machine learning, linear algebra, statistics, and other scientific or commercial problems.



    Recognized leader


    Differences in the initial formulation of tasks for the CPU and GPU led to significant discrepancies in the architecture of devices - a high frequency versus multi-core. For GPUs, this laid the computational potential, which is currently being fully realized. Video processors with an impressive amount of weaker computational cores do an excellent job with parallel computing. The central processor, historically designed to work with sequential tasks, remains the best in its field.

    For example, let us compare the values ​​in the performance of the central and graphic processor on the performance of a common task in neural networks — multiplication of high-order matrices. Choose the following devices for testing:

    • CPU. Intel Xeon E5-2680 v4 - 28 threads with HyperThreading, 2.4 GHZ;
    • GPU. NVIDIA GTX 1080 - 2560 CUDA Cores, 1607 Mhz, 8GB GDDR5X.

    We use the example of calculating the multiplication of matrices on the CPU and GPU in Jupyter Notebook:



    In the code above, we measure the time it took to calculate matrices of the same order on the central or graphics processor (“Runtime”). Data can be represented as a graph, on which the horizontal axis displays the order of the matrixes to be multiplied, and the vertical axis - Runtime in seconds:



    The line of the graph, highlighted in orange, shows the time it takes to create data in conventional RAM, transfer it to the memory of the GPU, and subsequent calculations. The green line shows the time it takes to calculate the data that has already been generated in the memory of a video card (without transferring it from RAM). Blue displays the counting time on the central processor. Matrices of the order of less than 1000 elements are multiplied by the GPU and CPU in almost the same time. The difference in performance is well manifested with matrices larger than 2000 by 2000, when the computing time on the CPU jumps to 1 second, and the GPU remains close to zero.

    More complex and practical tasks are more effectively solved on the device with graphic processors than without them. Since the problems that our clients solve on GPU equipment are very diverse, we decided to find out which most popular usage scenarios exist.

    Who in Selectel lives well with the GPU?


    The first option, which immediately comes to mind and turns out to be a correct guess, is mining, but it is interesting to note that some people use it as an auxiliary way to load equipment to the “maximum”. In case of renting a dedicated server with video cards, the time free from workloads is used to mine cryptocurrencies that do not require specialized facilities (farms) to receive.

    The problems that have already become somewhat classical, associated with graphic processing and rendering, always find their place on Selectel servers with graphic accelerators. The use of high-performance equipment for such tasks allows us to obtain a more efficient solution than the organization of dedicated workplaces with video cards.

    During the conversation with our clients, we also met with representatives of the Yandex Data Analysis School, which uses the power of Selectel to organize test training environments. We decided to learn more about what students and teachers do, what machine learning directions are now popular, and what future awaits the industry after young professionals join the ranks of employees of leading organizations or launch their start-ups.

    Data science


    Perhaps, among our readers there will not be those who would not have heard the phrase “neural networks” or “machine learning”. Rejecting marketing variations on the theme of these words, it turns out a dry residue in the form of an emerging and promising data science.

    The modern approach to working with data includes several main areas:

    • Big Data. The main problem in this area is the colossal amount of information that cannot be processed on a single server. From the point of view of infrastructure support, it is required to solve the problems of creating cluster systems, scalability, fault tolerance, and distributed data storage;
    • Resource-intensive tasks (machine learning, deep learning and others). In this case, the issue of using high-performance computing that requires a large amount of RAM and processor resources is raised. In such tasks, systems with graphic accelerators are actively used.


    The boundary between these directions is gradually erased: the main tools for working with big data (Hadoop, Spark) introduce support for GPU computing, and machine learning tasks cover new areas and require more data. Teachers and students of the School of Data Analysis will help us to understand more.



    It is difficult to overestimate the importance of competent data manipulation and the appropriate implementation of advanced analytical tools. It is not even about big data, their “lakes” or “rivers”, but about intellectual interaction with information. What is happening now represents a unique situation: we can collect a wide variety of information and use advanced tools and services for in-depth analysis. The business introduces such technologies not only to obtain advanced analytics, but also to create a unique product in any industry. It is the latter point that in many ways forms and stimulates the growth of the data analysis industry.

    New direction


    Everywhere we are surrounded by information: from the logs of Internet companies and banking operations to the testimony in experiments at the Large Hadron Collider. The ability to work with this data can bring millions of profits and provide answers to fundamental questions about the structure of the Universe. Therefore, data analysis has become a separate area of ​​research among the business and scientific community.

    The school of data analysis prepares the best specialized specialists and scientists who in the future will become the main source of scientific and industrial developments in this field. The development of the industry affects us as an infrastructure provider - more and more customers are requesting server configurations for data analysis tasks.

    The specifics of the tasks facing our customers depend on what equipment we have to offer to customers and in what direction our product line should be developed. Together with Stanislav Fedotov and Oleg Ivchenko, we interviewed students and teachers of the School for Data Analysis and found out what technologies they use to solve practical problems.



    Data Analysis Technologies


    During training, students from the basics (basic higher mathematics, algorithms and programming) reach the most advanced areas of machine learning. We collected information on those that use servers with GPUs:

    • Deep learning;
    • Reinforcement training;
    • Computer vision;
    • Automatic text processing.

    Students use specialized tools in their learning tasks and research. Some libraries are designed to bring data to the desired form, others are designed to work with a specific type of information, such as text or images. Depth learning is one of the most difficult areas in data analysis that makes extensive use of neural networks. We decided to find out what kind of framework teachers and students use to work with neural networks.



    The presented tools have different support from the creators, but nevertheless, continue to be actively used for educational and working purposes. Many of them require productive equipment to handle tasks in adequate time.

    Further development and projects


    As with any science, the direction of data analysis will change. The experience that students receive today will undoubtedly become the basis for future developments. Therefore, it is worth noting separately the high practical orientation of the program - some students, during their studies or after, begin internships at Yandex and apply their knowledge already on real services and services (search, computer vision, speech recognition and others).

    We talked about the future of data analysis with the teachers of the School of Data Analysis, who shared with us their vision of the development of data science.

    According to Vlad Shahuro, a teacher of the course “Image and Video Analysis”, the most interesting tasks in computer vision are to ensure security in crowded places, control an unmanned vehicle and create an application using augmented reality. To solve these problems, it is necessary to be able to qualitatively analyze video data and develop primarily algorithms for detecting and tracking objects, recognizing a person by face, and three-dimensional reconstruction of the observed scene. Lecturer Viktor Lempitsky , who leads the course “Depth education”, separately highlights auto-encoders in his area, as well as generative and adversary networks.

    One of the mentors of the School of Data Analysis shares his opinion on the distribution and the beginning of the mass use of machine learning:

    “Machine learning from the lot of the few obsessed researchers is turning into another tool of an ordinary developer. Previously (for example, in 2012), people wrote low-level code for learning convolutional networks on a pair of video cards. Now, anyone can in a matter of hours:

    • download the weight of the already trained neural network (for example, in keras);
    • use it to make a decision for your task ( fine-tuning, zero-shot learning );
    • embed it in your website or mobile application (tensorflow / caffe 2).

    Many large companies and startups have already won on such a strategy (for example, Prisma), but even more tasks are yet to be discovered and solved. And, perhaps, this whole story with machine / in-depth training will someday become as commonplace as python or excel now. ”

    According to Oleg Ivchenko , administrator of the server infrastructure of the ShAD, for standard deep learning tasks on standard data sets (for example, CIFAR , MNIST) The following resources are required:

    • 6 CPU cores;
    • 16 Gb of RAM;
    • 1 GPU card with 6-8 Gb of video memory. This corresponds to such video cards as PNY NVIDIA GeForce GTX 1060 or MSI GeForce GTX 1070.

    Accurately predict the technology of the future today, no one can, but when there is a certain motion vector, you can understand what should be learned now. And there are a lot of opportunities for this in the modern world.

    Opportunities for beginners


    The study of data analysis is limited to high requirements for students: extensive knowledge in the field of mathematics and algorithms, the ability to program. Truly serious machine learning tasks require the availability of specialized equipment. And for those who want to learn more about the theoretical component of data science, the School of Data Analysis, together with the Higher School of Economics , launched an online course on Introduction to Machine Learning .

    Instead of conclusion


    The growth of the graphics processor market is supported by the growing interest in the capabilities of such devices. The GPU is used in home gaming systems, rendering and video processing tasks, as well as where general high performance computing is required. The practical application of data mining tasks will penetrate deeper into our daily lives. And the execution of such programs is most effectively carried out with the help of the GPU.

    We thank our customers, as well as teachers and students of the School of Data Analysis for the joint preparation of the material, and invite our readers to get to know them better .

    And experienced and experienced in the field of machine learning, data analysis and not only we offer to look at the proposals from Selectel for renting server hardware with graphics accelerators: from simple GTX 1080 to Tesla P100 and K80 for the most demanding tasks.
    Tags:

    Also popular now: