March 14, 2016 at 17:30BDRA - Modern Architecture for Big Data Analytics
Big data is usually understood as a series of approaches, tools and methods for processing structured and unstructured data, which are distinguished by huge volumes and significant diversity. The purpose of such processing is to obtain human-perceived results.
The data stream can come from different sources, this data is heterogeneous and transmitted in various formats: text, documents, images, videos and much more. To extract useful information from such data, the hardware and software platform is of decisive importance.
Typically, big data solution developers seek to combine data processing capabilities of various nature. Often, the role of full-featured platforms for working with big data is played by traditional platforms adapted to new conditions. In other cases, suppliers offer enterprises specialized tailored solutions.
A common approach is to use the Hadoop platform. It works on the principle of moving the calculations closer to the data storage location: processing is usually performed on large clusters of servers created using standard hardware. The combination of the Hadoop platform with standard servers is the basis for a cost-effective and high-performance analytic platform for parallel applications.
A typical platform for big data is a cluster of identical nodes, usually represented by standard dual-processor servers and the storage system associated with each server.
In terms of efficiency, dual-processor servers are the best option for most Apache Hadoop workloads. Typically, these servers are more efficient for distributed computing environments than multiprocessor platforms. However, they do not always provide sufficient performance, and in other situations, on the contrary, it is redundant. Some workloads, such as simple data sorting, do not require the power of Intel Xeon processors. In such cases, it is more rational to perform such light workloads on microservers. Other tasks, on the contrary, require significant processing power and the use of “accelerators”. Moreover, usually the failure of one node in such architectures requires considerable time for the redistribution of data in the system.
Reconfiguring clusters to process big data for different tasks consumes significant resources and time. And this has already become a headache for IT departments in some Russian companies that are actively using big data technologies. HPE was able to solve the problem and offer a beautiful, effective and flexible solution.
The proposed developers HPE the reference architecture for large data (HPE Big Data Reference Architecture, BDRA ) with optimized performance intended for the creation of flexible and rapidly deployable high-performance solutions based on Hadoop. It consists of computing nodes combined with data storage resources. Unlike the architecture described above, BDRA takes a more flexible approach with load optimization and modern network architecture. As a result, the platform for consolidating, storing and processing big data turned out to be scalable, easy to deploy and use.
As noted above, general-purpose servers are typically used as nodes in a Hadoop cluster. But what if we use the compact HPE Moonshot servers for calculations and assign the storage function to devices with a large number of hard disks - the HPE Apollo 4500 or HPE Apollo 4200? What if the data will be stored not on local media, but on the disks of external devices connected to the computing nodes of a high-speed Ethernet network?
This is exactly what the HPE developers did. In addition to the obvious benefits - cost savings and easier system management - a significant increase in performance (read / write) was achieved, confirmed by various tests.
In collaboration with Cloudera and Hortonworks collaborative solutions were created based on HPE Big Data Architecture.
Most modern Hadoop systems use the Hadoop Distributed File System (HDFS) with high access speed and low latency. In 2012, Hadoop YARN (Yet Another Resource Negotiator) improved management and improved the utilization of HDFS resources. YARN uses so-called containers (RAM, CPU, and network bandwidth) to specify the resources available to the application. With this approach, there is a “horizontal separation” of computing resources of individual nodes between applications.
YARN is a key Hadoop tool. It can be described as a distributed operating system for big data applications. The appearance of the concept of labels in YARN allows you to group compute nodes and send a task to a specific group. In this way, the load can be optimized.
HPE has proposed "vertical resource sharing" - you can send jobs to nodes optimized for a specific load. For example, Hadoop MapReduce jobs are for universal nodes, Hive for energy-efficient nodes with low-voltage processors, and Storm for nodes with accelerators. To automatically distribute tasks, they are assigned labels.
Instead of SAN, Hadoop uses the concept of Software Defined Storage (SDS) with a distributed file system on standard storage nodes. At the same time, the Hadoop Distributed File System (HDFS) is usually used to work with files, and Ceph when working with objects. In addition, Hadoop, if the load requires it, can now support tearing - automatic distribution of data among storage tiers. These levels are SSD, HDD, RAM, or archived object storage.
The asymmetric architecture of Big Data Reference Architecture includes heterogeneous computing nodes and data storage nodes. Computing nodes can be represented by low-cost energy-efficient modules, modules with graphic accelerators (GPU), with programmable processors (FPGA) or with increased RAM. Storage nodes can use SSD or hard disks, archive systems. Optimizing nodes for load speeds up the execution of various applications for working with big data.
What are the benefits of such a Hadoop cluster architecture?
For high-performance computing workloads such as Apache Spark, high-performance computing (HPC) nodes can be included in the same rack.
HPE Solution - Hadoop Asymmetric Cluster with Optimized Nodes.
Thus, HPE abandoned the traditional paradigm and demonstrated that independent computing and storage components in the Hadoop cluster allow the creation of very fast asymmetric systems. This solution can be improved by optimizing the load and using tools such as YARN, tearing and assignment of tasks to specialized nodes.
HPE BDRA Schematic Diagram: The 42U rack integrates compute modules with embedded switches, storage modules, and control modules.
The HPE BDRA reference architecture allows you to consolidate heterogeneous data pools and combine them into a single pool that you can work with using Hadoop, Vertica, Spark, etc. Technologies. The flexibility to adapt to future loads is embedded in the architecture itself. In this converged asymmetric cluster, storage resources are layered. SAN is not used - it is replaced by direct access storage (DAS). Loads and storage resources are assigned to nodes optimized for the respective tasks. The standard Ethernet is used as an interconnect with native Hadoop protocols for the exchange between computing resources and storage resources, such as HDFS and HBase.
As an example, the HPE BDRA concept can significantly improve the price / performance ratio and computational density compared to the traditional Hadoop architecture. Thanks to modern Ethernet factories, the system does not form bottlenecks during data exchange between the server and the storage subsystem. Testing shows that the read performance of HPE BDRA is 30% higher than that of a normal Hadoop cluster.
HPE BDRA is based on the following HPE technologies:
HPE Apollo 4200 Gen9 Storage Node.
The HPE Apollo 4510 host can store up to 544 TB of data. It is recommended to use it for storage of backup copies or archive.
Storage Nodes - The HPE Apollo 4200 Gen9 Servers form a single HDFS storage. The HPE Apollo 4510 System, a high-performance, high-density storage system, can be used as an option. It plays the role of backup / archive storage.
HPE Moonshot System chassis with HPE ProLiant m710p Server Cartridge server cartridges.
Computing Nodes - The HPE Moonshot System is a high-density computing system for computing tasks and load optimization. HPE ProLiant m710p Server Cartridge or HPE ProLiant XL170r Gen9 servers can serve as compute nodes.
HPE BDRA components: compute nodes, storage nodes, and high-speed network.
Computing nodes and data storage nodes BDRA connects a high-speed network. The result is an asymmetric architecture where each level can be scaled individually. The ratio of processors and storage resources is not set rigidly. Since there is no mutual linking between these resources, you can take advantage of the many benefits of converged architecture. For example, scale them independently by simply adding the appropriate nodes to the system. HP testing shows that the load responds almost linearly. In addition, you can select a configuration with one or another ratio of resources for the type of load.
Independent scaling of computing resources and data storage resources: it is possible to choose the configuration according to the “hot” (computing) and “cold” loads. In the first case, the proportion of compute nodes increases, in the second, storage resources.
In HPE BDRA, YARN-compatible tools such as HBase and Apache Spark can directly use the HDFS storage system. Others, such as SAP HANA, require appropriate connectors to access data.
The HPE BDRA was tested with the Moonshot 1500 chassis and the latest Moonshot server cartridges. This solution allows to obtain high computational density. The Moonshot 1500 chassis with ProLiant m710p server cartridges connects to external switches with eight Direct Attach Copper (DAC) cables, each 40GbE.
HPE FlexFabric 5930 Switch.
The HPE FlexFabric 5930 Switch , configured through the HPE Intelligent Resilient Framework (IRF), is used as network switches in the HPE BDRA . The optional HPE 5900 Switch connects to 1GbE HPE Integrated Lights-Out (HPE iLO) management ports.
HPE Moonshot Server Chassis
The Moonshot System chassis contains two 45-port 10GbE switches to serve the internal network. Each switch is connected to an external infrastructure via four 40GbE uplinks. The HPE Apollo 4200 System, HPE SL4540, and Apollo 4510 System connect to high-performance ToR switches through a pair of 40GbE ports.
It is also important that in BDRA you can scale the desired level if necessary - computational or storage - without additional costs and redistribution of data. Consolidating data in BDRA avoids storing redundant copies and minimizes data movement.
HPE BDRA supports a variety of data management tools including Hadoop, Spark, HBase, Impala, Hive, Pig, R, and Storm. Thanks to the centralized storage and use of tools such as YARN tags, this solution provides access to data (direct or via connectors) and is a suitable platform for current and future enterprise applications.
One of the key benefits of HPE BDRA is its open standards, implementation of Open Source. This makes it possible for other vendors to work with the HPE solution. For example, they can use their designs to optimize workloads in the HPE BDRA. So Mellanox created its own hardware accelerator for its network card. This technology is integrated into the Moonshot cartridge.
Here are the benefits of the HPE BDRA solution again. The most obvious are density and price / performance ratio. Others include:
• Elasticity. The HPE BDRA architecture was designed for maximum flexibility. You can flexibly allocate computing nodes to tasks without redistributing data; you do not need to observe the ratio of storage resources and computing resources. You can expand the system, scale it. YARN-compatible loads get direct access to big data via HDFS, others can access the same data through their respective connectors.
• Consolidation of data. The HPE BDRA architecture is based on HDFS. And HDFS has enough performance and capacity to serve as a single source of data in any organization.
• Load optimization. To work with big data, a set of management tools is used. After choosing the right tool, you can start the task on the node that is best suited for a given load.
• Advanced storage capacity management. Computing nodes can be assigned dynamically, "on the fly", and managing a single repository reduces costs.
• Fast result. Typically, working with big data requires several management tools. In HPE BDRA, data is not fragmented. They are consolidated into a single “data lake” and tools for working with them can access the same data through YARN or a connector. As a result, more time is spent on analysis and less time on data delivery. The result is faster.
HPE BDRA is a reference architecture, but customers are offered documents for its implementation (Bills of Materials, BOM) in specific solutions. You can deploy such a system yourself, on your own, or use the services of HPE Technical Services or authorized partners. HPE BDRA is “customizable”: component configurations adapt to customer needs.
The data stream can come from different sources, this data is heterogeneous and transmitted in various formats: text, documents, images, videos and much more. To extract useful information from such data, the hardware and software platform is of decisive importance.
“Standard” Big Data Platform: System Architecture
Typically, big data solution developers seek to combine data processing capabilities of various nature. Often, the role of full-featured platforms for working with big data is played by traditional platforms adapted to new conditions. In other cases, suppliers offer enterprises specialized tailored solutions.
A common approach is to use the Hadoop platform. It works on the principle of moving the calculations closer to the data storage location: processing is usually performed on large clusters of servers created using standard hardware. The combination of the Hadoop platform with standard servers is the basis for a cost-effective and high-performance analytic platform for parallel applications.
A typical platform for big data is a cluster of identical nodes, usually represented by standard dual-processor servers and the storage system associated with each server.
In terms of efficiency, dual-processor servers are the best option for most Apache Hadoop workloads. Typically, these servers are more efficient for distributed computing environments than multiprocessor platforms. However, they do not always provide sufficient performance, and in other situations, on the contrary, it is redundant. Some workloads, such as simple data sorting, do not require the power of Intel Xeon processors. In such cases, it is more rational to perform such light workloads on microservers. Other tasks, on the contrary, require significant processing power and the use of “accelerators”. Moreover, usually the failure of one node in such architectures requires considerable time for the redistribution of data in the system.
Reconfiguring clusters to process big data for different tasks consumes significant resources and time. And this has already become a headache for IT departments in some Russian companies that are actively using big data technologies. HPE was able to solve the problem and offer a beautiful, effective and flexible solution.
Reference architecture of the big data platform
The proposed developers HPE the reference architecture for large data (HPE Big Data Reference Architecture, BDRA ) with optimized performance intended for the creation of flexible and rapidly deployable high-performance solutions based on Hadoop. It consists of computing nodes combined with data storage resources. Unlike the architecture described above, BDRA takes a more flexible approach with load optimization and modern network architecture. As a result, the platform for consolidating, storing and processing big data turned out to be scalable, easy to deploy and use.
As noted above, general-purpose servers are typically used as nodes in a Hadoop cluster. But what if we use the compact HPE Moonshot servers for calculations and assign the storage function to devices with a large number of hard disks - the HPE Apollo 4500 or HPE Apollo 4200? What if the data will be stored not on local media, but on the disks of external devices connected to the computing nodes of a high-speed Ethernet network?
This is exactly what the HPE developers did. In addition to the obvious benefits - cost savings and easier system management - a significant increase in performance (read / write) was achieved, confirmed by various tests.
In collaboration with Cloudera and Hortonworks collaborative solutions were created based on HPE Big Data Architecture.
Load optimization
Most modern Hadoop systems use the Hadoop Distributed File System (HDFS) with high access speed and low latency. In 2012, Hadoop YARN (Yet Another Resource Negotiator) improved management and improved the utilization of HDFS resources. YARN uses so-called containers (RAM, CPU, and network bandwidth) to specify the resources available to the application. With this approach, there is a “horizontal separation” of computing resources of individual nodes between applications.
YARN is a key Hadoop tool. It can be described as a distributed operating system for big data applications. The appearance of the concept of labels in YARN allows you to group compute nodes and send a task to a specific group. In this way, the load can be optimized.
HPE has proposed "vertical resource sharing" - you can send jobs to nodes optimized for a specific load. For example, Hadoop MapReduce jobs are for universal nodes, Hive for energy-efficient nodes with low-voltage processors, and Storm for nodes with accelerators. To automatically distribute tasks, they are assigned labels.
Instead of SAN, Hadoop uses the concept of Software Defined Storage (SDS) with a distributed file system on standard storage nodes. At the same time, the Hadoop Distributed File System (HDFS) is usually used to work with files, and Ceph when working with objects. In addition, Hadoop, if the load requires it, can now support tearing - automatic distribution of data among storage tiers. These levels are SSD, HDD, RAM, or archived object storage.
The asymmetric architecture of Big Data Reference Architecture includes heterogeneous computing nodes and data storage nodes. Computing nodes can be represented by low-cost energy-efficient modules, modules with graphic accelerators (GPU), with programmable processors (FPGA) or with increased RAM. Storage nodes can use SSD or hard disks, archive systems. Optimizing nodes for load speeds up the execution of various applications for working with big data.
What are the benefits of such a Hadoop cluster architecture?
- The cluster can be represented as an integrated solution, including computing resources, combined by an integrated network infrastructure;
- The built-in network factory allows increasing the traffic intensity in the east-west direction. As a result, the throughput of the entire cluster increases, switching becomes more intelligent;
- The use of nodes optimized for load types with specialized CPUs and graphics processors, “server-on-chip” (server-on-chip, SOC) increases productivity, computational density and energy efficiency;
- Hyperscale capabilities. For example, one HPE Moonshot chassis can act as a cluster of 45 nodes;
- The ability to build clusters with heterogeneous computing nodes, which is in demand for the tasks of deep analysis (Deep Learning) and neural networks.
For high-performance computing workloads such as Apache Spark, high-performance computing (HPC) nodes can be included in the same rack.
HPE Solution - Hadoop Asymmetric Cluster with Optimized Nodes.
Thus, HPE abandoned the traditional paradigm and demonstrated that independent computing and storage components in the Hadoop cluster allow the creation of very fast asymmetric systems. This solution can be improved by optimizing the load and using tools such as YARN, tearing and assignment of tasks to specialized nodes.
HPE BDRA Schematic Diagram: The 42U rack integrates compute modules with embedded switches, storage modules, and control modules.
The HPE BDRA reference architecture allows you to consolidate heterogeneous data pools and combine them into a single pool that you can work with using Hadoop, Vertica, Spark, etc. Technologies. The flexibility to adapt to future loads is embedded in the architecture itself. In this converged asymmetric cluster, storage resources are layered. SAN is not used - it is replaced by direct access storage (DAS). Loads and storage resources are assigned to nodes optimized for the respective tasks. The standard Ethernet is used as an interconnect with native Hadoop protocols for the exchange between computing resources and storage resources, such as HDFS and HBase.
As an example, the HPE BDRA concept can significantly improve the price / performance ratio and computational density compared to the traditional Hadoop architecture. Thanks to modern Ethernet factories, the system does not form bottlenecks during data exchange between the server and the storage subsystem. Testing shows that the read performance of HPE BDRA is 30% higher than that of a normal Hadoop cluster.
Solution Composition
HPE BDRA is based on the following HPE technologies:
HPE Apollo 4200 Gen9 Storage Node.
The HPE Apollo 4510 host can store up to 544 TB of data. It is recommended to use it for storage of backup copies or archive.
Storage Nodes - The HPE Apollo 4200 Gen9 Servers form a single HDFS storage. The HPE Apollo 4510 System, a high-performance, high-density storage system, can be used as an option. It plays the role of backup / archive storage.
HPE Moonshot System chassis with HPE ProLiant m710p Server Cartridge server cartridges.
Computing Nodes - The HPE Moonshot System is a high-density computing system for computing tasks and load optimization. HPE ProLiant m710p Server Cartridge or HPE ProLiant XL170r Gen9 servers can serve as compute nodes.
HPE BDRA components: compute nodes, storage nodes, and high-speed network.
Configuration flexibility and scaling
Computing nodes and data storage nodes BDRA connects a high-speed network. The result is an asymmetric architecture where each level can be scaled individually. The ratio of processors and storage resources is not set rigidly. Since there is no mutual linking between these resources, you can take advantage of the many benefits of converged architecture. For example, scale them independently by simply adding the appropriate nodes to the system. HP testing shows that the load responds almost linearly. In addition, you can select a configuration with one or another ratio of resources for the type of load.
Independent scaling of computing resources and data storage resources: it is possible to choose the configuration according to the “hot” (computing) and “cold” loads. In the first case, the proportion of compute nodes increases, in the second, storage resources.
In HPE BDRA, YARN-compatible tools such as HBase and Apache Spark can directly use the HDFS storage system. Others, such as SAP HANA, require appropriate connectors to access data.
The HPE BDRA was tested with the Moonshot 1500 chassis and the latest Moonshot server cartridges. This solution allows to obtain high computational density. The Moonshot 1500 chassis with ProLiant m710p server cartridges connects to external switches with eight Direct Attach Copper (DAC) cables, each 40GbE.
HPE FlexFabric 5930 Switch.
The HPE FlexFabric 5930 Switch , configured through the HPE Intelligent Resilient Framework (IRF), is used as network switches in the HPE BDRA . The optional HPE 5900 Switch connects to 1GbE HPE Integrated Lights-Out (HPE iLO) management ports.
HPE Moonshot Server Chassis
The Moonshot System chassis contains two 45-port 10GbE switches to serve the internal network. Each switch is connected to an external infrastructure via four 40GbE uplinks. The HPE Apollo 4200 System, HPE SL4540, and Apollo 4510 System connect to high-performance ToR switches through a pair of 40GbE ports.
It is also important that in BDRA you can scale the desired level if necessary - computational or storage - without additional costs and redistribution of data. Consolidating data in BDRA avoids storing redundant copies and minimizes data movement.
Open standards
HPE BDRA supports a variety of data management tools including Hadoop, Spark, HBase, Impala, Hive, Pig, R, and Storm. Thanks to the centralized storage and use of tools such as YARN tags, this solution provides access to data (direct or via connectors) and is a suitable platform for current and future enterprise applications.
One of the key benefits of HPE BDRA is its open standards, implementation of Open Source. This makes it possible for other vendors to work with the HPE solution. For example, they can use their designs to optimize workloads in the HPE BDRA. So Mellanox created its own hardware accelerator for its network card. This technology is integrated into the Moonshot cartridge.
What is the result?
Here are the benefits of the HPE BDRA solution again. The most obvious are density and price / performance ratio. Others include:
• Elasticity. The HPE BDRA architecture was designed for maximum flexibility. You can flexibly allocate computing nodes to tasks without redistributing data; you do not need to observe the ratio of storage resources and computing resources. You can expand the system, scale it. YARN-compatible loads get direct access to big data via HDFS, others can access the same data through their respective connectors.
• Consolidation of data. The HPE BDRA architecture is based on HDFS. And HDFS has enough performance and capacity to serve as a single source of data in any organization.
• Load optimization. To work with big data, a set of management tools is used. After choosing the right tool, you can start the task on the node that is best suited for a given load.
• Advanced storage capacity management. Computing nodes can be assigned dynamically, "on the fly", and managing a single repository reduces costs.
• Fast result. Typically, working with big data requires several management tools. In HPE BDRA, data is not fragmented. They are consolidated into a single “data lake” and tools for working with them can access the same data through YARN or a connector. As a result, more time is spent on analysis and less time on data delivery. The result is faster.
HPE BDRA is a reference architecture, but customers are offered documents for its implementation (Bills of Materials, BOM) in specific solutions. You can deploy such a system yourself, on your own, or use the services of HPE Technical Services or authorized partners. HPE BDRA is “customizable”: component configurations adapt to customer needs.
