June 17, 2013 at 13:14Fujitsu HPC Cluster Suite Computing Cluster Software Package
May 29 this year, Fujitsu introduced a special software package for computing clusters Fujitsu HPC Cluster Suite (HCS). This package is designed to manage clusters based on Fujitsu PRIMERGY servers of standard x86 architecture and is a set of fully tested high-performance software components. Initially, this product was developed for the supercomputer of the Institute of Physical and Chemical Research (RIKEN) K computer , which held the championship for two seasons and is currently in third place on the TOP500 list.with a performance exceeding 10 PFLOPS, which in itself already speaks about the potential and possibilities of the presented solution. Now with the release of the HCS package, the accumulated knowledge and experience is transferred to the classic x86 architecture and becomes available to almost any company.
The main components of any high-performance cluster are its computing nodes or nodes, the performance of the entire cluster depends on their number and configuration, because directly on them and settlement tasks are carried out. It is generally believed that a cluster consists of the same server components in terms of hardware, but, nevertheless, there are often situations when different servers are found in the cluster. In this case, nodes of different configurations within a single cluster are combined into groups of servers of the same type.
The interconnect used is important in the computing cluster, i.e. inter-node communications used. Interconnect should provide interaction between processes running on computing nodes in order to synchronize and exchange data. Usually, a high-speed Infiniband bus (40 or 56 Gb / s) is used to solve this problem. In addition, traditional Ethernet is used to control the computing nodes.
The operation of computing nodes is controlled by control nodes. At least one such node should be present even in a small cluster, while in large clusters there are usually several of them in order to ensure fault tolerance. Control nodes form a task queue, distribute them between computing nodes and their groups, while the tasks themselves are launched by users directly from their workplace.
Since computing nodes work with data, an important issue may be their location in the cluster structure. So, a separate network storage system can be used, to which all nodes can connect, or, especially in the case of large clusters, the so-called parallel file system can be used. A parallel file system provides distributed data storage when parts of it are stored on different nodes, but are accessible to each of them. It provides distribution between nodes of not only tasks, but also the corresponding data.
As for the HPC environment, it is focused primarily on non-interactive tasks performed on a set of computing nodes. Moreover, several tasks can be performed simultaneously on one cluster.
The main feature of the HPC software stack is the need to install it on dozens and thousands of servers of the same type (computing nodes), while all systems must have corresponding basic sets of different settings. For example, user and group identifiers, as well as corresponding passwords, should be the same on all computing nodes, which should also have the same access to the file system. In addition, you must have tools that allow you to quickly commission new or replace failed computing nodes.
Obviously, in such conditions, changing, for example, only one password per thousand nodes can lead to significant time losses if you do not use special software that allows you to either centrally manage passwords or automates their replacement on all servers in the cluster.
The next problem faced by cluster administrators is the selection of software and drivers. At the moment, you can find a huge number of different software versions used in cluster solutions, for example, various compilers, resource managers or MPI. Accordingly, the selection of these components, their testing for mutual compatibility and compatibility with hardware, their configuration and synchronization is a very time-consuming process and can take a lot of time.
It is here that the comprehensive HPC Cluster Suite solution offered by Fujitsu can help, since it is a set of pre-tested and tested software created in an alliance with leading developers that is guaranteed to work on Fujitsu equipment.
For a better understanding of the purpose and functionality of this product, consider a typical HPC software stack, which is presented in the diagram below. In this diagram, the dotted line outlines the components of the stack that are implemented in the HPC Cluster Suite. As you can see, in fact, HCS allows you to fill all the necessary space between the application itself and the computing nodes, being a full-featured package for managing clusters built on the basis of Fujitsu PRIMERGY servers.
Now, based on the diagram above, we briefly analyze the main components of the HPC Cluster Suite:
Software support for nVidia Tesla K20 / K20X accelerators and Intel Xeon Phi coprocessors. This suggests that the tasks performed on the cluster can focus not only on the computing power of the central processors, but also on the heterogeneous and parallel computing capabilities available on the named accelerators.
An important role in the presented product is played by Cluster Deployment Manager (CDM)- a package developed directly by Fujitsu, providing automated installation and configuration of computing nodes, as well as their commissioning; contains an administrator interface for performing the above tasks and monitoring the status of cluster nodes, managing user environments, and verifying cluster health.
As a Workload manager , you can choose from three open source software tools (TORQUE, SGE, SLURM - the last 2 will be available starting from the third quarter of 2013) or a commercial product (PBS Professional - only in HCS Advanced edition), which have already established themselves on the market.
Parallel middleware (middleware)includes third-party software, in particular, open source options: OpenMPI, MPICH, MVAPICH. In addition, the HPC Cluster Suite is fully compatible with Intel MPI (sold separately).
Libraries (Scientific Libraries) include Lapack, ScalaPack, BLAS, netcdf, netcdf-devel, hdf5, fftw, fftw-devel, atlas, atlas-devel, GMP, Global Arrays, MKL. Support for the following compilers, optimization and profiling tools is also offered : GNU c, c ++, gfort, Open64 (PathScale compiler), Intel Cluster studio, Allinea DDT, Intel vtune, PAPI, TAU. Moreover, all commercial products are purchased separately, in fact, their compatibility and integration capabilities are guaranteed.
An important component is the parallel file system developed by FujitsuFujitsu Exabyte File System (FEFS) based on Luster. It is no coincidence that the name Exabyte is present in its name, because the maximum size of its file system and one file is 8 EB, and parallel I / O implementation allows reaching speeds of the order of TB / s.
The graphical user interface in HCS is called HPC Gateway, it is a portal created on the basis of Liferay, and allows you to organize the joint work of users and their groups through the integration of blogs, forum, Wiki, knowledge base and other components.
HPC Cluster Suite is represented by three editions - Open, Basic and Advanced, featuring a set of software included in the package, a subscription to updates and technical support, and a limit on the number of computing nodes in the cluster. As for the delivery kits of various revisions, they are clearly presented in the table below.
As for the other features of the editions, for each of the editions except the Open edition, which is devoid of technical support subscription options and has limited access to updates, there are one, three, and five-year subscription options. Also, the Open edition supports up to 128 nodes in a cluster, while the rest of the editions do not have such a restriction and actually allow you to use an unlimited number of nodes. The final price of the HCS package depends on the package edition, usage segment (academic or commercial license) and the number of nodes in the cluster.
Typical HPC Cluster Architecture
The main components of any high-performance cluster are its computing nodes or nodes, the performance of the entire cluster depends on their number and configuration, because directly on them and settlement tasks are carried out. It is generally believed that a cluster consists of the same server components in terms of hardware, but, nevertheless, there are often situations when different servers are found in the cluster. In this case, nodes of different configurations within a single cluster are combined into groups of servers of the same type.
The interconnect used is important in the computing cluster, i.e. inter-node communications used. Interconnect should provide interaction between processes running on computing nodes in order to synchronize and exchange data. Usually, a high-speed Infiniband bus (40 or 56 Gb / s) is used to solve this problem. In addition, traditional Ethernet is used to control the computing nodes.
The operation of computing nodes is controlled by control nodes. At least one such node should be present even in a small cluster, while in large clusters there are usually several of them in order to ensure fault tolerance. Control nodes form a task queue, distribute them between computing nodes and their groups, while the tasks themselves are launched by users directly from their workplace.
Since computing nodes work with data, an important issue may be their location in the cluster structure. So, a separate network storage system can be used, to which all nodes can connect, or, especially in the case of large clusters, the so-called parallel file system can be used. A parallel file system provides distributed data storage when parts of it are stored on different nodes, but are accessible to each of them. It provides distribution between nodes of not only tasks, but also the corresponding data.
As for the HPC environment, it is focused primarily on non-interactive tasks performed on a set of computing nodes. Moreover, several tasks can be performed simultaneously on one cluster.
HPC Software Stack Features and HPC Cluster Suite Applications
The main feature of the HPC software stack is the need to install it on dozens and thousands of servers of the same type (computing nodes), while all systems must have corresponding basic sets of different settings. For example, user and group identifiers, as well as corresponding passwords, should be the same on all computing nodes, which should also have the same access to the file system. In addition, you must have tools that allow you to quickly commission new or replace failed computing nodes.
Obviously, in such conditions, changing, for example, only one password per thousand nodes can lead to significant time losses if you do not use special software that allows you to either centrally manage passwords or automates their replacement on all servers in the cluster.
The next problem faced by cluster administrators is the selection of software and drivers. At the moment, you can find a huge number of different software versions used in cluster solutions, for example, various compilers, resource managers or MPI. Accordingly, the selection of these components, their testing for mutual compatibility and compatibility with hardware, their configuration and synchronization is a very time-consuming process and can take a lot of time.
It is here that the comprehensive HPC Cluster Suite solution offered by Fujitsu can help, since it is a set of pre-tested and tested software created in an alliance with leading developers that is guaranteed to work on Fujitsu equipment.
For a better understanding of the purpose and functionality of this product, consider a typical HPC software stack, which is presented in the diagram below. In this diagram, the dotted line outlines the components of the stack that are implemented in the HPC Cluster Suite. As you can see, in fact, HCS allows you to fill all the necessary space between the application itself and the computing nodes, being a full-featured package for managing clusters built on the basis of Fujitsu PRIMERGY servers.
HPC Cluster Suite Components
Now, based on the diagram above, we briefly analyze the main components of the HPC Cluster Suite:
Software support for nVidia Tesla K20 / K20X accelerators and Intel Xeon Phi coprocessors. This suggests that the tasks performed on the cluster can focus not only on the computing power of the central processors, but also on the heterogeneous and parallel computing capabilities available on the named accelerators.
An important role in the presented product is played by Cluster Deployment Manager (CDM)- a package developed directly by Fujitsu, providing automated installation and configuration of computing nodes, as well as their commissioning; contains an administrator interface for performing the above tasks and monitoring the status of cluster nodes, managing user environments, and verifying cluster health.
As a Workload manager , you can choose from three open source software tools (TORQUE, SGE, SLURM - the last 2 will be available starting from the third quarter of 2013) or a commercial product (PBS Professional - only in HCS Advanced edition), which have already established themselves on the market.
Parallel middleware (middleware)includes third-party software, in particular, open source options: OpenMPI, MPICH, MVAPICH. In addition, the HPC Cluster Suite is fully compatible with Intel MPI (sold separately).
Libraries (Scientific Libraries) include Lapack, ScalaPack, BLAS, netcdf, netcdf-devel, hdf5, fftw, fftw-devel, atlas, atlas-devel, GMP, Global Arrays, MKL. Support for the following compilers, optimization and profiling tools is also offered : GNU c, c ++, gfort, Open64 (PathScale compiler), Intel Cluster studio, Allinea DDT, Intel vtune, PAPI, TAU. Moreover, all commercial products are purchased separately, in fact, their compatibility and integration capabilities are guaranteed.
An important component is the parallel file system developed by FujitsuFujitsu Exabyte File System (FEFS) based on Luster. It is no coincidence that the name Exabyte is present in its name, because the maximum size of its file system and one file is 8 EB, and parallel I / O implementation allows reaching speeds of the order of TB / s.
The graphical user interface in HCS is called HPC Gateway, it is a portal created on the basis of Liferay, and allows you to organize the joint work of users and their groups through the integration of blogs, forum, Wiki, knowledge base and other components.
HPC Cluster Suite Editions
HPC Cluster Suite is represented by three editions - Open, Basic and Advanced, featuring a set of software included in the package, a subscription to updates and technical support, and a limit on the number of computing nodes in the cluster. As for the delivery kits of various revisions, they are clearly presented in the table below.
As for the other features of the editions, for each of the editions except the Open edition, which is devoid of technical support subscription options and has limited access to updates, there are one, three, and five-year subscription options. Also, the Open edition supports up to 128 nodes in a cluster, while the rest of the editions do not have such a restriction and actually allow you to use an unlimited number of nodes. The final price of the HCS package depends on the package edition, usage segment (academic or commercial license) and the number of nodes in the cluster.
