December 29, 2017 at 19:23Intel Xeon Scalable - a new word in memory
Hello GT! In 2017, a lot of interesting things happened on the server processor market. Perhaps the most significant event was the presentation of the new Intel Xeon Scalable server processors, as well as the related Purley platform. And today we will talk about how these new products work with memory, how they differ from their predecessors, and which modules are better to use with them.
According to IDC, Intel owns more than 90% of the server processor market, but this year AMD introduced new powerful EPYC processors. The leaders did not remain in debt and in the summer showed the whole world the Purley platform, which differs from previous solutions, as well as from competitors' products, with a new memory scheme.
Perhaps the most important feature of Purley is its architecture. The manufacturer introduced at the same time Intel Xeon Scalable processors with integrated controllers and special optimizers, as well as Intel Optane SSD components and Intel Xeon Phi chipsets. Provided that high-performance DRAM memory is installed, all this will work at maximum speed, opening up new opportunities for "cloud computing, virtualization, next-generation telecommunication networks (5G), machine learning and artificial intelligence."
The Intel Xeon Scalable processors themselves have much higher performance than the previous generation. According to Intel, growth is about 65%. This applies to the top Intel Xeon Scalable Platinum processors, which can contain up to 28 cores on a chip (there are versions with a smaller number) operating at a frequency of up to 2.4 GHz. Due to new technologies for transferring data between processors and computing cores, new products allow you to perform poorly parallelized tasks when you cannot predict in advance what information will be needed at the next moment. Let's look at how the new platform works with data.
In addition to having a 6-channel DDR4 memory controller, Intel Xeon Scalable processors can also work directly with Intel Optane SSDs. Thanks to special optimizations, connecting via the PCIe 3.0 interface, they actually create a new level of operational data storage, providing processors with access to an extensive memory field. The processors support up to 48 PCIe interfaces on the board, which allows you to install additional Intel Optane drives in a fairly large number. The PCIe data transfer rate is 8 gigatransactions per second (equivalent to 32 Gb / s), and Optane can run at about 2 Gb / s per drive.
According to Intel, when installing 6 Intel Optane drives and using Intel SPDK, you can reduce response times by up to 40 times, as well as increase IOPS (number of I / O operations) by 5.2 times and reduce latency by 3.3 times compared to work on traditional drives. This is due to accelerated access to information and Tier-ing data placement on various drives.
How much memory does this get? Let's count: each Intel Xeon Scalable processor supports 6 memory channels with 2 modules each. Thus, you can set 12 * 128 GB = 1.5 TB of RAM. Supplementing them with 6 SSDs with a capacity of 512 GB, you can get 1.5 + 3 = 4.5 TB of high-speed memory for EVERY processor. Moreover, the use of Intel Memory Drive Technology (MDT) allows you to create software-defined storage for each specific server. A special driver is loaded to the OS and combines all the RAM and drives into a single two-tier storage. As a result, the operating system receives a ready-made memory storage with automated distribution of data between “fast” and “slow” segments.
This is really an incredible result, considering that you can install quite a lot of capacious, but slower disks for static storage of data sets in each server. For example, 10 SATA drives with a capacity of 2 TB each can add 20 TB of “slow” storage, in order to achieve a higher speed, the choice can be made in favor of SSD-drives. Intel Xeon Scalable processors have a built-in VMD (Virtual Management Device) module, which independently creates RAID arrays from drives connected via PCIe and SATA, supporting hot-swapping of failed components, and also directly interacts with a network controller to speed up work with data in the entire computing cluster.
Now let's get back to the processor itself. The SkyLake architecture is changing the structure of the cache itself. The L1 cache is located inside the kernel, next to each core is an “add-on” to the 768 KB L2 cache, which allows it to reach 1 MB. And the L3 cache, from which each core can receive data directly, is located in a separate layer of the chip and is 39 MB - that is, 1,375 MB per core. This cache is non-inclusive - the data comes from memory directly to L2, and data lines that are already unnecessary or common for several cores are pushed into the L3 cache.
As you can see in the above diagram, the internuclear interaction takes place not on the ring bus, as it was in the previous generation of processors, but according to the Mesh architecture. It accelerates the exchange of information and qualitatively improves the operation of new chips at high loads, typical for the tasks of virtualization and complex analytical systems, especially when core requests for memory are almost impossible to predict.
By the way, the same architecture is used to exchange data between processors in a multiprocessor server. Thanks to the OmniPath bus, the “communication” of the chips with each other is much faster, and the Remote Direct Memory Access architecture allows you to access directly to “foreign” memory cells, bypassing the OS level. Thus, computing cores can work with data located in the memory field of another processor or even another node of the computing cluster.
The cache hierarchy, as well as technology for accessing data stored in RAM of other processors, including over the network, make the large and accessible field of RAM one of the main advantages of the new Intel platform. And if traditional drives connected via the SATA interface can be replaced in the hot swap mode, then the main memory must be initially selected as reliable and stable as possible. In cloud data centers and in heavy analytical systems, RAM plays a key role, and Kingston already has a proposal created specifically for new processors.
The Purley platform allows the installation of RDIMM register memory modules or LRDIMM or 3DS LRDIMM modules for energy efficiency. With the advent of the new Intel and AMD platforms, Kingston has certified its memory modules for innovative server platforms.
By the way, note that the Kingston server memory line is now labeled KSM (Kingston Server Memory), not KVR, KCP, KTH, KTD, KTL, KCS - or something else. So far, this concerns modules with a speed of 2666 MHz, but all new Kingston branded server memory modules will be marked KSM, including those operating at higher frequencies, which are scheduled for release in 2018. However, in the case of Xeon Scalable, this does not matter yet, since the top-end Intel Xeon Scalable has a built-in memory controller at a frequency of 2666 MHz and simply does not need faster memory in Purley. And for real tasks, the most expensive chips are not always needed at all. In most cases, you can get by with the Gold 51xx, Silver 41xx and Bronze 31xx processors on the same architecture,
As you can see, with a reasonable approach, you can also save on memory, since Kingston, of course, offers products with any frequencies from the above scheme. It is enough to determine the range of tasks that the server will perform and install in it a memory corresponding to the processor capabilities. For example, in the Bronze 31xx series, it makes no sense to buy even DDR4-2400 MHz, since the processor will not use its capabilities.
Ordering a new generation of memory — KSM modules — has become noticeably simpler. There are no more marking differences. If you bought Kingston memory for servers, you are well aware that before we had two types of server memory - Server Premier and Value RAM. All KSM memory has the properties of Server Premier, despite the fact that the price of modules has been reduced in relation to the premium series. In addition, if earlier it was necessary to check whether the memory marking has the suffix “i” (speaking of Intel certification), now you can forget about it - the entire KSM series is initially certified. Therefore, new products will be easier to choose for both builders and users of server systems.
All KSM modules use a fixed BOM (Bill of Materials). This means that Kingston experts carefully select the manufacturers of the chips themselves and allow only the highest quality products in the series. Engineers test each memory cell at the production control stage, and also test the printed circuit boards themselves. We control everything, up to the revision of the chips and the manufacturer of the register chip. Thus, the KSM series modules are Kingston's most carefully controlled memory series for professional applications.
All memory information is now easily read in its part number. For example, if you read the following number on a module:
This will mean that the manufacturer of the H chip is Hynix, the revision of the chip is A, and the manufacturer of the register chip is IDT. By the way, different companies can be manufacturers of register chips. In addition to IDT (I), components from Rambus, previously known as Inphi ® and Montage (M), are also used.
In general, more transparent marking not only reflects Kingston's more holistic approach to Server memory release, but also helps to upgrade the modules by controlling all parameters, up to the chip manufacturer. This will avoid possible conflicts or reduce the productivity of equipment due to incomplete compatibility, as well as purchase the same type of modules for several types of servers, simplifying the logistics and maintenance of systems.
To summarize a short summary. To get the most out of the new platform, you need to carefully select all the components and use all the Intel optimization tools that help take advantage of the new processors and the Purley platform as a whole. The advantages of Kingston memory for servers are already used by leading hosting companies , and if you install the most productive and reliable modules, switching to Intel Xeon Scalable will give the maximum possible effect for solving difficult tasks - from virtualization to analytics and modeling.
Subscribe and stay with us - it will be interesting!
For more information on Kingston and HyperX products, visit the company's official website..
Our home country is wide, so we prepared stocks at once in several network partner stores - everyone should have the opportunity to get HyperX accessories as close to home as possible. Until January 12, you can purchase peripherals with maximum discounts.
• 1000 rubles discount for HyperX Cloud Stinger headset
• 2000 rubles discount for HyperX Alloy FPS keyboard with Cherry MX switches (optional Brown / Blue / Red)
• 1000 rubles discount for HyperX Cloud Silver headset
• 700 rubles discount for mouse HyperX Pulsefire FPS
According to IDC, Intel owns more than 90% of the server processor market, but this year AMD introduced new powerful EPYC processors. The leaders did not remain in debt and in the summer showed the whole world the Purley platform, which differs from previous solutions, as well as from competitors' products, with a new memory scheme.
Perhaps the most important feature of Purley is its architecture. The manufacturer introduced at the same time Intel Xeon Scalable processors with integrated controllers and special optimizers, as well as Intel Optane SSD components and Intel Xeon Phi chipsets. Provided that high-performance DRAM memory is installed, all this will work at maximum speed, opening up new opportunities for "cloud computing, virtualization, next-generation telecommunication networks (5G), machine learning and artificial intelligence."
The Intel Xeon Scalable processors themselves have much higher performance than the previous generation. According to Intel, growth is about 65%. This applies to the top Intel Xeon Scalable Platinum processors, which can contain up to 28 cores on a chip (there are versions with a smaller number) operating at a frequency of up to 2.4 GHz. Due to new technologies for transferring data between processors and computing cores, new products allow you to perform poorly parallelized tasks when you cannot predict in advance what information will be needed at the next moment. Let's look at how the new platform works with data.
New memory hierarchy
In addition to having a 6-channel DDR4 memory controller, Intel Xeon Scalable processors can also work directly with Intel Optane SSDs. Thanks to special optimizations, connecting via the PCIe 3.0 interface, they actually create a new level of operational data storage, providing processors with access to an extensive memory field. The processors support up to 48 PCIe interfaces on the board, which allows you to install additional Intel Optane drives in a fairly large number. The PCIe data transfer rate is 8 gigatransactions per second (equivalent to 32 Gb / s), and Optane can run at about 2 Gb / s per drive.
According to Intel, when installing 6 Intel Optane drives and using Intel SPDK, you can reduce response times by up to 40 times, as well as increase IOPS (number of I / O operations) by 5.2 times and reduce latency by 3.3 times compared to work on traditional drives. This is due to accelerated access to information and Tier-ing data placement on various drives.
How much memory does this get? Let's count: each Intel Xeon Scalable processor supports 6 memory channels with 2 modules each. Thus, you can set 12 * 128 GB = 1.5 TB of RAM. Supplementing them with 6 SSDs with a capacity of 512 GB, you can get 1.5 + 3 = 4.5 TB of high-speed memory for EVERY processor. Moreover, the use of Intel Memory Drive Technology (MDT) allows you to create software-defined storage for each specific server. A special driver is loaded to the OS and combines all the RAM and drives into a single two-tier storage. As a result, the operating system receives a ready-made memory storage with automated distribution of data between “fast” and “slow” segments.
This is really an incredible result, considering that you can install quite a lot of capacious, but slower disks for static storage of data sets in each server. For example, 10 SATA drives with a capacity of 2 TB each can add 20 TB of “slow” storage, in order to achieve a higher speed, the choice can be made in favor of SSD-drives. Intel Xeon Scalable processors have a built-in VMD (Virtual Management Device) module, which independently creates RAID arrays from drives connected via PCIe and SATA, supporting hot-swapping of failed components, and also directly interacts with a network controller to speed up work with data in the entire computing cluster.
Cache and special memory access
Now let's get back to the processor itself. The SkyLake architecture is changing the structure of the cache itself. The L1 cache is located inside the kernel, next to each core is an “add-on” to the 768 KB L2 cache, which allows it to reach 1 MB. And the L3 cache, from which each core can receive data directly, is located in a separate layer of the chip and is 39 MB - that is, 1,375 MB per core. This cache is non-inclusive - the data comes from memory directly to L2, and data lines that are already unnecessary or common for several cores are pushed into the L3 cache.
As you can see in the above diagram, the internuclear interaction takes place not on the ring bus, as it was in the previous generation of processors, but according to the Mesh architecture. It accelerates the exchange of information and qualitatively improves the operation of new chips at high loads, typical for the tasks of virtualization and complex analytical systems, especially when core requests for memory are almost impossible to predict.
By the way, the same architecture is used to exchange data between processors in a multiprocessor server. Thanks to the OmniPath bus, the “communication” of the chips with each other is much faster, and the Remote Direct Memory Access architecture allows you to access directly to “foreign” memory cells, bypassing the OS level. Thus, computing cores can work with data located in the memory field of another processor or even another node of the computing cluster.
And again, it's all a matter of memory!
The cache hierarchy, as well as technology for accessing data stored in RAM of other processors, including over the network, make the large and accessible field of RAM one of the main advantages of the new Intel platform. And if traditional drives connected via the SATA interface can be replaced in the hot swap mode, then the main memory must be initially selected as reliable and stable as possible. In cloud data centers and in heavy analytical systems, RAM plays a key role, and Kingston already has a proposal created specifically for new processors.
The Purley platform allows the installation of RDIMM register memory modules or LRDIMM or 3DS LRDIMM modules for energy efficiency. With the advent of the new Intel and AMD platforms, Kingston has certified its memory modules for innovative server platforms.
By the way, note that the Kingston server memory line is now labeled KSM (Kingston Server Memory), not KVR, KCP, KTH, KTD, KTL, KCS - or something else. So far, this concerns modules with a speed of 2666 MHz, but all new Kingston branded server memory modules will be marked KSM, including those operating at higher frequencies, which are scheduled for release in 2018. However, in the case of Xeon Scalable, this does not matter yet, since the top-end Intel Xeon Scalable has a built-in memory controller at a frequency of 2666 MHz and simply does not need faster memory in Purley. And for real tasks, the most expensive chips are not always needed at all. In most cases, you can get by with the Gold 51xx, Silver 41xx and Bronze 31xx processors on the same architecture,
As you can see, with a reasonable approach, you can also save on memory, since Kingston, of course, offers products with any frequencies from the above scheme. It is enough to determine the range of tasks that the server will perform and install in it a memory corresponding to the processor capabilities. For example, in the Bronze 31xx series, it makes no sense to buy even DDR4-2400 MHz, since the processor will not use its capabilities.
Ordering a new generation of memory — KSM modules — has become noticeably simpler. There are no more marking differences. If you bought Kingston memory for servers, you are well aware that before we had two types of server memory - Server Premier and Value RAM. All KSM memory has the properties of Server Premier, despite the fact that the price of modules has been reduced in relation to the premium series. In addition, if earlier it was necessary to check whether the memory marking has the suffix “i” (speaking of Intel certification), now you can forget about it - the entire KSM series is initially certified. Therefore, new products will be easier to choose for both builders and users of server systems.
All KSM modules use a fixed BOM (Bill of Materials). This means that Kingston experts carefully select the manufacturers of the chips themselves and allow only the highest quality products in the series. Engineers test each memory cell at the production control stage, and also test the printed circuit boards themselves. We control everything, up to the revision of the chips and the manufacturer of the register chip. Thus, the KSM series modules are Kingston's most carefully controlled memory series for professional applications.
All memory information is now easily read in its part number. For example, if you read the following number on a module:
This will mean that the manufacturer of the H chip is Hynix, the revision of the chip is A, and the manufacturer of the register chip is IDT. By the way, different companies can be manufacturers of register chips. In addition to IDT (I), components from Rambus, previously known as Inphi ® and Montage (M), are also used.
In general, more transparent marking not only reflects Kingston's more holistic approach to Server memory release, but also helps to upgrade the modules by controlling all parameters, up to the chip manufacturer. This will avoid possible conflicts or reduce the productivity of equipment due to incomplete compatibility, as well as purchase the same type of modules for several types of servers, simplifying the logistics and maintenance of systems.
Conclusion
To summarize a short summary. To get the most out of the new platform, you need to carefully select all the components and use all the Intel optimization tools that help take advantage of the new processors and the Purley platform as a whole. The advantages of Kingston memory for servers are already used by leading hosting companies , and if you install the most productive and reliable modules, switching to Intel Xeon Scalable will give the maximum possible effect for solving difficult tasks - from virtualization to analytics and modeling.
Subscribe and stay with us - it will be interesting!
For more information on Kingston and HyperX products, visit the company's official website..
Our home country is wide, so we prepared stocks at once in several network partner stores - everyone should have the opportunity to get HyperX accessories as close to home as possible. Until January 12, you can purchase peripherals with maximum discounts.
• 1000 rubles discount for HyperX Cloud Stinger headset
• 2000 rubles discount for HyperX Alloy FPS keyboard with Cherry MX switches (optional Brown / Blue / Red)
• 1000 rubles discount for HyperX Cloud Silver headset
• 700 rubles discount for mouse HyperX Pulsefire FPS
