February 18, 2016 at 12:09Xeon Phi: Why coprocessors are used to create trading applications
In our blog on Habré we write a lot about the development of trading robots and the construction of infrastructure for online trading. In previous materials, we examined the topic of using FPGAs and GPUs , and today, we will focus on creating trading applications using Xeon Phi coprocessors.
Modern stock exchanges broadcast information about the market situation using special feed handlers, which contain information about stock quotes and orders for the purchase and sale. With the increase in the number of applications and the number of traded financial instruments, the performance of trading systems should also dramatically increase - otherwise delays in trading are inevitable, which is often unacceptable.
In addition, many exchanges broadcast data in various formats, including multicast broadcasting and point-to-point transmission over TCP / IP. The complexity of working with proprietary financial protocols leads to the fact that in some cases financial companies and private HFT traders prefer not to develop their own software processors for financial data flows, but to use commercial hardware solutions to increase the performance of their applications.
There are two approaches to building trading applications. The first one can be described as “general” - in this case, the solution consists of a software application running on some widely used processors like Intel Xeon and standard OSs like Linux. The second approach involves creating a “customized” solution using FPGA or ASIC, which allows you to process financial data at significantly higher speeds. The second way is also characterized by increasing costs of developing and supporting the solution.
An important aspect of any high-performance solution for working in the financial market is also the amount of delay introduced by processing software. In the HFT world, even microsecond delays can make trading unprofitable. The delay of the processors depends on the number of cores that can be allocated for them - usually their number is not particularly large, because the hardware that performs the trading operations also needs hardware resources. The use of Xeon Phi coprocessors, which allow data processing at speeds comparable to the speed of their receipt, helps to solve the problem of reducing the insertion delay.
Package Processing Architecture
Intex Xeon Phi coprocessors allow you to create applications with extremely high levels of concurrency. The computing platform consists of the following components:
- A large number of sequential cache cores and threads to process many independent data streams for optimal system performance.
- SIMD support using 512-bit vectors, instead of using narrower MMX, SSE or AVX2.
- Support for high-performance instructions for calculating square root, exponent, and return values.
- Large amount of available bandwidth for memory.
- High-performance communication tools - to the receiver, PCI-bus between the host and the connected coprocessors.
As a result, one or more streams on the host machine receive packets — this may be market data — through a socket connection from an external source using a standard network instrument (NIC). The host forms a FIFO queue from packets for further processing. The associated thread on the coprocessor side, running on the core, processes each packet received through FIFO and passes each packet through the flow processing algorithm. The result of his work is then again copied to FIFO for further processing.
Architecture of a packet processing application using coprocessors
Effective cross-site communication between two adjacent processes is achieved using the Symmetric Communication Interface (SCIF). As a result, applications transmit data using the SCIF API, which works similarly to the Berkeley Sockets API. SCIF drivers and libraries are symmetrically located on both sides of the PCI bus - the connection setup process is similar to the sockets API, where one side listens and receives incoming connections, and the other connects to the remote process.
Intel Many-Core Platform Software (MPSS) stack, including a SCIF driver and a library for working in user mode
For more information on working with memory and optimizing trading applications using Xeon Phi, see the bookHigh Performance Parallelism Pearls Volume Two: Multicore and Many-core .
Next, we will examine the test implementation of the application on Xeon Phi.
Calculation of LIBOR interest rate swaps using the Monte Carlo method
Xcelerit, a developer of high-frequency trading solutions, published on its blog a description of the implementation of the LIBOR application for processing interest rate swaps using the Monte Carlo method. We will take advantage of the highlights of this material.
The Monte Carlo simulation is used to determine the value of a LIBOR swap portfolio. With its help, thousands of possible future options for the development of LIBOR indicators of interest are simulated - normally distributed random numbers are used for this. LIBOR uses the mechanism described by Professor Mike Giles to calculate LIBOR rates and returns .
Portfolio sensitivity is calculated using algorithmic differentiation.(Adjoint Algorithmic Differentiation, AD). The total values of swaps in the portfolio and the Greeks in Greek letters indicate the sensitivity of the option premium to changes in various values.
The implementation uses the Xcelerit SDK. To get normally distributed random samples, you need a random number generator, then a LIBOR rate calculation module, portfolio and Greek definitions. Then comes the final identification of the portfolio and values of the Greeks.
The table below compares two possible implementations of such an application - one using serial C ++, and the second when using the SDK to work with Xeon Phi.
The application uses the native Xeon Phi mode without using the main processor. The test environment had the following configuration:
- CPU : Haswell Xeon E5-2697 v31 (14 cores, 2x HT) and Xeon Phi 7120P1 (61 cores, 4x HT)
- HT : enabled Hyper-threading
- OS : RedHat Enterprise Linux 6 (64bit)
- RAM : 64GB
- Development Tools : Xcelerit SDK 3.0.0b / ICC 15.0
During the experiment, the computational time of the test application on two processors and the sequential implementation on the same core of Haswell CPU were compared. By computation time is meant the execution of all stages of the algorithm described above.
The graph below compares the speed of the work of the implementation launched on Haswell for double and single precision numbers:
When using Xeon Phi, the results are as follows:
If in the previous case we managed to speed up the processing of 50x and 125x for single and double precision, then with the help of coprocessors these values were increased to 75x and 150x respectively.
