High Frequency Trading: A Guide from the Flash Boys Book Hero

Original author: Brad Katsuyama
  • Transfer


Translator’s note: Earlier in our blog on Habré, we considered various stages of developing trading systems (there are also online courses on the topic), and even described the development of an event-oriented backtest module in Python . Today we bring to your attention a brief guide to high-frequency trading by Brad Katsuyama - the famous quantum and bestselling hero Michael Lews " Flash Boys: A Wall Street Revolt " (we published an adaptation of this work on the blog).

In issueWall Street Week broadcasts Brad Katsuyama talked about high-frequency trading (HFT) and market structure, and we made a review on the topic of HFT-trading and financial regulation in terms of the process of executing orders on the exchange. Video is available here .

What is high frequency trading (HFT)?


In fact, high-frequency trading (HFT) is an electronic trading at a very high speed. Despite the fact that the activities of HFT traders are often criticized, only certain types of HFT trading create chaos in the modern financial market. The line between algorithmic trading, electronic marketmaking and harmful HFT trading is rather blurred, and high-frequency trading often means electronic trading. In fact, the phenomenon of HFT trading in itself is neither good nor bad, but the devil is in the details.

In order to clearly represent the possibilities of HFT trading, it is worth considering in more detail some types of market activity.

What is algorithmic / system trading?

  • Algorithmic / system trading is the common name for the process of applying programmable systems that use a specific mathematical model to automatically complete transactions. A person creates a program on a computer for a specific financial strategy based on this criterion and manages the developed system from this computer.
  • HFT trading is a type of algorithmic trading, but not all algorithmic trading can be considered high-frequency.

What is manual / discretionary trading?

Manual / discretionary trading is a common name for a set of subjective decisions made by a person, usually based on a number of subjective criteria. A manual trader can be a small investor who makes one deal a month, and an intraday trader who makes several hundred deals a day, and a large organization that trades in large blocks of shares at various intervals.

In 2011, the Commodity Futures Trading Commission (CFTC) admitted that it was not “trying to find an exact definition” for high-frequency trading. Instead, she proposed "the seven key features of HFT trading":

  1. Using systems that implement extremely fast placement, cancellation and change of an order in less than 5 milliseconds or with almost minimal delay.
  2. The use of computer programs or algorithms in order to automate the decision-making process, during which the placing, execution, direction and execution of orders is determined by the system and does not require human intervention in the case of each individual order or transaction.
  3. Use of colocation services [eng. co-location - placing foreign servers near the exchange], direct access to the market or a dedicated data channel offered by exchanges and other organizations in order to reduce network and other delays.
  4. Very short time frames for opening and closing a position.
  5. A high daily turnover of the securities portfolio and / or a high proportion of applications in relation to the number of transactions [eng. order-to-trade ratio].
  6. Placing a large number of orders that are canceled immediately or within a few milliseconds.
  7. The end of the trading day in a position as close to zero as possible [eng. flat position] (without holding large unhedged positions at night).

How the stock market works


The stock market was created so that companies could gain access to public investment and, thus, attract financing in order to increase their growth rates. Stock exchanges were supposed to be a place where ordinary sellers and buyers would get together and conclude transactions among themselves in the prescribed manner. However, after the appearance of various intermediaries, the stock market was divided into a large number of segments, and its structure became much more complicated.

At present, trading in the USA is conducted by 11 public stock exchanges, about 50 alternative trading systems (ATS), also called “hidden pools”, and about 200 internalizers (as a rule, these are broker-dealers who can sell / buy securities, acting on your own or someone else's name). The largest public exchanges are the New York Stock Exchange (NYSE) and NASDAQ. Public exchanges are heavily influenced by regulators, while ATS systems, typically controlled by large banks and financial institutions, are less influenced by the authorities.

When you place an order on your online brokerage account, this order can be transferred to a specific exchange or to the ATS system. After submitting your application, you will almost immediately receive confirmation of the fact of the conclusion of the transaction, but you will not be able to see through whose hands your order managed to go in a few milliseconds.

Well-studied HFT strategies


Shortly after the sharp crash of US stock indexes in May 2010 (Flash Crash), the Securities and Exchange Commission attempted to gather more information about the different types of HFT strategies that adversely affect the market.

1. Passive marketing

  • Passive marketing in general consists in placing applications for purchase (bid) and sale (ask) (limit orders) in order to ensure liquidity in the market. Players buying by bid and selling by ask get profit from the commission for adding liquidity, which is paid in accordance with the maker-taker exchange model.
  • Regulators are concerned about the quality of liquidity supposedly provided by passive HFT marketing. Improper stimulation in the maker-taker model leads to leering [eng. Layering - market manipulation by artificially shifting quotes for the purchase and sale of securities] and a high level of early refusals (at least 90%).

2. Arbitration

  • Arbitration strategies are aimed at making profit, based on the difference in prices for the same assets that are traded on different trading floors. Arbitration will always be present in financial markets, but it should not be based on the desire to win in speed and gain access to exchanges in related markets earlier than others.
  • Statistical arbitrage consists in making a profit from the price difference between correlating securities or markets. This type of arbitration uses mathematical modeling methods and can be applied at any time intervals.
  • Arbitration of Delays latency arbitrage] is the use of modern technology in order to obtain an advantage in speed. Delay refers to the time that elapses from the moment a signal is sent until it is received. When discussing the differences between harmful and harmless types of HFT trading, statistical arbitrage is often confused with delay arbitrage.

3. Structural strategies

  • Structural strategies, as a rule, are aimed solely at exploiting vulnerabilities in the market structure and obtaining the necessary market data earlier than others. To do this, you need to place your servers close enough to the exchange or get direct access to its information. Having access to the latest quotation data, companies can earn money on the execution of orders in the event that they find players to which information reaches more slowly.

4. Directional strategies

  • The goal of directional strategies is to profit by predicting the directional movement of securities prices. In this they differ from the three previous types of strategies, assuming unhedged risks.
  • Using a directional strategy with anticipation, the player tries to detect large orders that can affect the price and place his order before them. Thus, he can use these large orders as “free options” (exit points) if the value of the security does not move in the expected direction.
  • When using the strategy of “kindling momentum” [eng. momentum ignition] the company places a group of orders in an attempt to “ignite” a sharp price movement in a certain direction. The essence of the principle, called “spoofing”, is the manipulation of algorithms and manual traders who are forced to conduct aggressive trading. In addition, when using this strategy, the player can cause a further price movement due to the set stop-losses. Spoofing is considered an illegal strategy, but it is difficult to detect and prove.

What is the regulation of the national market system?


The National Market System Regulation (Reg NMS) is a set of rules and regulations approved in 2005 by the US Securities and Exchange Commission and entered into force in 2007. They are aimed at modernizing the US stock exchanges by establishing market equity in pricing, in ways of displaying quotes and in providing access to market data.

The following are the basic rules for Reg NMS:

  1. Rule of profitable execution of orders: It guarantees investors that their orders for securities will be executed at the best price on any exchange. The rule was created to protect investors by eliminating the shortcomings in the traditional trading rules, since they allow unfavorable execution of orders, especially in the case of limit orders. Despite the fact that this rule was developed with a noble purpose, its unforeseen consequences negatively affected the market structure.
  2. Access Rule: A consequence of this rule was the emergence of a system called “Securities Data Processor” [eng. Securities Information Processor, SIP], the principle of which is to collect quotation information on all public exchanges and issue the best buy / sell quotes (NBBO) for each security. Data is transferred from all data centers of exchange exchanges to one central processor, which then issues a universal spread. Regulators introduced this rule in order to eliminate the negative effects of the growing fragmentation of public exchanges, but instead only increased their fragmentation and triggered increased trade in hidden pools.

Unforeseen consequences of the introduction of the rule of profitable execution of orders


An organization that is about to execute a large order is likely to do so in a hidden pool, so as not to disclose its intentions before entering into a transaction. The stock spread consists of the highest bid (sell order) and the lowest ask (buy order). If you place a market order for the purchase of 200 shares, you will be selected an application for sale at the lowest price (given that the size of this application is at least 200 shares), and your order will be executed at this price.

But what if you want to buy 1,000 or 10,000 shares? As the application size increases, more problems arise. Suppose you want to buy 1,000 shares, and the application with the lowest price on the "domestic market" is designed for only 100 shares. Suppose that the price per share in this application for sale is $ 10.10, another 1,000 shares are put up for sale a little more expensive - $ 10.11 per share, and then it is proposed to buy 900 shares at a price of $ 10.15 per share.

If you place a market order for 1,000 shares, you must first execute it at the "best price": in our case, it is 100 shares at $ 10.10. Immediately after the execution of your application, the exchange on which it was executed issues a confirmation of the transaction for 100 shares at $ 10.10 each. After that, you can hope that 900 of the 1,000 shares sold for $ 10.11 will be added to the rest of your bid. However, often things are somewhat different.

As soon as confirmation of the purchase of 100 shares at $ 10.10 appears on the market, the computer may reject the offer to sell 1,000 shares at $ 10.11, and your application will automatically accept the next "best" offer to sell 900 shares at $ 10.15. The average price of your transaction is $ 10,145. In this case, the problem is that it is impossible to accept the offer at a price of $ 10.11 after buying shares at $ 10.10, without giving the market a preliminary signal of its intention.

For a small individual investor, the difference in price may not be critical, but for large organizations like pension and mutual funds [eng. mutual funds], which stores the money of most of the US population, a regular increase in the average transaction price over time can lead to significant financial losses.

How hidden pools appeared


Hidden pools are believed to have gained even greater relevance as a result of economic incentives arising from imperfect legislation. Hidden pools appeared in the 1980s, when large investment organizations tried to find a way to conclude transactions with each other in which there was no need to pay commissions and at the same time remain hidden from the eyes of public exchanges. They wanted to be able to buy and sell large blocks of shares without disclosing their plans to public exchanges and, thus, conducting more profitable transactions.

In 1998, the Securities and Exchange Commission laid the foundation for today's market structure after the regulation of alternative trading systems (Reg ATS) came into force, and then in 2007 introduced a number of additional regulation for the national market system (Reg NMS).

Increased hidden pool trading


More transparent public exchanges are usually referred to as “white markets”, while less transparent alternative trading systems and hidden pools are usually referred to as “shadow markets”. The situation described above is one of the main reasons for the development of trade in the "shadow markets", conducted, in particular, by large investment organizations.

In 2005, even before the regulation of the national market system came into force, hidden pools accounted for 3-5% of the market. Today, this figure is 15-18% and continues to grow. Approximately 40% of the total volume of transactions goes through the reporting system on the trading activities of enterprises [eng. Trade Reporting Facility, TRF]. It covers all “over-the-counter trading”, including the activities of alternative trading systems, trading in the “upper tier” market [eng. upstairs trading] (when exchange traders directly negotiate with each other and enter into major transactions) and the retail flow of applications. In addition, Tabb Group estimates that high-frequency trading accounts for 54% of the derivatives market.

Why? In simple terms, large organizations can trade in hidden pools anonymously and not report the closure of a large position. The development of shadow trading has had a negative impact on the natural pricing process. Broker-dealers and major investment banks, taking into account the consequences of the entry into force of new regulation, hastened to organize several hidden pools in order to lure the flow of orders of large organizations.

Now, instead of becoming a meeting place for ordinary buyers and sellers - and originally the stock exchanges were supposed to be this place - for some institutions, they were the last chance to get liquidity. Despite the fact that 60% of trades are still held in the “white markets”, exchanges cause a lot of problems because of the orders involved in the commission payment system in the maker-taker model (and the reverse taker-maker model). And although public exchanges today are a source of stable liquidity, the reality is that their activity and reliability are gradually declining.

Unexpected Consequences of Securities Data Processing


Among other things, the regulation of the national market system provides for the creation of a securities data processor (SIP), which is a regulation norm that allows for the implementation of arbitration strategies. These strategies are one of the main topics that worry opponents of HFT trading.

SIP is a centralized processor to which all exchanges send their market data in order to create a universal “internal market” for each security listed on the exchange. There are two SIP processors in total: one works with securities registered on the NASDAQ and is placed on its technology platforms, and the second works with securities of the New York Stock Exchange and other organizations trading using the NYSE technology platforms. In a word, all 11 public exchanges are connected via cable to SIP processors, which, in turn, collect the received data, analyze it and issue the best quotes (NBBO) of securities.

The process of transferring market data from the exchange to SIP processors takes a split second. So, given the fact that the signal is transmitted at different speeds depending on how close the servers of its clients are to the exchange, a small lag arises in comparison with the “direct data stream”.

What does direct data flow mean? Each exchange has a direct data stream that provides quotation data to those who are “subscribed” to it, faster than others. The closer you are to the exchange server (or use the microwave connection), the faster you will receive and transmit this data. As for the delay, you should understand that we live in a world where several milliseconds or even microseconds can play a decisive role.

HFT-firms are ready to lay out a decent amount of money to place their servers next to the exchange servers. So they can receive data faster than from a slower SIP processor. The exchange engine, which brings together buy and sell orders, was once in the trading floor of the New York Stock Exchange, and a separate specialist worked with it. Today, such engines are placed in large rooms along with the exchange servers.

Proponents of HFT trading claim that everyone can get direct access to the exchange and install their own servers next to it, however, in reality, reputable companies that have the means to develop and implement HFT strategies on a large scale are engaged in this. The services of hosting servers near the exchange bring good profit to it, if we recall that the exchange is primarily a commercial organization. HFT companies pay tens of millions of dollars to install their servers in the exchange building: the demand for the services of providing space, communication, speed and bandwidth is quite high.

As a result, HFT firms receive data faster through direct access to the exchange, while the rest of the market monitors quotes that are transmitted by a slower SIP processor. Thanks to this, HFT companies can use arbitrage delays in order to get ahead of other orders and, thus, make money on a large number of transactions. Formally, such a strategy differs from the “advancing game” [eng. front-running] and, rather, is a loophole that has arisen as a result of a conflict of interest and differences in incentive mechanisms.

Layering and cancellation of orders


If you look at the current state of the US stock market, it will seem to you that this is the most liquid financial market in history, but it has a number of serious drawbacks.

Advocates of HFT trading believe that the stock market is in the most favorable condition for an ordinary individual investor. The spread decreased, the volume of transactions increased, and this leads to a more profitable execution of orders and low commission fees. At first glance, this situation should suit relatively small individual investors who do not enter into major transactions on an ongoing basis.

However, the devastating effect of the arbitration of HFT companies based on delays has affected most large organizations that profit from large transactions. High-frequency trading revenue is distributed throughout the market, but most of it is concentrated in the hands of several HFT firms.

Earlier, we talked about the method by which aggressive HFT companies apply delay arbitrage strategies: in this way, they can manipulate a glass of orders, preventing market participants from executing their orders at the best average price. Although this is only one of the possible scenarios, the market is really clogged with orders that no one was ever going to execute: they rather serve as a decoy to attract other market players and take possession of their money.

When proponents of HFT trading talk about the merits of fast automated trading, they lose sight of the fact that most of the liquidity is just an illusion. It often happens that we see the minimum spread of any shares, but the size of the application in the domestic market can be so small that not a single trader will execute it. When a trader places a small market order, all other larger orders outside the domestic market immediately disappear, and the trader realizes that he will not be able to conclude such a deal. Bank of England chief economist Andy Haldane once said: “High-frequency trading adds liquidity in the rainy season and takes it in the dry season” [in English, the word “liquidity” means not only liquidity, but also the state of fluidity - approx. transl.].

Model maker-taker


With the development of shadow trading, commercial exchanges have to keep the flow of orders from which they receive their income. Collocation services and additional fees make up a significant part of their profits, but exchanges are forced to maintain a stable flow of orders in order to remain competitive.

In order to lure order flows flowing into hidden pools - which can usually make a better offer - exchanges resort to such controversial practices as payment for order flows. Its essence lies in the fact that exchanges and wholesale enterprises pay each individual broker-dealer for sending his order on a specific route. As compensation for the sent order flow, the exchange pays a broker-dealer a fee for limit orders for each share. This remuneration usually amounts to a fraction of a penny per share (0.002 or 0.003 cents), but when it comes to several million shares, the total amount is quite impressive.

One of the variants of such a scheme is used when sending a stream of orders by large organizations: this model is called maker-taker. Not so long ago, it caused real anger of opponents of HFT trading. According to this model, you are considered a “source” (maker) of liquidity when you place buy and sell orders (limit orders) on the stock exchange and receive a reward immediately after the other party accepts the order you placed. At the same time, you become a “taker” of liquidity when you accept buy or sell orders (market orders or aggressive limit orders) and actually pay a fine for it. Over the past few years, the amount of remuneration has grown markedly, as each exchange struggles for its piece of a decreasing pie from the stream of orders still passing through the “white” markets.

The introduction of fees for order flows and the very fact of using the maker-taker model are far from the real purpose of the financial markets, which is to create a platform that brings buyers and sellers together in the prescribed manner and favors natural pricing, which is the result of the law of supply and demand. If the situation had not been distorted as a result of improper stimulation, which contributed to the development of shadow markets, the introduction of remuneration would be unnecessary.

Conclusions and the future of high frequency trading


If Michael Lewis somewhat embellished the current state of the stock market, calling it “technically advanced” in his Flash Boys book, then at least he launched a discussion that is very important for the entire financial industry on whether financial markets are moving in the right direction .

The high-speed arms race has led to a glut and consolidation of the HFT industry. Large fish eat smaller ones; Today, only the fastest HFT companies with the most resources survive. Any HFT-strategy can be changed for yourself, which means that if one HFT-company learns about the strategy of another, then it can develop a high-speed algorithm on its basis and take advantage. If the pie, consisting of the potential profit from arbitration delays, still remains large, over time, many players in the HFT industry will get smaller pieces.

To restore justice in the financial market, we need to improve its regulation and use the opportunities for free trade. New investor-oriented exchanges will help balance the market. Regulation should always be as simple and effective as possible in order to meet the original goal of the stock market, namely to become a place where pairs of buyers and sellers will be organized in an organized manner. Incorrect motivation led to the appearance of entire groups of intermediaries, which does not correspond to our ultimate goal - to complete transactions in a free market.

Also popular now: