Algorithmic Surrealism: A Guide to High Frequency Trading for 900 Million Microseconds. Part 1

Original author: Brett Scott
  • Transfer


Translator's note: In the ITinvest blog on Habré, we write a lot about technologies related to trading on modern exchanges. Today we present to your attention the first part of an adapted translation of an article by a London financier and trader Brett Scott, in which he talks in detail about the phenomenon of high-frequency trading (HFT). This material will help to better understand all the advantages and disadvantages arising from the activities of such high-frequency traders.

While you are reading this sentence, an algorithm for high-frequency trading (HFT) that communicates with the stock exchange through the “low-latency” trading infrastructure might be able to conduct 1000 transactions.

I say “maybe” because it depends on what kind of pause you take in places where commas are placed. If during reading you prefer to stop after each comma, then you give the algorithm a chance to place several hundred more orders.

I will try to clarify the situation. This means that computers located in a certain place that the company owns (or rents) can: 1) access the data of the stock exchange; 2) process them using a coded system of sequential operations (algorithm) and decide whether to trade or not; 3) send back to the stock exchange a message with an order to buy / sell shares of the company - for example, a company manufacturing children's toys; 4) wait until the order is executed, and receive confirmation; 5) repeat all these steps, say, 250 times per second.

Honestly, these are also not entirely accurate numbers. Only a few know how fast algorithmic engines actually perform trading operations. But even if they make 50 or even 10 transactions per second, it is still incredibly fast.

Given that I was engaged in trading in financial markets - albeit in slower OTC swap markets - and participated in various promotions related to financial trading, this topic is very interesting to me. However, the purpose of this article is not so much to convince you whether it is good or bad to do HFT trading. Rather, try to look at this phenomenon from a different angle and learn to understand discussions and news, where, without a doubt, the discussion of this topic will continue.

1.1: More on HFT




People: So Meeled

Once upon a time, back in the 1970s, only people were the actors in the stock market trading. Regardless of whether it was prudent investors buying a stake in a pension fund, or desperate speculators, in a hurry buying up or selling these shares, the process was always limited by human thinking, as well as the time during which it was necessary to pick up the phone and transfer the order to execution. Even the fastest speculator would need a few minutes to close the deal.

Today, the situation is different. The use of information technologies, coding methods and communication tools allows traders to automate the processes of human thinking, turning them into algorithms that are implemented due to the movement of light rays through fiber optic cables. So the time required to complete the transaction was reduced to a few milliseconds or even microseconds, and this is a few thousandths and millionths of a second.

Thanks to this, such a "surreal" area as high-frequency trading has appeared. It did not arise out of nowhere: its methods have been developed since the advent of " software trading"In the 1980s, the speed of which gradually increased. However, only recently have people begun to increasingly monitor the activities of HFT firms. In particular, HFT trading attracted attention during the so-called “ Black Crash” (Flash Crash) in 2010, when for some reason there was a sharp collapse of the US stock market, recovering after a few minutes - an event that many attributed to the incorrect operation of the HFT- algorithms.

1.2: How should I feel about this?




Mixed feelings?

I'm not sure I know how you should relate to all this. People are used to worrying about things that do not pose any threat, and to remain completely indifferent to what can cause serious harm. Nevertheless, it can be assumed that for most people who are engaged in certain work at their regular work, the idea of ​​using robots that make 100 transactions for the time that you take a sip of tea is difficult to perceive. Such work may seem unnatural, too complex, unreliable, or just plain weird.

Even if the algorithm doesn’t do anything bad, it’s hard to explain what its work and the work of its creators are. For example, engineers in the field of rocket science are doing something incredibly complicated, and I can’t say what exactly their work consists of. However, I can imagine what they do in principle: they create aircraft that allow people to travel long distances. What then is the creator of the algorithms for high-frequency trading actually doing?

Well, we know that they make money, but as a rule, people earn it, bringing some benefit to society: they connect pipes in our toilets, make business cards for us, or slaughter cattle to make hamburgers for us. However, if we asked: “What is the purpose of high-frequency trading?”, People would probably think before answering. The unusual nature of the methods and goals of such work naturally gives rise to the suspicion that this is just another scam conducted by the emboldened financial elite for the sole purpose of profiting from ordinary citizens.

Of course, financial professionals working in the field of HFT trading can find those who are not enthusiastic about their activities, superstitious, illiterate ignoramuses who do not know anything about the markets. They are trying to tell such people that they are not familiar with the HFT field, and they better calm down:

“We are modern scientists and rationalists, so stop talking nonsense. This is a natural process: we would not receive money if there were no “demand” for our services, would we? ”

They find helpers in the person of qualified economists and begin to describe the problem from a moral point of view:

“We have a positive impact on the markets by offering liquidity and pricing services. If you stop us, you will regret bitterly. ”

1.3: From trading to technical trading, algorithmic trading and HFT trading




Let's go back a bit and try to consider this phenomenon in the context of the current situation. Financial markets, in particular stock markets, stimulate the purchase and sale of financial instruments - contracts that give the right to a gradual return of funds. At different times, different players may be participants in such markets. On the one hand, there are large investment organizations, such as pension funds. From time to time, they enter the market and make large investments, buying up a large number of shares, often in order to own them for several years. On the other hand, there are faster, less constant players — we call them traders — who make money by quickly entering and leaving the market, like agile sharks swimming between large flocks of large whales.

However, there are several types of trading. If you want to imagine how high-frequency trading came about ...

  1. To get started, understand the basic idea of ​​trading : Financial traders are involved in the purchase and sale of financial instruments, for example, shares of some companies. They hope to purchase them at a better price, and then sell them more expensive, thus making a profit from the transaction.
  2. Now try to understand the essence of technical trading : Traders use a variety of methods of speculation. For example, they can spend several hours studying the archives of a particular company. This method is called fundamental trading . In addition, they can analyze the activities of other players in the market and make an appropriate decision. Such "technical analysis" of data on prices, orders and transaction volumes conducted by other traders forms the basis of technical trading .
  3. Then imagine how this process is automated : The technical trading process can be automated, for example, by developing an algorithm that analyzes the incoming stream of data on prices, orders and trading volume and, under certain conditions, makes deals. We call this process algorithmic trading m. (Note: you can find the difference between algorithmic and automated trading, but for convenience we will assume that this is one and the same).
  4. Now imagine how this process accelerates : If you “accelerate” the process of automated / algorithmic trading to high speeds, you get high-frequency trading.


Thus, HFT-trading is easiest to imagine as a very fast algorithmic trading, which is an automated technical trading, which, in turn, is a kind of trading as a whole. It can be compared, for example, with a slower fundamental trading, which is carried out by specialists like George Soros (he and his analysts monitor everything that happens in the world from his office and then make forecasts for certain events). In the end, do not forget that the whole world of trading can be compared with the world of long-term investment, which is what large pension funds do. Returning to the analogy with marine animals, HFT companies are piranhas swimming next to sharks and whales.

1.4: How to create an HFT company




“It is necessary. To be. Faster. Everyone! ”

The reasons for the entry of trading organizations into the HFT business may vary slightly. For example, some banks use HFT trading in order to get a large order and break it into many small orders. It’s like using a nozzle on a fire hose so that instead of a regular jet, you get sprayed liquid, barely visible to the eye. At the same time, most HFT players are speculators working with short-term deals, firms specializing in proprietary trading, and hedge funds. If you intend to open one of these companies, you need to know the following.

First, you need start-up capital: it will be either your personal savings, or the money of very rich people. Secondly, register a company and capitalize it (you can create a management company in London, where you will actually work, then create a separate company in the Cayman Islands, where your money will be stored, and then draw up a contract according to which the London company will work for a company from the Cayman Islands).

Then you hire several employees, perhaps through a dedicated recruitment agency or by looking at LinkedIn HFT profiles . Maybe you will hire Steve , who knows how to work with expensive equipment for HFT trading, or Fabiowho can write C ++ code and develop the necessary software. Mark’s profile says almost nothing. It can be assumed that he works exclusively with recruitment agencies, and this is typical for Cambridge graduates and former Goldman Sachs employees.

Offer your talented employees a ton of money so they can develop several algorithms. Start by developing a concept, after which your programmer will write C ++ code for it . You may even need to patent your algorithms or system architecture.



You will not find in this article a full description of the true nature of HFT technology. If you are interested in details, then the Internet is full of abstruse discussionsand documentation, overloaded with terms, but the essence of the process is this: you must somehow agree with a brokerage firm and a stock exchange so that your ingenious algorithm is placed as close to the stock exchange as possible! You need to minimize the physical distance between the computer that implements your algorithm and the computer that runs the system for selecting the appropriate orders [eng. order-matching system] of the given exchange in order to establish a reliable high-speed dialogue between them.

Around building low latency infrastructure that would provide direct market access, a separate purely technical subregion is formed. In a normal situation, if a person wants to buy or sell shares, he needs to contact a broker who is a member of the exchange on which he is going to trade. But for an HFT trader, this is sooooooooo long. In this case, you need to get to the very heart of the exchange, without going through the usual brokerage procedures. Ideally, it is better to find a way to "settle in the neighborhood" [Eng. co-locate] with the exchange ( for the Moscow Exchange , ITinvest offers such services - approx .). The meaning of this unusual phrase is that you literally need to install your computer in a room located next to the computers on the exchange.

If you want to understand the topic of “neighborhood”, you should familiarize yourself with the services of the NYSE exchanges ,Nasdaq , the London Stock Exchange , Eurex , CME and even the Johannesburg Stock Exchange . The Tokyo Stock Exchange on its website talks about the difference between its area "in the neighborhood" and the area near the exchange (both provide a power source of 100-200 volts, but in the area "next door" there is a more advanced 8kVa cooling system, so this area will be preferable in if your algorithm can melt the computer). Here you can watch her promotional video on the same topic. Exchanges provide a wide range of services to "connect" to them. These may include the use of high-quality cable and cooling systems, as well as the allocation of an additional communication channel.

I'm not talking about the fact that the whole process is described (at first) in complex terms , for the most part related to technology , but in the end it comes down to a rather simple formula: you rent a computer near the exchange; upload your algorithms to it; then, using these algorithms, the exchange transmits a large data stream by cable; your algorithms process this data and send orders to the exchange via the same cable. And you need to make sure that this whole process goes faster than that of your competitors. You can sit in the office a few kilometers from the exchange, monitoring the progress of the process and trying to improve your system.

If you need help installing equipment, you can contact one of the support services or consulting companies specializing in trading infrastructure with low latency. Among them are Sungard , Cisco , Algospan , Interactive Data, and Lato Networks . Or you can ask a question to professionals - the HFT companies themselves, who have already formed their infrastructure. Like most other influential secret organizations, these companies have unusual little-spread names and uninformative and somewhat obscure sites. For example, you can look at Virtu , ATD , KCG , Tradebot ,Tradeworx , Liquid Capital , Chopper Trading , Citadel Tactical Trading , Tower Research, and RGM .

1.5: Mastering the Art of Electronic Warfare




We pump our own algorithm

. Each company has its own strategy . Some apply statistical analysis and various arbitration methods , while others use their knowledge of the "microstructure of the market", which supposedly consists in understanding the technical aspects of the functioning of exchange systems and their use for personal gain. Someone, in turn, is engaged in flash trading , which is considered a legitimate lead game. front-running]. In addition, you can “ hammer ” the market with your orders [eng. order stuffing] (a tactic that Dave Lauer, an informant from the HFT trading industry, calls a financial DDoS attack ).

You could use the principle of leering[eng. layering], perhaps in order to cause a “momentum kindling” [eng. momentum ignition ], which is considered one of the types of easy market manipulation. Some use aggressive trading strategies aimed at carefully following trends and using opportunities, while others act more passively, as, for example, when using electronic Aikido bots that consume a minimum of energy. If you are interested in some of the strategies, I recommend reading this article by Irene Aldridge. This material may also be useful.

By the way, if you want to understand the nature and essence of this phenomenon, it will always be useful to read discussions on technical forums for professional financial experts, such as Wilmott forums . These experts follow the latest news from the world of finance and, as a rule, know this area to the smallest detail. If you want to understand the terminology, I advise you to read the materials of the Quantumar user, who likes to scatter abstruse financial phrases (even if you do not understand anything, it would be useful to at least try to catch the point):

“[High-frequency trading] for the most part is only an accelerated version of arbitration, which is carried out either on the futures market or on shares traded on the OTC market and sold / acquired again. In this case, all or most of the income is generated from bonuses received from stock quotes. Some firms do instant trading. momentum trading] at a speed of several milliseconds: when these firms find out that someone is going to place their orders, they place them first (since they have an advantage in speed), move the market one cent up and then enter into a sell deal with original buyer ...

In addition, they can use instant warrants. flash orders] to jump over larger orders. Some delve into the details of accounting to make more profitable deals. There are several other strategies used when working with securities: some companies study options markets and arbitrage using delta hedgers ... Most strategies are no longer associated with mathematics, but with the microstructure of markets ...

HFT organizations operate at ultra-high speeds: they can place several million orders per day, depending on the number of markets and trading activity. Such companies require very good C ++ programming skills, knowledge of API development, and minimal computer architecture knowledge, for example, how to bypass stack protection and apply the so-called kernel trick .

They also need LAN / WAN specialists who can help transfer data to the network a few microseconds faster. Such companies use expensive specialized equipment. An ordinary switch with decent speed will cost 50 thousand [dollars - approx. transl.]. Information owned by HFT organizations is available to all traders. The only difference is that these organizations manage to use this information in the auction before it appears on your computer; in how quickly they receive it and act in the market. The delays of their networks and computers are not even milliseconds, but a matter of microseconds. "

1.6: (Highly specialized) academic discussion




- Plato, according to our data, HFT-trading improves market liquidity.
- I do not care, your data sucks. I get my data directly from the Agora.


In addition to all these explanations on YouTube, reports of journalists and discussions of the topic of HFT trading on forums, it is obvious that research is being conducted in this area. If you are looking for a full-fledged scientific explanation of these or those phenomena, and not tips like “This is what really happens ...” from Quantumar, you can read scientific journals. Research is carried out, of course, in areas such as finance and economics, but besides them, also in a number of other disciplines.

A friend of mine who teaches finance at the university noted that one of the possible problems with studying HFT trading is that scientists rely on data provided by HFT firms and thus are constantly dependent on these firms, possibly limiting their own capabilities. It is worth adding that scientific articles are trying to write as boring and highly specialized as possible. It seems that what is happening does not depend on politics, culture, history or even on the people themselves, and the articles are terribly uninteresting and difficult to perceive. In addition, research subjects have a very narrow focus with a special bias towards the influence of HFT trading on liquidity and pricing .

In essence, liquidity means ease of bidding. If, having entered the market, I can immediately buy or sell something, then its liquidity is high. However, if it takes a long time to start trading, then liquidity is low. One controversial issue regarding HFT trading and liquidity is whether HFT trading adds or only absorbs liquidity. In other words, do HFT companies facilitate trading with other participants or, conversely, prevent them from trading? This also includes questions about whether they offer real liquidity or not. For example, trading robots can constantly show that they are ready to enter the auction, but they themselves disappear.

“Pricing” is another ingenious concept that denotes a process as a result of which the actual price of a certain product is formed by a group of market participants, 1) responding to information 2) by placing orders to buy or sell 3) using a certain market infrastructure. Therefore, if it is announced that one company should go bankrupt, and the value of its shares increases sharply, then something went wrong in the pricing process. The question is, does HFT trading help determine the real situation on the market or only leads to instability and all kinds of anomalous phenomena like “Black Tuesday”?

Moreover, a new field of research is emerging, the purpose of which is to find out whether the activities of HFT companies are legitimate or if they somehow manipulate the market and “play ahead of the curve” to the detriment of other participants. And finally, articles are beginning to appear on the role of man in HFT trading, on the people who conduct all these operations, on the political and cultural aspects of this phenomenon, and how it reflects changes in the global economy.

1.7: Research (and lobbying) influences authorities




“Provide your evidence”

Most news about HFT trading today is related to political battles and threats from government agencies to stop the activities of HFT firms. In theory, the opinion of the authorities should take into account scientific discoveries, but, besides this, there is a suspicion that lobbying also affects their opinions .

Lobbying in itself is a demonstration by a group of people of certain scientific articles to government bodies. For example, members of Modern Markets Initiative selected the most suitable collection of scientific papers to confirm the opinion that HFT companies are implementing market utopiain which “the savings of individual investors are carried out by lowering transaction prices”, and “the market is becoming more democratic”.

When the authorities are not overwhelmed with the requests of such organizations (obviously acting in their own interests), they themselves try to study such scientific articles, stamping their reports and summaries . In addition, they accept applications from organizations that are negatively inclined towards HFT trading. Among them are associations such as Themis Trading , an organization, according to Michael Lewis, “who has done more than anyone to explain to everyone how robbers act in the modern stock market” (you can see their huge collectionarticles criticizing the activities of HFT firms). These include Nanex, the data provider, and the aforementioned David Lauer . Here you can also meet a number of daring financial critics who conduct thematic blogs about the negative impact of HFT companies.

Among the measures taken by the authorities, it is possible to single out taxes on HFT firms (similar to the introduction of taxes on financial transactions) and restrictions on the speed of trade, the size of orders and the OTR coefficient [eng. order-to-trade ratio] (the number of orders that a trader can place in relation to the number of transactions conducted on it). These discussions are at various stages in the USA, Europe and Asia. As an example, you can readThe High Frequency Trading Act in Germany.

Incidentally, it is also worth following the development of events in the struggle of the authorities against speculation in the commodity market. Commodity exchanges, such as the CME Group, pointed to a single article written by an employee of the University of Illinois to show why speculation does not adversely affect price destabilization; although many other studies prove otherwise. HFT companies, as well as companies engaged in trading in commodity markets, enjoy the current confusion and slowness of public authorities. And those now take the position of “Innocent until proven guilty” instead of “Until justified, precautionary measures must be taken.”

To be continued…

Also popular now: