Arbitration system for beginners, part 1

About 7 years ago, he had experience writing a terminal for the Moscow Exchange. Part of the team was fond of algorithmic trading, including me. However, I never perceived this business as a real source of income, although there were some successes in this. It is clear that to compete with banks and various funds, with their teams of mathematicians and programmers is difficult and easier to implement in other areas.

Now on hearing of cryptocurrency, there is a huge number of exchanges. Based on the assumption that the difference in rates on different exchanges, you can earn, I decided to explore the possibility of creating an arbitration robot. But mainly to start learning python with a real example. So let's get started.

First of all, it is required to detect currency pairs for which arbitrage trading is possible. We need couples who, as a first approximation, are actively trading, and prices on different exchanges diverge and converge.

Offhand, the work plan should be:

• Creating a database where prices of currency pairs will be stored.
• The server that will save data to the database.
• Primary analysis.

Sources are available by reference arb_analysis .

Database creation

In order to store data, you need 3 tables.

A list of stock exchanges will be stored in this table.

CREATETABLE`exchange` (
      `id`int(11)     NOTNULL AUTO_INCREMENT,
      `name`varchar(50) NOTNULL,
	  PRIMARY KEY (id)
    ) ENGINE=MyISAM;

List of cryptocurrency pairs.

CREATETABLE`market` (
  `id`int(11)  NOTNULL AUTO_INCREMENT,
  `name`char(50) NOTNULL,
  `id_exchange`int(11)  NOTNULL,
  PRIMARY KEY (id)
) ENGINE=MyISAM;
CREATEINDEX id_exchange ON market (id_exchange);

The table with transactions, information will also be stored here, according to data from the stock exchange glass.

CREATETABLE`ticker` (
  `id`int(11) NOTNULL AUTO_INCREMENT,
  `id_market`int(5) NOTNULL,
  `local_time`int(9)    NOTNULL,
  `timestamp`int(9)    NOTNULL,
  `last`DECIMAL (11,11) NOTNULL,
  `low`DECIMAL (11,11),
  `high`DECIMAL (11,11),
  `bid`DECIMAL (11,11),
  `ask`DECIMAL (11,11),
  PRIMARY KEY (id)
) ENGINE=MyISAM;
CREATEINDEX id_market ON ticker (id_market);

Receiving data from exchanges

For convenient access to the exchanges there is an open source project ccxt. With the help of which it is possible to apply to different exchanges in the same style. However, it turned out that not everything is so rosy, information could not be obtained on a number of exchanges, and some methods did not work.

In the file create_markets.py, initialization takes place, we get a list of cryptocurrency pairs, by exchanges. This uses the load_markets () method, which returns a list of pairs for the exchange.

name_exchange = ["acx", "binance", "bitfinex", "bitfinex2", "wex"]
defcreate_exchange_and_market_in_db():
    exchanges = {}
    for id in name_exchange:
        exchange = getattr(ccxt, id)
        exchanges[id] = exchange()
        id_exchage = db_helper.insert_exchage_to_db(exchanges[id],cnx,cursor)
        markets = exchanges[id].load_markets()
        for mark in markets:
            id_market = db_helper.insert_market_to_db( id_exchage, mark, cnx,cursor)

Next in the ex_data_saver.py file, start saving the price change for pairs:

defsave():
    markets = db_helper.get_data_exch()
    exchanges = {}
    for id in name_exchange:
        exchange = getattr(ccxt, id)
        #isHas = exchange.hasFetchTickers#if isHas:
        exchanges[id] = exchange({
            'enableRateLimit': True,  # or .enableRateLimit = True later
        })
    cnx = db_helper.CreateConnection()
    cursor = cnx.cursor()
	loop = asyncio.get_event_loop()
    whileTrue:
        start_time = time.time()
        input_coroutines = [fetch_ticker(exchanges, name) for name in exchanges]
        exch_tickers = loop.run_until_complete(asyncio.gather(*input_coroutines, return_exceptions=True))
        count_exchange = 0
        delta = time.time() - start_time
        for tickers in exch_tickers:
            if  tickers isnotNone:
                count_exchange+=1
        inserted_start = time.time()
        db_helper.insert_tick(markets,exch_tickers,cnx,cursor)
        inserted_time = time.time()
        print(count_exchange," ", delta, ' ', inserted_start - inserted_time)

Asynchronous receiving of ticks for a specific pair is performed using the ccxt fetchTickers () method.

asyncdeffetch_ticker(exchanges, name):
    item = exchanges[name]
    try:
        ticker = await item.fetchTickers()
        return {name:ticker}

Preliminary data analysis

First of all, it is interesting on which exchanges and on which pairs, the most active trading takes place, we are interested in liquid pairs. To do this, you need to count the number of transactions with grouping by exchanges and specific pairs. As a result, we get a pair, which is active trading.

SELECT ex.name as exchange_name, m.name as market_name, count(*) as count_deals 
FROMexchange ex
LEFTJOIN market m ON m.id_exchange = ex.id
LEFTJOIN ticker t ON t.id_market =m.id 
GROUPBY ex.id, t.id_market
ORDERBY m.name
HAVING count_deals > 10000;

Using SQL queries, you can find various patterns and filter out the data, but for detailed analysis, you need to create a test robot that works on data accumulated from various exchanges.

The next article will be devoted to the creation of this robot. And a test server - emulating the work of a real exchange. In the test server I want to lay down the following points:

• Latency
• Slip.
• Commission.