How Python Helps Replace Financial Advisors

In continuation of the article on the dangers of excessive diversification, we will create useful tools for selecting stocks. After that, we will do a simple rebalancing and add unique conditions of technical indicators, which are so often lacking in popular services. And then we compare the profitability of individual assets and different portfolios.

In all this, we use Pandas and minimize the number of cycles. We group the time series and draw graphs. Let's get acquainted with multi-index and their behavior. And all this in Jupyter in Python 3.6.

If you want to do something well, do it yourself.
Ferdinand Porsche

The described tool will allow selecting the optimal assets for the portfolio and excluding instruments imposed by consultants. But we will see only a general picture - without taking into account liquidity, time for recruiting positions, broker fees and the cost of one share. In general, with monthly or annual rebalancing of large brokers, this will be a minor cost. However, before applying the chosen strategy, it is still worth checking in the event-driven back tester, for example, Quantopian (QP), in order to eliminate potential errors.

Why not immediately in QP? Time. There the easiest test lasts about 5 minutes. And the current solution will allow you to check hundreds of different strategies with unique conditions in a minute.

Loading raw data

To download the data, take the method described in this article . I use PostgreSQL to store daily prices, but now there are a lot of free sources from which you can generate the necessary DataFrame.

The code for downloading the price history from the database is available in the repository. The link will be at the end of the article.

DataFrame structure

When working with price history, for convenient grouping and access to all data, the best solution is to use a multi-index (MultiIndex) with a date and tickers.

df = df.set_index(['dt', 'symbol'], drop=False).sort_index()
df.tail(len(df.index.levels[1]) * 2)

Using a multi-index, we can easily access the entire price history for all assets and can group the array separately by dates and assets. We can also get a price history for one asset.

Here is an example of how you can easily group history by week, month, and year. And all this is shown on the graphs by Pandas:

# Правила обработки колонок при группировке
agg_rules = {
    'dt': 'last', 'symbol': 'last',
    'open': 'first', 'high': 'max', 'low': 'min', 'close': 'last',
    'volume': 'sum', 'adj': 'last'
}
level_values = df.index.get_level_values
# Графики
fig = plt.figure(figsize=(15, 3), facecolor='white')
df.groupby([pd.Grouper(freq='W', level=0)] + [level_values(i) for i in [1]]).agg(
    agg_rules).set_index(['dt', 'symbol'], drop=False
    ).close.unstack(1).plot(ax=fig.add_subplot(131), title="Weekly")
df.groupby([pd.Grouper(freq='M', level=0)] + [level_values(i) for i in [1]]).agg(
    agg_rules).set_index(['dt', 'symbol'], drop=False
    ).close.unstack(1).plot(ax=fig.add_subplot(132), title="Monthly")
df.groupby([pd.Grouper(freq='Y', level=0)] + [level_values(i) for i in [1]]).agg(
    agg_rules).set_index(['dt', 'symbol'], drop=False
    ).close.unstack(1).plot(ax=fig.add_subplot(133), title="Yearly")
plt.show()

To correctly display the area with the chart legend, we move the index level with tickers to the second level above the columns using the Series (). Unstack (1) command. With DataFrame (), this number will not work, but there is a solution below.

When grouping by standard periods, Pandas uses the latest calendar date of the group in the index, which is often different from actual dates. To fix this, update the index.

monthly = df.groupby([pd.Grouper(freq='M', level=0), level_values(1)]).agg(agg_rules) \
          .set_index(['dt', 'symbol'], drop=False)

An example of getting the price history of a certain asset (we take all the dates, the ticker QQQ and all the columns):

monthly.loc[(slice(None), ['QQQ']), :]  # история одно актива

Monthly Asset Volatility

Now we can see in a few lines on the chart the change in the price of each asset for the period of interest to us. To do this, we obtain the percentage change in price, grouping the dataframe by the multi-index level with the asset ticker.

monthly = df.groupby([pd.Grouper(freq='M', level=0), level_values(1)]).agg(
              agg_rules).set_index(['dt', 'symbol'], drop=False)
# Ежемесячные изменения цены в процентах. Первое значение обнулим.
monthly['pct_close'] = monthly.groupby(level=1)['close'].pct_change().fillna(0)
# График
ax = monthly.pct_close.unstack(1).plot(title="Monthly", figsize=(15, 4))
ax.axhline(0, color='k', linestyle='--', lw=0.5)
plt.show()

Compare the return on assets

Now we use the Series (). Rolling () window method and derive the return on assets for a certain period:

Portfolio Rebalance Methods

So we got to the most delicious. In the examples, we will look at the results of the portfolio in the allocation of capital on pre-determined shares between several assets. And also we will add unique conditions according to which we will refuse some assets at the moment of capital distribution. If there are no suitable assets, then we will assume that the capital lies with the broker in the cache.

In order to use the Pandas methods when rebalancing, we need to store the distribution shares and the rebalancing conditions in the DataFrame with grouped data. Now consider the rebalancing functions that will be passed to the DataFrame (). Apply () method:


In order:

• rebalance_simple is the simplest function that will distribute the profitability of each asset by shares.
• rebalance_sma is a function that distributes capital to assets that have a 50-day moving average higher than the value for 200 days at the time of rebalancing.
• rebalance_rsi is a function that distributes capital among assets whose RSI indicator value for 100 days is above 50.
• rebalance_custom is the slowest and most universal function, where we will calculate the indicator values ​​from the asset's daily price history at the time of rebalancing. Here you can use any conditions and data. Even download every time from external sources. But without a cycle can not do.
• drawdown - auxiliary function, showing the maximum drawdown of the portfolio.

In the rebalancing functions, we need an array of all the data on the date in terms of assets. The DataFrame (). Apply () method, by which we will calculate the results of the portfolios, will pass an array to our function, where the columns will be the row index. And if we make a multi-index, where tickers will be the zero level, then the multi-index will come to us. We will be able to expand this multi-index into a two-dimensional array and get the data of the corresponding asset on each line.

Rebalancing portfolios

Now it is enough to prepare the necessary conditions and make a calculation for each portfolio in a cycle. First of all, let's calculate the indicators on the daily price history:

# Смещаем данные на 1 день вперед, чтобы не заглядывать в будущее
df['sma50'] = df.groupby(level=1)['close'].transform(lambda x: talib.SMA(x.values, timeperiod=50)).shift(1)
df['sma200'] = df.groupby(level=1)['close'].transform(lambda x: talib.SMA(x.values, timeperiod=200)).shift(1)
df['rsi100'] = df.groupby(level=1)['close'].transform(lambda x: talib.RSI(x.values, timeperiod=100)).shift(1)

Now group the history under the desired rebalancing period using the methods described above. We will take in this case the values ​​of the indicators at the beginning of the period in order to exclude a peek into the future.

We describe the structure of portfolios and indicate the necessary rebalancing. Portfolios are calculated in a cycle, since we need to specify unique shares and conditions:


This time we will need to turn the trick with the indices of columns and rows to get the desired multi-index in the rebalancing function. We achieve this by calling the DataFrame (). Stack (). Unstack ([1, 2]) methods in succession. This code will transfer the columns to the lower-order multi-index, and then return the multi-index back with tickers and columns in the desired order.

Ready portfolios for graphics

Now it remains to draw everything. To do this, once again run a cycle of portfolios, which will display data on the graphs. At the end we will draw SPY as a benchmark for comparison.

Conclusion

The considered code allows you to select different portfolio structures and rebalancing conditions. With it, you can quickly check whether, for example, it is worth holding gold (GLD) or emerging markets (EEM) in a portfolio. Try it yourself, add your own indicator conditions or select the parameters already described. (But remember the survivor’s mistake and that fitting to past data may not meet future expectations.) And after that, decide who you trust your portfolio to - Python or finco-sultant?

Repository: rebalance.portfolio