We write an educational application on Go and Javascript to assess the real return on equity. Part 1 - backend
Let's try to write a small training, but quite a complete information system, consisting of the server part on Go and the client web application on Javascript + Vue JS.
First, a few words about what this application is and what it is for. Some time ago I faced the question of how to save some amount of money, which I had formed. Even to me, a person far from the world of finance, it was obvious that keeping money in cash is bad for at least two reasons:
It was decided to study the issue and choose the appropriate investment tool. The main criteria were the reliability and protection of savings from the above risks.
I studied the question and as a result I came to the conclusion that the only adequate investment instrument for a resident of Russia are shares of exchange-traded funds (ETF), and specifically those that are traded on the Moscow Exchange.
Thus, I propose to write an educational application that would show the profitability of all ETFs that are represented on the Moscow Exchange.
You can say that this yield can be viewed on the exchange website itself, and the application, even if it is a training one, should be of some use. I agree, so we will try to display some conditional real stock returns. By this conditional real return, I will understand the return adjusted for inflation in Russia.
In the first part of the article we will analyze the server part of the application. Our backend is written on Go and during development we will try to use such language features as parallel code execution, interfaces, testing, and so on.
TK requirements:
So, let's design the software architecture of the server part of our system.
First , let's figure out the structure of the application packages. According to the statement of work, the application will consist of a web server that will provide the REST API and send files to our web application (later we will write the SPA on Vue). In addition, according to the TOR, we must make several packages for the data warehouse suppliers.
At this point, you should stay in more detail. How can I provide the ability to switch between providers of some functionality in Go? Answer: using interfaces. Thus, we will have to develop an interface (contract) for packages, each of which will fulfill a contract for its own type of storage. The article will consider the storage of data in RAM, but by analogy you can easily add any DBMS. The final package structure will be as follows:
Secondly , let's define the types of data in which we will store the received information and the contract for the storage providers.
We will need data types for stock quotes and inflation. We will take quotes and inflation by months, this scale is quite suitable for such a non-speculative instrument as ETF.
The contract will require methods to populate the repository with data from the Mosbirzh server (initialization) and provide quotes data on request. Everything is very simple.
As a result, in the storage module we put types for storing quotes and an interface:
Inflation data for simplicity is encoded in the server module:
Third , let's describe the end points of our API. There will be only two: for quotes and inflation. Only HTTP GET method.
Actually receiving and processing data from the Mosbirzhi site is carried out in the initialization method. Data is taken according to the Exchange API reference .
What you should pay attention to: we have to use a separate request for each security (and there are already a couple of dozen of them). Execution of data initialization sequentially, in one stream, would take a lot of time. Therefore, we will use the pride of Go - gorutiny. Notice the following piece of code:
In the data initialization function, we parallelize server requests. In practice, this site parsing has a number of problems:
For simplicity, all these moments are omitted.
For the purpose of the curriculum, the built-in HTTP request router is enough for us. In more complex systems, you probably want to use some other one. Personally, I use a router from the Gorilla project, but in general there are plenty of them.
Subsequently, we will add a point to return the files of our web application. Looking ahead, I’ll say that for this you just need to use the return of file content.
So let's write our server:
I will not give the storage implementation code in memory here, everything is available on GitHub .
To test our API:
inflation
quotes
. This completes the first part of the article. In the second part we will write tests and performance measurements for our packages. In the third part we will develop a web application.
First, a few words about what this application is and what it is for. Some time ago I faced the question of how to save some amount of money, which I had formed. Even to me, a person far from the world of finance, it was obvious that keeping money in cash is bad for at least two reasons:
- Money eats inflation (inflation risk)
- The ruble may depreciate (exchange rate risk)
It was decided to study the issue and choose the appropriate investment tool. The main criteria were the reliability and protection of savings from the above risks.
I studied the question and as a result I came to the conclusion that the only adequate investment instrument for a resident of Russia are shares of exchange-traded funds (ETF), and specifically those that are traded on the Moscow Exchange.
Thus, I propose to write an educational application that would show the profitability of all ETFs that are represented on the Moscow Exchange.
You can say that this yield can be viewed on the exchange website itself, and the application, even if it is a training one, should be of some use. I agree, so we will try to display some conditional real stock returns. By this conditional real return, I will understand the return adjusted for inflation in Russia.
In the first part of the article we will analyze the server part of the application. Our backend is written on Go and during development we will try to use such language features as parallel code execution, interfaces, testing, and so on.
TK requirements:
- The server part of the application must provide, upon request, data on quotes of all ETFs of the Moscow Exchange and data on inflation for all months of trading for each paper
- The server part of the application must support multiple data storage providers; switching between providers should not require code changes
- The server part of the application must provide the API via the http protocol to retrieve data from the storage.
So, let's design the software architecture of the server part of our system.
First , let's figure out the structure of the application packages. According to the statement of work, the application will consist of a web server that will provide the REST API and send files to our web application (later we will write the SPA on Vue). In addition, according to the TOR, we must make several packages for the data warehouse suppliers.
At this point, you should stay in more detail. How can I provide the ability to switch between providers of some functionality in Go? Answer: using interfaces. Thus, we will have to develop an interface (contract) for packages, each of which will fulfill a contract for its own type of storage. The article will consider the storage of data in RAM, but by analogy you can easily add any DBMS. The final package structure will be as follows:
Secondly , let's define the types of data in which we will store the received information and the contract for the storage providers.
We will need data types for stock quotes and inflation. We will take quotes and inflation by months, this scale is quite suitable for such a non-speculative instrument as ETF.
The contract will require methods to populate the repository with data from the Mosbirzh server (initialization) and provide quotes data on request. Everything is very simple.
As a result, in the storage module we put types for storing quotes and an interface:
// Package storage описывает общие требования к поставщику хранилища и используемые типы данныхpackage storage
// Security - ценная бумагаtype Security struct {
ID string// ticker
Name string// полное имя бумаги
IssueDate int64// дата выпуска в обращение
Quotes []Quote // котировки
}
// Quote - котировка ценной бумаги (цена 'close')type Quote struct {
SecurityID string// ticker
Num int// номер измерения (номер месяца)
TimeStamp int64// отметка времени в формате Unix Time
Price float64// цена закрытия
}
// Interface - контракт для драйвера хранилища котировокtype Interface interface {
InitData() error // инициализирует хранилище данными с сервера Мосбиржи
Securities() ([]Security, error) // получить список бумаг с котировками
}
Inflation data for simplicity is encoded in the server module:
var inflation = []struct {
Year int
Values [12]float64
}{
{
Year: 2013,
Values: [12]float64{0.97, 0.56, 0.34, 0.51, 0.66, 0.42, 0.82, 0.14, 0.21, 0.57, 0.56, 0.51},
},
{
Year: 2014,
Values: [12]float64{0.59, 0.70, 1.02, 0.90, 0.90, 0.62, 0.49, 0.24, 0.65, 0.82, 1.28, 2.62},
},
{
Year: 2015,
Values: [12]float64{3.85, 2.22, 1.21, 0.46, 0.35, 0.19, 0.80, 0.35, 0.57, 0.74, 0.75, 0.77},
},
{
Year: 2016,
Values: [12]float64{0.96, 0.63, 0.46, 0.44, 0.41, 0.36, 0.54, 0.01, 0.17, 0.43, 0.44, 0.40},
},
{
Year: 2017,
Values: [12]float64{0.62, 0.22, 0.13, 0.33, 0.37, 0.61, 0.07, -0.54, -0.15, 0.20, 0.22, 0.42},
},
{
Year: 2018,
Values: [12]float64{0.31, 0.21, 0.29, 0.38, 0.38, 0.49, 0.27, 0.01, 0.16, 0.35, 0.50, 0.84},
},
}Third , let's describe the end points of our API. There will be only two: for quotes and inflation. Only HTTP GET method.
// API нашего сервера
http.HandleFunc("/api/v1/securities", securitiesHandler) // список бумаг с котировками
http.HandleFunc("/api/v1/inflation", inflationHandler) // инфляция по месяцамActually receiving and processing data from the Mosbirzhi site is carried out in the initialization method. Data is taken according to the Exchange API reference .
What you should pay attention to: we have to use a separate request for each security (and there are already a couple of dozen of them). Execution of data initialization sequentially, in one stream, would take a lot of time. Therefore, we will use the pride of Go - gorutiny. Notice the following piece of code:
// InitData инициализирует хранилище данными с сервера Мосбиржиfunc(s *Storage)InitData()(err error) {
securities, err := getSecurities()
if err != nil {
return err
}
// объект синхронизации горутинvar wg sync.WaitGroup
// увеличиваем счётчик горутин по количеству ценных бумаг
wg.Add(len(securities))
for _, security := range securities {
gofunc(item storage.Security) {
// уменьшаем счётчик перед завершением функцииdefer wg.Done()
var quotes []storage.Quote
quotes, err = getSecurityQuotes(item)
if err != nil {
fmt.Println(item, err)
return
}
item.Quotes = quotes
err = s.Add(item)
if err != nil {
return
}
}(security)
}
// ожидаем выполнения всех горутин
wg.Wait()
return err
}
In the data initialization function, we parallelize server requests. In practice, this site parsing has a number of problems:
- May cause automatic blocking of requests due to DoS suspicion.
- You need to use the context module or the control channel to force termination of the gorutin
- Need to use the channel to return errors from the gorutiny
For simplicity, all these moments are omitted.
For the purpose of the curriculum, the built-in HTTP request router is enough for us. In more complex systems, you probably want to use some other one. Personally, I use a router from the Gorilla project, but in general there are plenty of them.
Subsequently, we will add a point to return the files of our web application. Looking ahead, I’ll say that for this you just need to use the return of file content.
So let's write our server:
// Package main реализует веб-сервер проетка moex-etfpackage main
import (
"encoding/json""fmt""log""moex_etf/server/storage""moex_etf/server/storage/inmemory""net/http"
)
var db storage.Interface
funcmain() {
// здесь мы можем, например, добавить проверку флагов запуска или переменной окружения// для выбора поставщика хранилища. выбрали память
db = inmemory.New()
fmt.Println("Inititalizing data")
// инициализация данных хранилища
err := db.InitData()
if err != nil {
log.Fatal(err)
}
// API нашего сервера
http.HandleFunc("/api/v1/securities", securitiesHandler) // список бумаг с котировками
http.HandleFunc("/api/v1/inflation", inflationHandler) // инфляция по месяцам// запускаем веб сервер на порту 8080const addr = ":8080"
fmt.Println("Starting web server at", addr)
log.Fatal(http.ListenAndServe(addr, nil))
}
// обработчик запроса котировокfuncsecuritiesHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "GET")
w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
if r.Method != http.MethodGet {
return
}
securities, err := db.Securities()
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
w.Write([]byte(err.Error()))
}
err = json.NewEncoder(w).Encode(securities)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
w.Write([]byte(err.Error()))
}
}
// обработчик запроса инфляцииfuncinflationHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "GET")
w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
if r.Method != http.MethodGet {
return
}
err := json.NewEncoder(w).Encode(inflation)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
w.Write([]byte(err.Error()))
}
}
// инфляция в России по месяцамvar inflation = []struct {
Year int
Values [12]float64
}{
{
Year: 2013,
Values: [12]float64{0.97, 0.56, 0.34, 0.51, 0.66, 0.42, 0.82, 0.14, 0.21, 0.57, 0.56, 0.51},
},
{
Year: 2014,
Values: [12]float64{0.59, 0.70, 1.02, 0.90, 0.90, 0.62, 0.49, 0.24, 0.65, 0.82, 1.28, 2.62},
},
{
Year: 2015,
Values: [12]float64{3.85, 2.22, 1.21, 0.46, 0.35, 0.19, 0.80, 0.35, 0.57, 0.74, 0.75, 0.77},
},
{
Year: 2016,
Values: [12]float64{0.96, 0.63, 0.46, 0.44, 0.41, 0.36, 0.54, 0.01, 0.17, 0.43, 0.44, 0.40},
},
{
Year: 2017,
Values: [12]float64{0.62, 0.22, 0.13, 0.33, 0.37, 0.61, 0.07, -0.54, -0.15, 0.20, 0.22, 0.42},
},
{
Year: 2018,
Values: [12]float64{0.31, 0.21, 0.29, 0.38, 0.38, 0.49, 0.27, 0.01, 0.16, 0.35, 0.50, 0.84},
},
}
I will not give the storage implementation code in memory here, everything is available on GitHub .
To test our API:
inflation
quotes
. This completes the first part of the article. In the second part we will write tests and performance measurements for our packages. In the third part we will develop a web application.