April 12, 2019 at 08:23We made a news aggregator with a focus on cryptocurrencies - intwt.com
Hi Habr!
The cryptocurrency market is growing every day, and with it the amount of information is growing.
Therefore, we decided to launch the intwt.com project - this is an aggregator of news and posts from social networks for traders and everyone who is interested in cryptocurrencies.
The simple, convenient and intuitive interface of the service is designed to make it a really effective tool for monitoring important information.
At the moment, we analyze more than 3 thousand news sources in English, Russian and Chinese, as a result we receive about 3 thousand new materials daily.
Each material is analyzed by the system for references to cryptocurrencies and popularity in social networks.
Using the news filter, you can configure your individual feed, save it in your account, and if you want to enable broadcasting in your telegram channel.
We constantly monitor important indicators for 2716 cryptocurrencies and monitor the emergence of new currencies on the market.
Using a special interface to view the list of cryptocurrencies, you can see the leaders of growth and fall in the market.
For each currency, you can see the latest news and all indicators on a separate page, for example, such as Price, Capitalization, etc., as well as a price chart for the entire currency existence period in the market.
In your personal account, you can create a cryptocurrency portfolio and track its dynamics on the chart.
We don’t think about monetization at the moment, because the service is very young and is gaining an audience, but most likely it will be advertising and paid access by subscription to PRO functions.
Service can be divided into two parts
All services work inside Docker, currently 19 containers have been launched. All this is deployed through GitLab CI. To monitor the system we use Prometheus and Grafana, and for the error log Sentry.
Development of a mobile application for ios and android, creation of a platform for specialists with the ability to post copyright articles, videos and reviews on cryptocurrencies. Subscription to the author. And of course, an automated regression analysis of the impact of news on the movement of currency prices.
We will be glad to hear criticism or ideas for the development of the project.
PS the real author of the post Dmitry , all the questions for him.
The cryptocurrency market is growing every day, and with it the amount of information is growing.
Therefore, we decided to launch the intwt.com project - this is an aggregator of news and posts from social networks for traders and everyone who is interested in cryptocurrencies.
The simple, convenient and intuitive interface of the service is designed to make it a really effective tool for monitoring important information.
At the moment, we analyze more than 3 thousand news sources in English, Russian and Chinese, as a result we receive about 3 thousand new materials daily.
Each material is analyzed by the system for references to cryptocurrencies and popularity in social networks.
Using the news filter, you can configure your individual feed, save it in your account, and if you want to enable broadcasting in your telegram channel.
We constantly monitor important indicators for 2716 cryptocurrencies and monitor the emergence of new currencies on the market.
Using a special interface to view the list of cryptocurrencies, you can see the leaders of growth and fall in the market.
For each currency, you can see the latest news and all indicators on a separate page, for example, such as Price, Capitalization, etc., as well as a price chart for the entire currency existence period in the market.
In your personal account, you can create a cryptocurrency portfolio and track its dynamics on the chart.
We don’t think about monetization at the moment, because the service is very young and is gaining an audience, but most likely it will be advertising and paid access by subscription to PRO functions.
Some technical details
Service can be divided into two parts
- The frontend is a SPA application written in Vue and a backend written in Go, which distributes minimal HTML with content for search engines and code for launching a SPA application. This approach allows you to avoid server rendering and make friends with search engines. Although Yandex immediately blocked us, like a doorway.
- The parser is dedicated to a separate service with its own base and admin panel so that it can be easily transferred to a separate server. Here we used Go, PostgreSQL, Beanstalkd for queuing parsing and Rotating TOR proxy which avoids IP blocking. To parse some sites, you have to use browserless chrome to bypass defense mechanisms. The admin for the parser is made on Laravel.
All services work inside Docker, currently 19 containers have been launched. All this is deployed through GitLab CI. To monitor the system we use Prometheus and Grafana, and for the error log Sentry.
What is planned next?
Development of a mobile application for ios and android, creation of a platform for specialists with the ability to post copyright articles, videos and reviews on cryptocurrencies. Subscription to the author. And of course, an automated regression analysis of the impact of news on the movement of currency prices.
We will be glad to hear criticism or ideas for the development of the project.
PS the real author of the post Dmitry , all the questions for him.