"Rubber host look for Alibaba cloud": we place Elasticsearch on Alibaba Cloud facilities

Some time ago, I talked about our experience in solving complex problems using Elasticsearch. It was a story about sausage , where we developed a search on the basis of at least 50,000 documents, which allows you to search for ingredients in the composition of the products and automatically form a description of the products.

Recently, an interesting project came to our company again, requiring the use of Elasticsearch. This time, we were faced with the task of deploying ES to work in an application on the Chinese cloud platform Alibaba Cloud. Here, in addition to technical problems, we encountered such an unexpected thing as the Chinese mentality.

This article focuses solely on our personal experience with Alibaba Cloud, and not on the interface or standard options. I will tell you what we managed to figure out in communication with representatives of the Elasticsearch and Alibaba Cloud platforms, and how we used this to solve our problem.

Swipe to China

At the beginning of the project, the customer said that the first launch of the application will be in China. This greatly complicated the task: the Chinese legislation gives a lot of inconvenience to developers and site owners. Just getting a domain because of the local bureaucracy is stretched for three months. And what if you host the application outside of China? - you ask. And in response, I wish you good luck with launching in the Chinese zone, because there such tricks are practically outlawed.

We were given access to the client server on Alibaba Cloud and asked to deploy docker containers with the application on it. This was the first, but not the main problem. We forwarded access from the container to the file system in order to have access to project files at the server level, and here the problem was that on the server itself, there was nothing except the docker: neither git, nor nginx, nor php. Everything was done at the container level, and the server was only a data warehouse. Therefore, a simple git pull turned into the following:

docker exec -it b33aee747c5e git pull

Because of this, we had to write a bash script for each container, for example, with the name git:

docker exec -it b33aee747c5e git $@

Further the adventure only became more complicated: we needed to somehow install ES on this. Through the Alibaba Cloud management console, we put the image of Elasticsearch 6.0.0 and ...
... And after a while, we started catching bugs. Then the access control system X-Pack will start “swearing” that after 28 days our license will expire, then the RAM will run out if some 500 documents are indexed. Not "soared."

Laovaev problems

All this time we thought that the reason for the failure was in the X-Pack, that it was he who did not let us work normally.

“Well, what to do,” said the customer, after listening to our version. - So, we will buy a license X-Pack. Write in Elasticsearch, maybe they themselves will advise something. ” I wrote to the account service, and the consultant from the Dutch office Driton Khalili answered me (if you happen to work with him, say hello from me, he is cool). Yes, Eastern Europe and Russia are in charge of their Dutch office, where Turks work.

A consultant told me about the X-Pack license payment system and asked where we are hosted.
In China, I replied.

“This is a problem,” he grieved. - Your customer should write to our Chinese department himself so that he can be helped with this problem.
- Are there any other options? - I asked.
- Try contacting Alibaba Cloud. The fact is that we recently entered into a partnership agreement with them, and now they provide Elasticsearch as a service.
- So we host them, - I was surprised.
“Then why are you bothering with the container instead of taking it from them directly ?!”
Having finished the conversation, I retold all this to the customer.
- I mean, does Alibaba have Elasticsearch as a Service ?! - there was no limit to the client’s indignation. After that, he went to talk with "Chinese colleagues", and the next day he told the following:
- These people said that I was "Laoway", and on my "Laoway" account I was not supposed to have Elasticsearch.
Laovai, as you might guess, the Chinese call foreigners, often using this word with a disparaging shade. You can say this is the Chinese "in large numbers."
- They drank Puer there, or what?
- I don’t know about Pu-erh, but after my inspiring speech that they are somewhat wrong and for which I pay money to them in general, I was given a “secret” link to Elasticsearch, which will now be displayed in our cloud console. Customize and will move.

What to do to "soared"

Three weeks later, a link to Alibaba Cloud Elasticsearch appeared in the management console menu (in the DTplus section, at the very bottom).



From this we can conclude that if the Chinese still give you a link, access will have to wait some time (I can’t say exactly about the price, in my opinion, this did not increase the cost of the subscription).

And again: you, of course, will be given access to the settings, a normal URL, Kibana for monitoring. Only now when you try to reach ES from somewhere else, you will get 505.

How did I solve this problem?

By default, the Alibaba Cloud Console Elasticsearch is only accessible from Kibana and proxies calls from it to 127.0.0.1 inside the server where the ES itself is located. Then I wondered: what are the IP-addresses inside between the containers? I entered the settings and saw that all of them are spinning in a fairly familiar 192.168.0. * Subnet. I was not completely sure, so I wrote myself a few more addresses, in case I didn’t “fly up”.

I also thought about what would happen if I myself would need to send requests from my workplace to ES, so I also wrote down my IP address on a piece of paper.

Next you need to do the following.

Go to the Alibaba Cloud Console, open Cluster -> Manage -> Network and Snapshots -> section Cluster Network Settings -> Public IP Address Whitelist -> Update. And we save here all the addresses that will be useful - both personal and public. After that, you can already work and knock on ES and from the containers with the application, and from other places.



The move took less than two hours. True, I still had paranoia for another week on the subject that everything might fall, but so far nothing has fallen. For three months, as never fell.

In total, we spent about three weeks trying to solve a problem that does not exist. It’s just that the Chinese are very cunning and do not want to give access to their services to all kinds of "laow". For a person with our or Western mentality it is strange, however, we coped with this problem,

Thanks for attention!