June 10, 2017 at 20:35It was in the evening. A load-balanced, scalable web service using the example of Bitrix on the Google Cloud Platform
It’s worth mentioning right away that this article was written more likely not to show the possibility of this product working on the Google Cloud Platform (GCP), it would work on it without it. Bitrix was taken for experiments simply as a popular platform. He himself knows how to build pools, nodes and more in his “web environment”, though with his own rake. And that is why even Debian machines for tests were taken, and not the beloved CentOS.
In fact, the material is applicable to many web projects. More precisely, this is a simple guide to building fault-tolerant and distributed applications based on Google Compute Engine virtual machines, Google Cloud SQL databases and Google load balancer.
Dear Bitrix experts and professionals, there are a lot of options for implementing this solution, only one is given here. You can consider virtual machines, containers, and the Google App Engine as a cloud platform. Plus, there will be the ability to connect Google Storage, which has been included in the engine for quite some time. In general, I will be happy to discuss the possibility of a joint Bitrix pilot on GCP, maybe your experience will be described next time as applicable specifically to Bitrix.
What is Bitrix, we will not explain for a long time. This is a professional web project management system and a huge number of corporate sites, online stores, portals and communities work on it.
Despite the amount of text with pictures, my entire deployment procedure took about two hours, taking into account the caught rake.
Menu → Cloud SQL → Create Instance
Choose MySQL, “Next”, “Create a second generation database”.
We indicate the database name, location, zone, machine size, disk type, storage size, backup schedule, root password, timeout for those. Service, add the network. For the test, I chose a 1vCPU machine, 1.7Gb RAM and 200GB regular HDD storage, then we can change it. When creating the base, we pay attention to the zone, the virtual machine will also be located in it.
The allowed network for tests can be set to 0.0.0.0/0 (all) and connected directly. But we will do the right thing and do it humanly, we won’t indicate anything here, security is more important.
Next, we need to set up a service account. Immediately make a reservation for what it is needed for. If you connect to the database directly by its IP address, you can skip this step. We need an account for secure connection through a proxy. Otherwise, the machine will be created with a default entry that does not have access to other services.
So, open the menu, go to IAM and administration → Service accounts → Create a service account . We indicate the name and ID according to our taste, select the Cloud SQL role → Cloud SQL Client . OK, done.
With the base sorted out.
Click on the “sandwich” (Menu) → Compute Engine → VM Instances → Create Instance .
Take the image of Debian 8 or whatever, to taste. We take a simple machine to start: 1 core, 3.5GB RAM.
We change the machine’s service account to the one that we created in the previous step (for communication with our Cloud SQL. We put the “allow HTTP” and “enable HTTPS” checkboxes depending on the planned protocol.
Next, we need to slightly automate our machine, or rather run it on it SQL proxy. For this, we open the "Configure management, disk, network and SSH-keys" parameters and in the Automation section → Startup script write:
Why? In fact, only the last line is needed to start, everything else is enough to be done once when setting up the machine. This is an example so as not to break apart the entire article.
Next is an important point. We go to the "Disks" tab and uncheck "Delete boot disk when deleting an instance." Yes, we will delete our machine after setting it up.
With the creation of the machine finished, click "create" and wait.
Upon completion, we connect via SSH to our virtual machine and configure it. I won’t write much about setting up the machine for Bitrix, there are a lot of materials both on Habré and on the Internet. There is even a ready-made script “Web environment”, but I did not use it for religious reasons. It was easier for me to run everything with my hands than to understand someone else's sandwich. I will describe only what I did to launch. And without any optimization and tuning, otherwise they can be overwhelmed with comments by the optimization gurus of PHP, Zend, NGINX and other things, no offense.
First, update:
MySQL client:
Apache:
PHP:
Restart Apache:
Next, we will need a Cloud SQL proxy for a direct-armed connection to the database. We wrote the script to launch it when creating the machine; it works every time it starts. Check if it works:
Enter the root password from the database, enjoy the line:
If the first time does not work, then check the start script of the machine and reboot it.
The usual wget is downloaded directly from the site:
The link is provided as an example to the start version. Next, unpack the files into the web server directory, I left it by default.
We’ve finished the machine, go to the list of virtual machines on Google Cloud and open our machine via HTTP (or HTTPS, if configured):
during the installation, specify the MySQL server 127.0.0.1, login and root password from our Cloud SQL database, and then go to the path we need in the wizard until the installation is complete. When everything is ready, we can check the configuration performance. What I did:
I, of course, drew attention to the performance of MySQL and was surprised at first. But, as it turned out later, this is quite adequate data, the database in reality gives out normal performance, it’s just that we still have a cloud.
Everything works, let's move on to the next stage.
I will do it in one region, in Europe. But to understand the service, it is still possible to spread it by region (for example, if we have partners in Asia). The principle of operation is similar, only it will be necessary to create two groups in two regions and a replica of the base there.
First you need to create a disk image. We delete our virtual machine. Yes, yes, just delete. It is important to remember here whether we removed the checkbox "Delete boot disk when deleting an instance" during its creation. If you forget, then clone (when you open the machine the button on top) and remove it in the parameters. After removing the machine, we will have access to its disk to create the image.
We go to the Compute Engine → Images section , click “create image”. When creating, there is nothing tricky, we indicate our free disk and everything is ready.
Next, we create a template for our future machines with Bitrix.
Go to Compute Engine -> Instance Templates . We create a template by analogy with our virtual machine, but from our image.
We indicate:
I repeat, this script was included when creating the machine, now only the last line is enough. The template is customized. Remember that
Create an instance group:
Open Compute Engine → Instance groups → Create instance group .
We indicate:
Created.
Click on the “sandwich” on the top left, Network → Load Balance → Create a load balancer .
HTTP / HTTPS load balancing -> Start configuration .
We indicate:
Done, create. It starts to work almost immediately, you can not wait.
After creating the balancer, open the “interface VMs” tab and see its external address. According to it, our Bitrix will work in the Google Cloud Platform. We go to the address, enjoy. We can write this address in DNS.
First, what we have:
I configured the start of additional machine instances with 100 simultaneous connections to show how it works. For a combat solution, you can use your own parameters. Testing was carried out in the GCP trial period with a limit on the number of cores for all machines of only eight pieces, because the eighth core was taken by the machine with which the test was directly launched.
First, we got such a test schedule:
It became clear that the machines did not have time to start. Therefore, their minimum number was increased to two, the number of connections per machine was reduced to 50, and the maximum load was reduced to 80%. The result was:
As a result, everything took off, automatic scaling worked as it should.
Of course, a lot more can be added to our decision. For example, tuning the server for Bitrix itself, copying and replicating the database, failover, etc. This topic is already for other materials. This article is probably written not to show the possibility of working with Bitrix and other web and mobile projects on Google Cloud. This is a simple guide to building fault-tolerant and distributed applications based on virtual machines using the Google load balancer.
To organize a distributed application, we just need to replicate the database to another region, create another group of virtual machines in another region and include the machines in our global balancer. Google itself will choose the best route and direct users to the application in the nearest location.
A little later, another article will be published on how to run Bitrix on the Google App Engine. I think that it will be more interesting.
Lastly, we touch on the cost of services:
Prices are naturally approximate, for a specific combat solution, a powerful database, distribution by region, etc. may be required. Everything is designed for location in the European data center without frills.
In addition to a modest price list for impressive resources, it is also encouraging that the Google Cloud Platform has finally become available in Russia for non-cash payments to organizations.
In fact, the material is applicable to many web projects. More precisely, this is a simple guide to building fault-tolerant and distributed applications based on Google Compute Engine virtual machines, Google Cloud SQL databases and Google load balancer.
Dear Bitrix experts and professionals, there are a lot of options for implementing this solution, only one is given here. You can consider virtual machines, containers, and the Google App Engine as a cloud platform. Plus, there will be the ability to connect Google Storage, which has been included in the engine for quite some time. In general, I will be happy to discuss the possibility of a joint Bitrix pilot on GCP, maybe your experience will be described next time as applicable specifically to Bitrix.
Let's get started
What is Bitrix, we will not explain for a long time. This is a professional web project management system and a huge number of corporate sites, online stores, portals and communities work on it.
Despite the amount of text with pictures, my entire deployment procedure took about two hours, taking into account the caught rake.
Let's start by creating a database
Menu → Cloud SQL → Create Instance
Choose MySQL, “Next”, “Create a second generation database”.
We indicate the database name, location, zone, machine size, disk type, storage size, backup schedule, root password, timeout for those. Service, add the network. For the test, I chose a 1vCPU machine, 1.7Gb RAM and 200GB regular HDD storage, then we can change it. When creating the base, we pay attention to the zone, the virtual machine will also be located in it.
The allowed network for tests can be set to 0.0.0.0/0 (all) and connected directly. But we will do the right thing and do it humanly, we won’t indicate anything here, security is more important.
Next, we need to set up a service account. Immediately make a reservation for what it is needed for. If you connect to the database directly by its IP address, you can skip this step. We need an account for secure connection through a proxy. Otherwise, the machine will be created with a default entry that does not have access to other services.
So, open the menu, go to IAM and administration → Service accounts → Create a service account . We indicate the name and ID according to our taste, select the Cloud SQL role → Cloud SQL Client . OK, done.
With the base sorted out.
Create a virtual machine
Click on the “sandwich” (Menu) → Compute Engine → VM Instances → Create Instance .
Take the image of Debian 8 or whatever, to taste. We take a simple machine to start: 1 core, 3.5GB RAM.
We change the machine’s service account to the one that we created in the previous step (for communication with our Cloud SQL. We put the “allow HTTP” and “enable HTTPS” checkboxes depending on the planned protocol.
Next, we need to slightly automate our machine, or rather run it on it SQL proxy. For this, we open the "Configure management, disk, network and SSH-keys" parameters and in the Automation section → Startup script write:
sudo wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64
sudo mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy
sudo chmod +x cloud_sql_proxy
sudo ./cloud_sql_proxy -instances==tcp:3306 & Why? In fact, only the last line is needed to start, everything else is enough to be done once when setting up the machine. This is an example so as not to break apart the entire article.
Next is an important point. We go to the "Disks" tab and uncheck "Delete boot disk when deleting an instance." Yes, we will delete our machine after setting it up.
With the creation of the machine finished, click "create" and wait.
Upon completion, we connect via SSH to our virtual machine and configure it. I won’t write much about setting up the machine for Bitrix, there are a lot of materials both on Habré and on the Internet. There is even a ready-made script “Web environment”, but I did not use it for religious reasons. It was easier for me to run everything with my hands than to understand someone else's sandwich. I will describe only what I did to launch. And without any optimization and tuning, otherwise they can be overwhelmed with comments by the optimization gurus of PHP, Zend, NGINX and other things, no offense.
First, update:
sudo apt-get update
sudo apt-get upgradeMySQL client:
sudo apt-get install mysql-clientApache:
sudo apt-get install apache2PHP:
sudo apt-get install php5 libapache2-mod-php5 php5-mysqlRestart Apache:
sudo systemctl restart apache2Next, we will need a Cloud SQL proxy for a direct-armed connection to the database. We wrote the script to launch it when creating the machine; it works every time it starts. Check if it works:
mysql -u root -p --host 127.0.0.1Enter the root password from the database, enjoy the line:
mysql>If the first time does not work, then check the start script of the machine and reboot it.
Expand Bitrix
The usual wget is downloaded directly from the site:
Wget https://www.1c-bitrix.ru/download/start_encode_php5.tar.gzThe link is provided as an example to the start version. Next, unpack the files into the web server directory, I left it by default.
We’ve finished the machine, go to the list of virtual machines on Google Cloud and open our machine via HTTP (or HTTPS, if configured):
during the installation, specify the MySQL server 127.0.0.1, login and root password from our Cloud SQL database, and then go to the path we need in the wizard until the installation is complete. When everything is ready, we can check the configuration performance. What I did:
I, of course, drew attention to the performance of MySQL and was surprised at first. But, as it turned out later, this is quite adequate data, the database in reality gives out normal performance, it’s just that we still have a cloud.
Everything works, let's move on to the next stage.
Create a scalable group of machines and put it on a load balancer
I will do it in one region, in Europe. But to understand the service, it is still possible to spread it by region (for example, if we have partners in Asia). The principle of operation is similar, only it will be necessary to create two groups in two regions and a replica of the base there.
First you need to create a disk image. We delete our virtual machine. Yes, yes, just delete. It is important to remember here whether we removed the checkbox "Delete boot disk when deleting an instance" during its creation. If you forget, then clone (when you open the machine the button on top) and remove it in the parameters. After removing the machine, we will have access to its disk to create the image.
We go to the Compute Engine → Images section , click “create image”. When creating, there is nothing tricky, we indicate our free disk and everything is ready.
Next, we create a template for our future machines with Bitrix.
Go to Compute Engine -> Instance Templates . We create a template by analogy with our virtual machine, but from our image.
We indicate:
- Name
- Size (we need the size of one VM, then they will "multiply")
- Image (in the section "Custom images" indicate the previously created)
- Service account (created by us earlier with the rights of "Cloud SQL Client")
- We allow traffic using the configured HTTP / HTTPS protocols
- We decompose “Configuring control parameters, disk and SSH keys” and write a startup script:
sudo ./cloud_sql_proxy -instances==tcp:3306 & I repeat, this script was included when creating the machine, now only the last line is enough. The template is customized. Remember that
Create an instance group:
Open Compute Engine → Instance groups → Create instance group .
We indicate:
- Name,
- Zone (I indicated the base location zone)
- The template (which we just created),
- Turn on autoscaling,
- The basis for autoscaling is the balancer
- The minimum and maximum number of copies is specified as we need
- We can create a status check. Specify the HTTP path and, if the machine does not respond, it will be nailed and another will start.
Created.
Load balancer
Click on the “sandwich” on the top left, Network → Load Balance → Create a load balancer .
HTTP / HTTPS load balancing -> Start configuration .
We indicate:
- The name, as we are comfortable
- Server VM configuration -> Server services and segments -> (expand) Server services -> Create Server service . Specify the group of instances that were created. We turn on the balancing mode we need (I turned on the frequency of requests), indicate the values of the indicators, Done. We immediately indicate the condition of the machines that we created, or create a new one. We can turn on the CDN.
- Interface VM configuration. This is something that will look into the outside world from our decision. We indicate the name, HTTP / HTTPS protocol, address type (statics of course), upload the certificate (if necessary).
Done, create. It starts to work almost immediately, you can not wait.
After creating the balancer, open the “interface VMs” tab and see its external address. According to it, our Bitrix will work in the Google Cloud Platform. We go to the address, enjoy. We can write this address in DNS.
Finally, a couple of performance and scaling tests
First, what we have:
- Cloud SQL Base - 1vCPU, 1.7Gb RAM
- A group of virtual machines with scaling from one to seven, each 1vCPU, 3,5Gb RAM (you can choose the best one for your project, I took standard machines)
- Load Balancer with one external IP address
I configured the start of additional machine instances with 100 simultaneous connections to show how it works. For a combat solution, you can use your own parameters. Testing was carried out in the GCP trial period with a limit on the number of cores for all machines of only eight pieces, because the eighth core was taken by the machine with which the test was directly launched.
First, we got such a test schedule:
It became clear that the machines did not have time to start. Therefore, their minimum number was increased to two, the number of connections per machine was reduced to 50, and the maximum load was reduced to 80%. The result was:
As a result, everything took off, automatic scaling worked as it should.
Of course, a lot more can be added to our decision. For example, tuning the server for Bitrix itself, copying and replicating the database, failover, etc. This topic is already for other materials. This article is probably written not to show the possibility of working with Bitrix and other web and mobile projects on Google Cloud. This is a simple guide to building fault-tolerant and distributed applications based on virtual machines using the Google load balancer.
To organize a distributed application, we just need to replicate the database to another region, create another group of virtual machines in another region and include the machines in our global balancer. Google itself will choose the best route and direct users to the application in the nearest location.
A little later, another article will be published on how to run Bitrix on the Google App Engine. I think that it will be more interesting.
Lastly, we touch on the cost of services:
- In the minimum configuration - 2 VM web servers (1 vCore, 3,5 Gb RAM), 1 MySQL VM (1 vCore, 1,7 Gb RAM), load Balancer) ~ 100 $ / month
- In the maximum configuration (7 VM web servers, 1 VM MySQL, load Balancer) ~ 230 $ / month
Prices are naturally approximate, for a specific combat solution, a powerful database, distribution by region, etc. may be required. Everything is designed for location in the European data center without frills.
In addition to a modest price list for impressive resources, it is also encouraging that the Google Cloud Platform has finally become available in Russia for non-cash payments to organizations.