January 5, 2010 at 21:28How to make a budget geo-cluster hosting
As a distributed hosting, let's take a classic example - creating a blog service based on mu-Wordpress.
The task is to assemble a fault-tolerant (as far as possible) geo-distributed system with a limited budget. Accordingly, all equipment is rented at various Data Centers.
And here it should be said that not all Data Centers are equally useful. The high-quality ones rent a server for $ 800, while the low-quality ones can be rented for about the same server for $ 100. And it is these features that must be taken into account when creating a geocluster.
Now about small hacks. By default in mu-Wordpress, the function of uploading downloadable content was made extremely unsuccessfully - through PHP. Accordingly, it was replaced by downloading a separate service and inserting downloadable content with a direct link to the statics.
The second hack was a modification of the cache control. In addition to the instructions to cache static design elements, another hack was introduced that forbade caching the record for the duration of its discussion (by default - 14 days), and after that it was given with a heading allowing caching. In addition, RSS feeds were cunningly cached.
The final hack was the database synchronization system - each INSERT / DELETE / UPDATE was executed on a "neighbor". It turned out such a software raid in the context of MySQL + PHP.
First about the DNS. Since mu-Wordpress used a subdomain for each blog, the most reasonable solution was to use the Slave DNS service from two independent registrars - gray clouds. It is inexpensive and quite reliable.
The high-quality Green Data Center, where two servers are rented, was used for primary DNS, the initial registration service and the upload form.
Two medium-quality Data Centers were used as the main “backbone” of the geoclast. Each server had its own neighbor and they synchronized among themselves all the database information + files.
Thus, it was possible to significantly reduce costs at this stage. However, after a while, a serious problem occurred, the name of which is search bots.
These bots created 200-300 simultaneous connections on each server, which of course did not lead to anything good - timeouts and 50x errors began. Of course, it would be possible to reduce the number of requests via robots.txt with the crawl-delay option, but ... who wants to have his blog indexed slowly?
And then cheap yellow Data Centers + monitoring and DNS settings on two servers in the green Data Center helped. How it all worked:
Two yellow Data Centers functioned as squid primary (parent) proxies. The remaining three used them as parents, which reduced the load on the blue "backbone".
Monitoring on servers in the green Data Center monitored the availability of yellow and blue, performing a DNS modification in the event of a failure.
Now let's talk about fault tolerance. What did we lose when we turned off the green, blue or yellow segments?
Thus, it was possible to achieve maximum availability + saving the blog in the proxy cache even when it is physically deleted. There was such a thing - one user accidentally deleted his blog - and such an opportunity is provided. Turning in time to support his records, they were pulled out of the cache, parsed and poured into the newly created blog.
PS Regarding server optimization issues for mu-WordPress, you can write me a comment - quite a lot of dogs were eaten under this case;)
The task is to assemble a fault-tolerant (as far as possible) geo-distributed system with a limited budget. Accordingly, all equipment is rented at various Data Centers.
And here it should be said that not all Data Centers are equally useful. The high-quality ones rent a server for $ 800, while the low-quality ones can be rented for about the same server for $ 100. And it is these features that must be taken into account when creating a geocluster.
Now about small hacks. By default in mu-Wordpress, the function of uploading downloadable content was made extremely unsuccessfully - through PHP. Accordingly, it was replaced by downloading a separate service and inserting downloadable content with a direct link to the statics.
The second hack was a modification of the cache control. In addition to the instructions to cache static design elements, another hack was introduced that forbade caching the record for the duration of its discussion (by default - 14 days), and after that it was given with a heading allowing caching. In addition, RSS feeds were cunningly cached.
The final hack was the database synchronization system - each INSERT / DELETE / UPDATE was executed on a "neighbor". It turned out such a software raid in the context of MySQL + PHP.
First about the DNS. Since mu-Wordpress used a subdomain for each blog, the most reasonable solution was to use the Slave DNS service from two independent registrars - gray clouds. It is inexpensive and quite reliable.
The high-quality Green Data Center, where two servers are rented, was used for primary DNS, the initial registration service and the upload form.
Two medium-quality Data Centers were used as the main “backbone” of the geoclast. Each server had its own neighbor and they synchronized among themselves all the database information + files.
Thus, it was possible to significantly reduce costs at this stage. However, after a while, a serious problem occurred, the name of which is search bots.
These bots created 200-300 simultaneous connections on each server, which of course did not lead to anything good - timeouts and 50x errors began. Of course, it would be possible to reduce the number of requests via robots.txt with the crawl-delay option, but ... who wants to have his blog indexed slowly?
And then cheap yellow Data Centers + monitoring and DNS settings on two servers in the green Data Center helped. How it all worked:
Two yellow Data Centers functioned as squid primary (parent) proxies. The remaining three used them as parents, which reduced the load on the blue "backbone".
Monitoring on servers in the green Data Center monitored the availability of yellow and blue, performing a DNS modification in the event of a failure.
Now let's talk about fault tolerance. What did we lose when we turned off the green, blue or yellow segments?
- Green - register and download files
- Blue - in the case of one nothing, in the case of all - the admin did not work + the last entries outside the cache were not displayed.
- Yellow - in the case of one nothing, in the case of all - just an increased load on the blue servers
Thus, it was possible to achieve maximum availability + saving the blog in the proxy cache even when it is physically deleted. There was such a thing - one user accidentally deleted his blog - and such an opportunity is provided. Turning in time to support his records, they were pulled out of the cache, parsed and poured into the newly created blog.
PS Regarding server optimization issues for mu-WordPress, you can write me a comment - quite a lot of dogs were eaten under this case;)