Configuring a dedicated server for the site running HostCMS
In the “life” of almost any web project — whether it’s a small online store or a site of a growing bar — sooner or later there comes a moment when there are not enough opportunities and resources for shared hosting, nor the means for a total reorganization of the application architecture. A few years ago, when I was still working in a small web studio, I often had to observe such a picture. In almost all such cases, the same decision was made - renting a dedicated server and transferring the project to it in the form in which it is. At that time, a lot of articles were available on the network on setting up servers with Linux on board. Moreover, almost all of them were not of the best quality and often contained such harmful advice that Mr. Oster could have applauded the authors of those materials.
“These are all things of bygone days” - that’s how I thought just recently, until my old friend asked me for help in solving a similar problem. As it turned out, the situation has not changed much since then: the required section of the documentation has not been updated, the developers themselves generally advise using shared hosting from their partners, and there has been no sensible material that takes into account the nuances of migration to a dedicated project server on HostCMS. I like the CMS itself, so I decided to fix this omission. If interested - welcome to cat.
First of all, a reservation. In this article I will not consider the issues of choosing a hosting provider - I think you can handle this yourself. Ubuntu Server 14.04 was chosen as the server OS as one of the most user-friendly. I assume that you have a minimal set of knowledge to work on Linux. Unfortunately, here you will not find the fine tuning of the PAM module for setting user limits for accessing files, etc. - If you are looking for such material, then most likely this article will be boring for you.
So, we have a dedicated server and data for accessing it via ssh. The first rule, it’s the main thing, try to avoid constant work on behalf of a privileged user. During the first session, create your own account and set a password for it. For example, like this:
We will carry out further work on behalf of the newly created user.
The nginx will be used as the HTTP server . I think that he does not need an introduction. We will install it from the repository , kindly deployed by the development team. To do this, you need to get the key that signed the installation packages:
And update the list of package sources by adding the following lines to the /etc/apt/sources.list file :
After that, we update and install nginx:
To set limits on the number of files the http server opens as a user, add the following lines to /etc/security/limits.conf :
The exact numbers should be selected based on the configuration of your server. The limits module is activated by adding the following line to /etc/pam.d/common-session :
Check that the limits are set by the following command:
HostCMS requires the following php modules to be enabled: curl, gd, xslt and, of course, mysql. Also, note that now the php5-json package is not virtual and needs to be installed separately. Among other things, connect the caching module opcode xcache. We will use PHP-FPM as the SAPI (interpreter launch mode), however, in order to be able to execute some scripts on schedule, PHP-CLI will also be installed.
Installing MySQL is quite simple. Several times, the installer will ask you for a password for the root of the database server, you can safely leave it empty - we will change it later using the mysql_secure_installation utility. When it starts, answer that you want to change the root password, delete the test database and test users, and update the permissions on the service database tables.
Read more about setting up a mysql server here . The article is well written, so I see no reason to duplicate the information here.
As a file transfer protocol, I suggest using SSH FTP (SFTP) . Firstly, it is safer than regular ftp, because the data will be transmitted in encrypted form. Secondly, you don’t have to install additional software: all we need is an ssh server, we already have it. And there are practically no minuses - all modern IDEs and data download clients are able to work with this protocol.
To determine who can connect via sftp, create an additional user group, for example, sftp :
And we activate the data transfer by adding the lines to the end of the / etc / ssh / sshd_config file :
Traditionally, files related to websites are located in the / var / www / directory . And we will not depart from this unspoken rule. Create a folder for virtual hosts and a future fast cache mount point:
Then we indicate that at the next boot, tmpfs will be mounted in this folder. Add to / etc / fstab :
It is worth noting that some editions of HostCMS have a built-in algorithm for caching responses to files. If you use one of these editions, it makes sense to mount tmpfs to the cache directory of the CMS itself.
If you plan to deploy several sites running HostCMS on your server, then the procedure described below will have to be repeated several times. In this case, it makes sense to try to automate the process of creating a new host. I advise you to pay attention to a small set of scripts described in this article and “finish” them for yourself.
For security reasons, all files associated with our site will belong to a specially wound user in the system. Connecting via sftp and executing PHP scripts will occur on its own behalf. To make it easier, you can call it by the name of your site:
For chroot to work correctly, make root the owner of this user's home directory:
The php-fpm pool will be launched on behalf of the user that we created in the previous step. To interact with the front-end server, a Unix socket will be used. In addition, you can configure the number of running processes for processing requests, the type of logging, and some other php-specific parameters for your site.
In the nginx host configuration file, you will need to specify the domain name of the site, the path for writing access logs and the address of the unix socket that php-fpm is listening to. To handle requests for non-existent files, we will use the named location - this way we will emulate the mod_rewrite operation for Apache2. Before giving the script to our backend for processing, we check its existence. This avoids the problem described here . In order to reduce the load on the site from unregistered users, we will use caching on the nginx side. To do this, create the configuration file /etc/nginx/conf.d/cache with the following contents:
And then we will connect it in the virtual host config.
Now almost everything is ready, it remains only to deploy the database and create a user on behalf of whom connection will be made to it. To do this, execute several simple commands in the mysql console:
Please note that connection to the database on behalf of this user is allowed from any address.
If you have a dump of the previously used database, then you can deploy it with the following set of commands all in the same mysql console:
If you did everything correctly, then you should already have a fully configured environment for launching your project. There are two last steps in the list, but not in significance, - setting up backup and log rotation. As a tool for creating backups, I recommend using backup-manager. The habr has an excellent article about it, so we will not dwell on it in detail.
To perform log rotation, we just need to create the correct config for the logrotate utility.
Perhaps this is all I wanted to say.
Perhaps someone will find the article not too relevant due to the dominance of hosting control panels. Although, in my opinion, they are good for mass provision of services and are completely unsuitable when it comes to honing server settings for a specific project.
Others will find it a little messy. It is possible that this is so: in the article I just tried to reflect my confused experience in the field of server administration.
In any case, I will be glad if this material helps someone. Comments and additions are welcome.
“These are all things of bygone days” - that’s how I thought just recently, until my old friend asked me for help in solving a similar problem. As it turned out, the situation has not changed much since then: the required section of the documentation has not been updated, the developers themselves generally advise using shared hosting from their partners, and there has been no sensible material that takes into account the nuances of migration to a dedicated project server on HostCMS. I like the CMS itself, so I decided to fix this omission. If interested - welcome to cat.
First of all, a reservation. In this article I will not consider the issues of choosing a hosting provider - I think you can handle this yourself. Ubuntu Server 14.04 was chosen as the server OS as one of the most user-friendly. I assume that you have a minimal set of knowledge to work on Linux. Unfortunately, here you will not find the fine tuning of the PAM module for setting user limits for accessing files, etc. - If you are looking for such material, then most likely this article will be boring for you.
First steps
So, we have a dedicated server and data for accessing it via ssh. The first rule, it’s the main thing, try to avoid constant work on behalf of a privileged user. During the first session, create your own account and set a password for it. For example, like this:
useradd user_name -s /bin/bash -U -m -G sudo
passwd user_name
We will carry out further work on behalf of the newly created user.
Installing the necessary software
nginx
The nginx will be used as the HTTP server . I think that he does not need an introduction. We will install it from the repository , kindly deployed by the development team. To do this, you need to get the key that signed the installation packages:
# иногда происходит неведомая фигня с добавлением ключей непосредственно из STDOUT
# поэтому сначала ключ сохраняем в файл и только потом добавляем
wget http://nginx.org/keys/nginx_signing.key
sudo apt-key add nginx_signing.key
rm nginx_signing.key
And update the list of package sources by adding the following lines to the /etc/apt/sources.list file :
# 12.04 = precise
# 14.04 = trusty
deb http://nginx.org/packages/ubuntu/ trusty nginx
deb-src http://nginx.org/packages/ubuntu/ trusty nginx
After that, we update and install nginx:
sudo aptitude update && sudo aptitude upgrade -y
sudo aptitude install nginx
To set limits on the number of files the http server opens as a user, add the following lines to /etc/security/limits.conf :
nginx hard nofile 32768
nginx soft nofile 32768
The exact numbers should be selected based on the configuration of your server. The limits module is activated by adding the following line to /etc/pam.d/common-session :
session required pam_limits.so
Check that the limits are set by the following command:
sudo su nginx --shell /bin/bash --command "ulimit -a"
Php
HostCMS requires the following php modules to be enabled: curl, gd, xslt and, of course, mysql. Also, note that now the php5-json package is not virtual and needs to be installed separately. Among other things, connect the caching module opcode xcache. We will use PHP-FPM as the SAPI (interpreter launch mode), however, in order to be able to execute some scripts on schedule, PHP-CLI will also be installed.
sudo aptitude install php5-common php5-fpm php5-cli php5-curl php5-gd php5-mysql php5-xsl php5-json php5-xcache
MySQL
Installing MySQL is quite simple. Several times, the installer will ask you for a password for the root of the database server, you can safely leave it empty - we will change it later using the mysql_secure_installation utility. When it starts, answer that you want to change the root password, delete the test database and test users, and update the permissions on the service database tables.
sudo aptitude install mysql-server
sudo mysql_secure_installation
Read more about setting up a mysql server here . The article is well written, so I see no reason to duplicate the information here.
File Upload Settings
As a file transfer protocol, I suggest using SSH FTP (SFTP) . Firstly, it is safer than regular ftp, because the data will be transmitted in encrypted form. Secondly, you don’t have to install additional software: all we need is an ssh server, we already have it. And there are practically no minuses - all modern IDEs and data download clients are able to work with this protocol.
To determine who can connect via sftp, create an additional user group, for example, sftp :
sudo groupadd sftp
And we activate the data transfer by adding the lines to the end of the / etc / ssh / sshd_config file :
Match Group sftp
ChrootDirectory %h
ForceCommand internal-sftp
AllowTcpForwarding no
File System Preparation
Traditionally, files related to websites are located in the / var / www / directory . And we will not depart from this unspoken rule. Create a folder for virtual hosts and a future fast cache mount point:
sudo mkdir -p -m 755 /var/www/data
sudo mkdir -m 777 /var/www/tmp
Then we indicate that at the next boot, tmpfs will be mounted in this folder. Add to / etc / fstab :
tmpfs /var/www/tmp/ tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777,size=128M 0 0
It is worth noting that some editions of HostCMS have a built-in algorithm for caching responses to files. If you use one of these editions, it makes sense to mount tmpfs to the cache directory of the CMS itself.
Configure Virtual Hosts
If you plan to deploy several sites running HostCMS on your server, then the procedure described below will have to be repeated several times. In this case, it makes sense to try to automate the process of creating a new host. I advise you to pay attention to a small set of scripts described in this article and “finish” them for yourself.
Host User Institution
For security reasons, all files associated with our site will belong to a specially wound user in the system. Connecting via sftp and executing PHP scripts will occur on its own behalf. To make it easier, you can call it by the name of your site:
sudo useradd -b /var/www/data -s /usr/lib/sftp-server -m -U -G sftp example.com
sudo passwd example.com
sudo su example.com --shell /bin/bash --command "mkdir -m 0755 ~/data ~/log && mkdir -m 0777 ~/tmp"
For chroot to work correctly, make root the owner of this user's home directory:
cd /var/www/data
sudo chown root:root example.com
Pool establishment PHP-FPM
The php-fpm pool will be launched on behalf of the user that we created in the previous step. To interact with the front-end server, a Unix socket will be used. In addition, you can configure the number of running processes for processing requests, the type of logging, and some other php-specific parameters for your site.
Pool config example
[example.com]
user = example.com
group = example.com
listen = /var/run/php5_example.com.sock
listen.backlog = 4096
listen.owner = nginx
listen.group = nginx
listen.mode = 0660
process.priority = 0
chdir = /
pm = dynamic
pm.max_children = 64
pm.start_servers = 8
pm.min_spare_servers = 4
pm.max_spare_servers = 16
pm.process_idle_timeout = 60s;
pm.max_requests = 256
access.log = /var/www/data/example.com/log/php.access.log
access.format = "%R # %{HTTP_HOST}e # %{HTTP_USER_AGENT}e # %t # %m # %r # %Q%q # %s # %f # %{mili}d # %{kilo}M # %{user}C+%{system}C"
slowlog = /var/www/data/example.com/log/php.slow.log
request_slowlog_timeout = 2s
request_terminate_timeout = 300s
php_admin_flag[display_errors] = off
php_admin_flag[log_errors] = on
php_admin_value[error_log] = /var/www/data/example.com/log/php.error.log
php_admin_value[memory_limit] = 32M
php_admin_value[open_basedir] = /var/www/data/example.com/:.
php_admin_value[upload_tmp_dir] = /var/www/data/example.com/tmp
php_admin_value[session.save_path] = /var/www/data/example.com/tmp
Creating a virtual host config
In the nginx host configuration file, you will need to specify the domain name of the site, the path for writing access logs and the address of the unix socket that php-fpm is listening to. To handle requests for non-existent files, we will use the named location - this way we will emulate the mod_rewrite operation for Apache2. Before giving the script to our backend for processing, we check its existence. This avoids the problem described here . In order to reduce the load on the site from unregistered users, we will use caching on the nginx side. To do this, create the configuration file /etc/nginx/conf.d/cache with the following contents:
fastcgi_cache_path /var/www/tmp/cache levels=1:2 keys_zone=cache:32m max_size=128m;
fastcgi_temp_path /var/www/tmp/proxy 1 2;
fastcgi_ignore_headers Expires Cache-Control;
fastcgi_cache_lock on;
fastcgi_cache_lock_timeout 60s;
fastcgi_cache_use_stale error timeout updating invalid_header;
fastcgi_cache_bypass $cookie_PHPSESSID;
fastcgi_no_cache $cookie_PHPSESSID;
fastcgi_cache_key $scheme$host$request_uri;
And then we will connect it in the virtual host config.
Nginx host config example
server {
listen 80;
server_name example.com www.example.com;
access_log /var/www/data/example.com/log/nginx.access.log main;
error_log /var/www/data/example.com/log/nginx.error.log;
root /var/www/data/example.com/data;
error_page 404 /404/;
location / {
index index.html index.php;
try_files $uri $uri/ @hostcms;
}
# php скрипты отдаем в php-fpm, предварительно проверяя их существование
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass unix:/var/run/php5_example.com.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
include /etc/nginx/conf.d/cache;
}
# все запросы, для которых не нашлось файлов, переадресуются на index.php
location @hostcms {
fastcgi_pass unix:/var/run/php5_example.com.sock;
fastcgi_param SCRIPT_FILENAME $document_root/index.php;
include fastcgi_params;
include /etc/nginx/conf.d/cache;
}
Creating a site database
Now almost everything is ready, it remains only to deploy the database and create a user on behalf of whom connection will be made to it. To do this, execute several simple commands in the mysql console:
CREATE USER 'example_com'@'localhost' IDENTIFIED BY 'ВашСуперСтойкийПароль';
GRANT USAGE ON * . * TO 'example_com'@'localhost' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0;
CREATE DATABASE IF NOT EXISTS example_com_db DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;
GRANT ALL PRIVILEGES ON example_com_db . * TO 'example_com'@'%';
Please note that connection to the database on behalf of this user is allowed from any address.
If you have a dump of the previously used database, then you can deploy it with the following set of commands all in the same mysql console:
use example_com_db;
source ПутьДоДампаБазыДанных;
Setting up backup and log rotation
If you did everything correctly, then you should already have a fully configured environment for launching your project. There are two last steps in the list, but not in significance, - setting up backup and log rotation. As a tool for creating backups, I recommend using backup-manager. The habr has an excellent article about it, so we will not dwell on it in detail.
To perform log rotation, we just need to create the correct config for the logrotate utility.
For example, such
/var/www/data/example.com/log/nginx*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root root
sharedscripts
postrotate
[ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
endscript
}
/var/www/data/example.com/log/php*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root root
postrotate
invoke-rc.d php5-fpm reopen-logs > /dev/null
endscript
}
Instead of a conclusion
Perhaps this is all I wanted to say.
Perhaps someone will find the article not too relevant due to the dominance of hosting control panels. Although, in my opinion, they are good for mass provision of services and are completely unsuitable when it comes to honing server settings for a specific project.
Others will find it a little messy. It is possible that this is so: in the article I just tried to reflect my confused experience in the field of server administration.
In any case, I will be glad if this material helps someone. Comments and additions are welcome.