Backup Using Open Source Solution - Bareos

Having an up-to-date backup at hand is an extremely important point, because no one is safe from unpleasant cases associated with media failure, loss of information, accidental deletion, etc. In such situations, the backup will save not only the nerves, but also relieve possible financial problems that may arise due to data loss.

Bareos - was chosen as a backup system for the following reasons:

• is open source;
• actively developing;
• has many useful features;
• can expand its functionality thanks to plugins.

Task description

Let's say we have 3 cars. Servers: Bareos, BitrixVm, Windows Server 2012 R2. For example, all this will be located within the same network, but Bareos itself, of course, allows you to back up if the machine is located outside it.

Bareos - 172.16.10.10
BitrixVM - 172.16.10.11
Windows Server - 172.16.10.12

Add a repository:

# wget http://download.bareos.org/bareos/release/latest/CentOS_7/bareos.repo -O /etc/yum.repos.d/bareos.repo

For a server with Bareos, we will use CentOS 7. A typical installation occurs:

# yum install -y bareos-client bareos-database-tools bareos-filedaemon bareos-database-postgresql bareos bareos-bconsole bareos-database-common bareos-storage bareos-director bareos-common

PostgreSQL will be used as the database. Installation is also simple:

# yum install -y postgresql-server postgresql-contrib

When installing Bareos in / etc / bareos, we have the following list of files and folders, with the exception of the ssl directory, we will move on to it a bit later:

/bareos-dir.d
/bareos-sd.d
/ssl
bareos-dir.conf
bareos-fd.conf
bareos-sd.conf
bconsole.conf
.rndpwd

After installation, prepared scripts for working with Bareos will appear in the directory / usr / lib / bareos / scripts /. Thanks to them, we will perform preliminary configuration of the database (we will create the database, tables, and rights):

su postgres -c /usr/lib/bareos/scripts/create_bareos_database
su postgres -c /usr/lib/bareos/scripts/make_bareos_tables
su postgres -c /usr/lib/bareos/scripts/grant_bareos_privileges

The important point is that the names of our machines must resolve, if not, add the appropriate lines to / etc / hosts

172.16.10.10	bareos-server
172.16.10.10	bareos-fd
172.16.10.11	bitrixvm
172.16.10.12	win-fd

bareos-server - the server itself, bareos-fd - we will also backup the server itself, i.e. at the same time, it will act as a client, then in the configuration it will be visible, bitrixvm - the name of the client speaks for itself, win-fd - a client with Windows Server 2012 R2.

The server part for BareOS, namely the director, must be configured in the bareos-dir.conf file. The director is responsible for all operations performed.

From the main directives:

Director — описание самого директора,
Storage — устройство на которое пишем бэкапы,
Catalog — содержит информацию по выполненным Job, сохраненным файлам, клиентам, статусам,
Messages — какие сообщения будут собраны и как их доставлять,
Console — настройка консоли для управления директором,
Client — описание клиента, с которого будут сниматься резервные копии,
Pool — позволяет управлять т.н. Volume куда будут писаться данные для разных типов бэкапа (Full, Incremental, Differential), ограничить сроки хранения Volume, размеры,
FileSet — что бэкапим и дополнительные атрибуты,
Schedule — расписание,
Job — описание задачи по бэкапу.

Из официальной документации схема взаимодействия между службами:




# Add two other clients as separate configuration files.

@/etc/bareos/bareos-dir.d/win.conf
@/etc/bareos/bareos-dir.d/bitrixvm.conf

It is important that there is a correspondence between the configurations:



После всех основных настроек нужно перезапустить службы bareos:

systemctl restart bareos-dir
systemctl restart bareos-sd
systemctl restart bareos-fd

In case of errors, we look at the state with the command:

systemctl status bareos-dir.service -l

Let's move on to installing the bareos client plugin for Windows. You can download it here .

Installing the plugin is also simple. From the list of offered components, we select only two indicated on the screenshot: then we set the name for the client, specify the parameters of the existing director:



We write such data:

Client Name - win-fd
Director Name - bareos-server
Password - zcx @ # $ BGj
Networks Address - bareos-server

The entered settings can always be corrected, the default configuration file is in "C: \ Program Data \ Bareos \ bareos-fd.conf".

In the hosts file, we also specify the IP for the BareOS server so that the name resolves. On the server side (bareos-server), the config for the client (win-fd) looks like this and is located on the path /etc/bareos/bareos-dir.d/win.conf:


From the FileSet directive we see that for the Job-win-Full task we copy the users folder “C: \ Users” and in the Job-win-Full-restore task we restore it to the folder if necessary “C: \ tmp”.

There is still one client (bitrixvm) for which you need to copy the root directory and in a separate task copy the mysql database.

Let's go to the bitrixvm machine and complete the client installation:
yum install -y bareos-client bareos-common Go

to the / etc / bareos directory

/bareos-dir.d
/bareos-fd.d
/bconsole.conf
/ssl
.rndpwd
tray-monitor.d

To configure the client, go to the directory /bareos-fd.d

/client
/director
/messages

By default, after installation, the configuration is scattered into the three directories shown above, so for further convenience, we will comment the partial configurations in the directories director and messages, and make all the settings in /client/myself.conf

We enter the parameters and restart the service:

/etc/init.d/bareos-fd restart

We write the hosts address for bareos-server in the hosts

From the server side (bareos-server), the configuration for the client (bitrixvm) looks like this and is /etc/bareos/bareos-dir.d/bitrixvm.conf:


In the RunScript section of the Command parameter, we run the mysqldump command, passing all the necessary parameters “mysqldump -uroot -ppassword --opt --all-databases> /tmp/dump.sql”.

The saved dump.sql file will then be backed up as indicated for the FileSet with the name “bareos-fileset-bitrixvm-mysql”.

Bareos uses the TLS protocol to encrypt data transfer. For each client and for the director, we must have a CA certificate, certificate and key. We will use self-signed certificates, all this can be done through openssl.

Let's go along the path of the openssl configuration file /etc/pki/tls/openssl.cnf

Generate a CA certificate:

#openssl req -config openssl.cnf -new -x509 -extensions v3_ca -keyout private/myca.key -out certs/myca.crt

After entering the password and the rest of the data, we get two files:

/etc/pki/CA/private/myca.key
/etc/pki/CA/certs/myca.crt

We’ll tweak / check the openssl.cnf settings file so that the paths to the certificates are correctly indicated, bold that needs to be changed is highlighted.

You also need to create index.txt and serial files in the / etc / pki / CA folder:

touch /etc/pki/CA/index.txt

We immediately enter the value “01” into the serial file, the file will contain the following number for the following certificates:

echo '01' > /etc/pki/CA/serial

We generate a key for the server, you will need to enter a password.

openssl genrsa -aes256 -out bareos-server.key.pem 4096

We generate a certificate request. The important point is that in this paragraph when generating a certificate, you must correctly specify the Common Name (CN) so that it matches the name of the machine, otherwise errors will occur when trying to perform any task. In this example, CN = bareos-server
openssl req -config openssl.cnf -key bareos-server.key.pem -new -sha256 -out bareos-server.csr.pem We

sign CSR via CA, we get the certificate itself

openssl ca -config openssl.cnf -in bareos-server.csr.pem -out bareos-server.cert.pem

We remove the password for the key, because bareos need it passwordless

openssl rsa -in bareos-server.key.pem -out bareos-server.nopass.key.pem

Convert the CA key and certificate into a single file with the .pem extension

cat /etc/pki/CA/private/myca.key /etc/pki/CA/certs/myca.cert > ca-chain.cert.pem

Generate DH Key

openssl dhparam -out dh1024.pem -5 1024

Copy the bareos-server.cert.pem bareos-server.nopass.key.pem ca-chain.cert.pem dh1024.pem certificates to the / etc / bareos / ssl folder, which you must first create and install a bareos group.

For each certificate, change the group on bareos

chgrp bareos *

Similar actions need to be done for each client (bitrixvm, win-fd):

openssl genrsa -aes256 -out bitrixvm.key.pem 4096
openssl req -config openssl.cnf -key bitrixvm.key.pem -new -sha256 -out bitrixvm.csr.pem
openssl ca -config openssl.cnf -in bitrixvm.csr.pem -out bitrixvm.cert.pem
openssl rsa -in bitrixvm.key.pem -out bitrixvm.nopass.key.pem

Using SCP, copy the 4 files bitrixvm.cert.pem bitrixvm.nopass.key.pem ca-chain.cert.pem dh1024.pem to the remote bitrixvm machine in the / etc / bareos / ssl folder, which you must first create and install the bareos group as for folder so for certificates.

In the bitrixvm client configuration (/etc/bareos/bareos-fd.d/client/myself.conf) in the Director section {it is necessary that client certificates are registered, but not directors. The server-side configuration (/etc/bareos/bareos-dir.d/bitrixvm.conf) in the Client {} section shows the director’s certificates. After making configuration changes, restart the services.

Creating certificates for the win-fd machine is identical.

You also need to register certificates for bconsole, a utility that allows you to manage the director in the /etc/bareos/bconsole.conf file:


Процедура бэкапа или восстановления выглядит следующим образом. Воспользуемся утилитой bconsole, в качестве приветствия увидим *
run Покажет все возможные Job:


The result of the command can be viewed either through the message command and in response we get something like this:


Или через команду status dir


Where in the status column you can view the backup status. Job number 75 was successful.

Recovery occurs by the restore command

*restore

After which a list of all possible options will be offered.

To select the JobIds, you have the following choices:

1: List last 20 Jobs run
2: List Jobs where a given File is saved
3: Enter list of comma separated JobIds to select
4: Enter SQL list command
5: Select the most recent backup for a client
6: Select backup for a client before a specified time
7: Enter a list of files to restore
8: Enter a list of files to restore before a specified time
9: Find the JobIds of the most recent backup for a client
10: Find the JobIds for a backup for a client before a specified time
11: Enter a list of directories to restore for found JobIds
12: Select full restore to a specified Job date
13: Cancel

We show the list of the last 20 tasks:



Click 3 and specify the JobID number, for example 75. After that, we automatically get to the console to select the files that we want to restore

cwd is: /
$

A list of all available commands can be viewed through the help command. We check what files are available in this backup, mark the necessary files with the mark command through the file indication or through *, thus selecting everything. Upon completion, we execute the done command, after which the dialogue with the system will go.

$ ls
tmp/
$ cd tmp/
cwd is: /tmp/
$ ls
dump.sql
$ mark *
1 file marked.
$ done

The job will require the following
Volume (s) Storage (s) SD Device (s)
=============================== ==============================================

bitrixvm-bareos-client- Full-0011 bareos-server-sd bareos-sd

Volumes marked with "*" are online.

1 file selected to be restored.

Select the recovery task we need (item 2)

The defined Restore Job resources are:
1: Job-Full-restore
2: Job-bitrixvm-mysql-restore
3: Job-bitrixvm-Full-restore
Select Restore Job (1-3): 2

Set the client for which we are performing recovery (2)

Defined Clients:
1: bareos-fd
2: bitrixvm
3: win-fd
Select the Client (1-3): 2

Using Catalog “bareos-server”
Run Restore job
JobName: Job-bitrixvm-mysql-restore
Bootstrap: /var/lib/bareos/bareos-server.restore.1.bsr
Where: / tmp
Replace: Always
FileSet: bareos-fileset- bitrixvm-mysql
Backup Client: bitrixvm
Restore Client: bitrixvm
Format: Native
Storage: bareos-server-sd
When: 2016-10-16 11:26:54
Catalog: bareos-server
Priority: 10
Plugin Options: * None *
OK to run ? (yes / mod / no): yes We

confirm the task through yes, with the mod command you can edit the recovery options before starting, including changing the directory for recovery. Next, the task is assigned JobId.

Job queued. JobId=76

По команде status dir можно увидеть состояние восстановления. Как видим все прошло успешно. После этого можно переходить на удаленный клиент, и в папке /tmp обнаружим восстановленный файл dump.sql.

76                1    1.100 M  OK       16-Oct-16 11:27 Job-bitrixvm-mysql-restore

В заключении можно сказать, что Bareos активно развивается, имеет в своем арсенале множество полезных функций, которые остались за пределами этой статьи, а наличие плагинов, которые могут расширить функционал также оказывают положительное впечатление (резервное копирование MS SQL, плагины для хранения данных на Ceph, Gluster, бэкап LDAP).

---

SIM-CLOUD — Отказоустойчивое облако в Германии

Выделенные серверы в надежных дата-центрах Германии!
Любая конфигурация, быстрая сборка и бесплатная установка