Cheap cloud backups with encryption for home
Greetings, Habrauser! I’m going to tell you about the C14 cloud storage and a small utility that facilitates the routine work when working with the features of this storage.
Looking ahead, I’ll say that the storage cost of 1 TB is 2 euros per month. This storage was already mentioned earlier on Habré: The nuclear bunker in Paris will be converted into the data center of the company online.net
If you are familiar with C14, you can skip this part and move on to the next.
Online.net has a wide variety of services and several repositories. We are only interested in C14. They take money for storing, deleting, opening an archive. You can approximately estimate your spending on their website . Operations mean archiving, deleting and opening an archive with files.
The following storage scheme is proposed:
We can create quite a lot of storages in C14 (here, and hereinafter referred to as safe in their terminology). Each storage can have several archives. Archive is an OS with limited access. You can connect via
At first, I wanted to put all the code in one file, but the reality was much harsher, so I decided to share my solution with all of them.
What the utility can do:
What she does not know how:
For use, you will need. For xmpp logs, you will need (optional) Download a script for those who do not have composer and do not need xmpp logs: Otherwise:
Register at online.net . Add billing information . Generate ssh keys (if you don’t have them yet) through
API requests are not always instantly applied. Therefore, the code is forced
I will be glad to answer your questions. Link to the repository .
Looking ahead, I’ll say that the storage cost of 1 TB is 2 euros per month. This storage was already mentioned earlier on Habré: The nuclear bunker in Paris will be converted into the data center of the company online.net
Vault C14
If you are familiar with C14, you can skip this part and move on to the next.
Online.net has a wide variety of services and several repositories. We are only interested in C14. They take money for storing, deleting, opening an archive. You can approximately estimate your spending on their website . Operations mean archiving, deleting and opening an archive with files.
The following storage scheme is proposed:
We can create quite a lot of storages in C14 (here, and hereinafter referred to as safe in their terminology). Each storage can have several archives. Archive is an OS with limited access. You can connect via
sshfs, rsync, scp, ftp, webdav
. The archive is available for connection within 2, 5 or 7 days, after which it will be automatically archived and placed very deep in the bunker. If you need to restore the data, a new OS will be created and /buffer
all your previously saved files will appear in the folder . If you try to change the restored archive, the content will be copied and the archive will be expanded.C14 backup tool
A few words about BackInTime
Since BackInTime does not work in such conditions (due to the fact that copying hard links to files goes through a remote server by running
rsync
through ssh
), and I want to automate and save data in case of fire, I wrote a program.At first, I wanted to put all the code in one file, but the reality was much harsher, so I decided to share my solution with all of them.
What the utility can do:
- Encrypt data on the client side.
- Do not backup the specified folders. Rsync regular expressions in this case are only available in non-encrypted mode.
- Automatically create safe archives, put access keys there and delete obsolete archives.
- And also, create incremental backups within one archive to reduce the cost of storage.
- Each archive is a fully working backup. Therefore, they can be deleted without fear (if the description of the archive contains at least one line with the date of successful archiving).
- Write large and not very xmpp logs.
- It does not require composer if you do not need xmpp logs.
- Several profiles for copying all kinds of data.
- Does not use an official API access program
- Saves the encfs config to each data archive
What she does not know how:
- Thoroughly check the config and necessary dependencies
- Work if you have more than 100 safe or more than 100 archives in one safe.
Installation
For use, you will need. For xmpp logs, you will need (optional) Download a script for those who do not have composer and do not need xmpp logs: Otherwise:
rsync
encfs
sshfs
ssh
composer
git clone https://github.com/TheSaturn/C14BackupTool.git
cd C14BackupTool
cp config.example.php config.php
composer create-project thesaturn/c14-backup-tool
cp config.example.php config.php
Setup:
Register at online.net . Add billing information . Generate ssh keys (if you don’t have them yet) through
ssh-keygen
. Edit config.php
: you will need to enter the API key, select folders, etc. Run the script ( php /path/to/main.php profileName
) and wait for a message about the successful completion of the backup. Add script to cron as desiredBumps:
API requests are not always instantly applied. Therefore, the code is forced
sleep()
. Mounting on sshfs
and connecting rsync
does not always happen the first time either. Support throws up his hands and claims that everything is fine. Therefore, several attempts are made to connect to the repository.A short version of the algorithm for those who want to understand better
- Sending process start message
- Finding safe with the given name
- Archive search in safe where to store backups, if the latter was opened 6 days ago, a new one is created for 7 days
- Create required folders
- Mount encfs (encrypted fs representation)
- The names of files included and excluded from backup are encrypted (otherwise rsync will not understand)
- Mount storage
- Copying the config encfs
- Clearing the folder for temporary backup (in case of unsuccessful backup, files may be located there)
- Creating hard links from past backup
- Sending a message about the beginning of this backup
- Rsync
- Rename temporary folder
- Recording information about the date of successful backup creation in the description of the C14 archive
- Sending a message that everything went well
- Unmount sshfs and encfs
I will be glad to answer your questions. Link to the repository .