Cheap cloud backups with encryption for home

    Greetings, Habrauser! I’m going to tell you about the C14 cloud storage and a small utility that facilitates the routine work when working with the features of this storage.
    image

    Looking ahead, I’ll say that the storage cost of 1 TB is 2 euros per month. This storage was already mentioned earlier on Habré: The nuclear bunker in Paris will be converted into the data center of the company online.net

    Vault C14


    If you are familiar with C14, you can skip this part and move on to the next.

    Online.net has a wide variety of services and several repositories. We are only interested in C14. They take money for storing, deleting, opening an archive. You can approximately estimate your spending on their website . Operations mean archiving, deleting and opening an archive with files.

    The following storage scheme is proposed:

    We can create quite a lot of storages in C14 (here, and hereinafter referred to as safe in their terminology). Each storage can have several archives. Archive is an OS with limited access. You can connect viasshfs, rsync, scp, ftp, webdav. The archive is available for connection within 2, 5 or 7 days, after which it will be automatically archived and placed very deep in the bunker. If you need to restore the data, a new OS will be created and /bufferall your previously saved files will appear in the folder . If you try to change the restored archive, the content will be copied and the archive will be expanded.

    C14 backup tool


    A few words about BackInTime
    Since BackInTime does not work in such conditions (due to the fact that copying hard links to files goes through a remote server by running rsyncthrough ssh), and I want to automate and save data in case of fire, I wrote a program.

    At first, I wanted to put all the code in one file, but the reality was much harsher, so I decided to share my solution with all of them.

    What the utility can do:

    • Encrypt data on the client side.
    • Do not backup the specified folders. Rsync regular expressions in this case are only available in non-encrypted mode.
    • Automatically create safe archives, put access keys there and delete obsolete archives.
    • And also, create incremental backups within one archive to reduce the cost of storage.
    • Each archive is a fully working backup. Therefore, they can be deleted without fear (if the description of the archive contains at least one line with the date of successful archiving).
    • Write large and not very xmpp logs.
    • It does not require composer if you do not need xmpp logs.
    • Several profiles for copying all kinds of data.
    • Does not use an official API access program
    • Saves the encfs config to each data archive

    What she does not know how:

    • Thoroughly check the config and necessary dependencies
    • Work if you have more than 100 safe or more than 100 archives in one safe.

    Installation


    For use, you will need. For xmpp logs, you will need (optional) Download a script for those who do not have composer and do not need xmpp logs: Otherwise:

    rsync
    encfs
    sshfs
    ssh


    composer



    git clone https://github.com/TheSaturn/C14BackupTool.git
    cd C14BackupTool
    cp config.example.php config.php




    composer create-project thesaturn/c14-backup-tool
    cp config.example.php config.php


    Setup:


    Register at online.net . Add billing information . Generate ssh keys (if you don’t have them yet) through ssh-keygen. Edit config.php: you will need to enter the API key, select folders, etc. Run the script ( php /path/to/main.php profileName) and wait for a message about the successful completion of the backup. Add script to cron as desired

    Bumps:


    API requests are not always instantly applied. Therefore, the code is forced sleep(). Mounting on sshfsand connecting rsyncdoes not always happen the first time either. Support throws up his hands and claims that everything is fine. Therefore, several attempts are made to connect to the repository.

    A short version of the algorithm for those who want to understand better
    1. Sending process start message
    2. Finding safe with the given name
    3. Archive search in safe where to store backups, if the latter was opened 6 days ago, a new one is created for 7 days
    4. Create required folders
    5. Mount encfs (encrypted fs representation)
    6. The names of files included and excluded from backup are encrypted (otherwise rsync will not understand)
    7. Mount storage
    8. Copying the config encfs
    9. Clearing the folder for temporary backup (in case of unsuccessful backup, files may be located there)
    10. Creating hard links from past backup
    11. Sending a message about the beginning of this backup
    12. Rsync
    13. Rename temporary folder
    14. Recording information about the date of successful backup creation in the description of the C14 archive
    15. Sending a message that everything went well
    16. Unmount sshfs and encfs


    I will be glad to answer your questions. Link to the repository .

    Also popular now: