Guide to turning the flash drive into the center of synchronization of anything and everything

    I think that every person who uses several computers, sooner or later there was a need for data synchronization between them.

    In my case, the data was: notes (created in the beautiful tree-based ZIM editor ), messenger logs ( Pidgin uses an excellent log format, very convenient for synchronization), program settings (any Linux user and not only will agree that over time the configuration files acquire the value of gold :)), as well as small and not very bash scripts, accumulated over several years.
    In this guide, I will tell you how I solved the problems of synchronizing, protecting and restoring this data in unforeseen situations. Of course, these solutions are not universal, not final (they have been developed over several years and, possibly, will change many times), but for some they are completely primitive, but at the moment they satisfy all my requirements and, in my opinion worthy of publication. I hope someone finds at least something useful for themselves here! :)


    So let's get started. Article I divided into several parts, namely:

    1. Problem Statement
    2. Control versions of important files
    3. Data Protection
    4. Installation of portable programs for the Windows
    5. Backup data
    6. Combine all actions into one

    formulation of the problem

    Once upon a time, I created a folder with the very thoughtful name of saves and began to put more or less important files for me there. In the case of documents, this was convenient enough, but if the file was required to be in a certain place (for example, the configuration file of the program), then a copy of it got there, which was not entirely good.

    After getting to know Linux and its file systems, such configuration files moved to the place of their copy, and a soft link to this file was put in the right place.

    They also got there:

    * File with passwords in keepass format (Linux version of the program - KeePassX )
    * Address book in .vcf format
    * Notes of ZIM editor
    * Bash (and other) scripts
    * And also a lot of other and small

    things. After I got two computers (a working and a home laptop, Debian Etch is installed on both), it became necessary to have exact copies of this folder on both computers and synchronize the changes.
    Over time, the Nokia N800 tablet and a Windows computer were added to the list, which I visit once a week, but this does not eliminate the need to have the same set of greasemonkey scripts for Firefox as the rest.

    Version control of important files

    I decided that simple synchronization would not be enough for the saves folder , but it is better to use some version control system that programmers love so much :)
    After long comparisons, the choice fell on Bazaar - a distributed version control system supported by the notorious Canonical.
    For her, she just had to create a script that performs the sequence of actions I need: adds new files to the repository, offers to enter a comment for new changes (if they are, of course) and merge the changes with the parent repository (for the N800 this is a computer, for a computer it flash drive, but the flash drive does not have a parent repository).
    Now I always have the same saves folder on all computers and in case of any malfunction I can return to the previous version of any file.

    Little addition:
    Very often, the program configuration files, in addition to the actual configuration, contain all sorts of frequently changing parameters. For example, Firefox also saves the last time you use bookmarks in a bookmarked file, which changes every time a bookmark is used. It is clear that such data does not need synchronization, so I created a special prepare_saves script that removes them from the files, as well as the mother of developers who do this :).

    Data protection

    Since the saves folder already contains a sufficient amount of data that I did not want to fall into the wrong hands about the loss of a flash drive, this is the time to think about protecting it.
    Thanks to a habr learned about the TrueCrypt program. Using it, I created a 200 mb file container on a flash drive with the NTFS file system (why NTFS, see the next section) and moved the saves folder there . Since in this case the purpose of encryption is not protection from the FSB, but simply from getting the flash drive into the wrong hands, password protection was enough.
    Hmm ... such an important question, and there is nothing more to say :)

    Installing portable programs for Windows

    Since sometimes you still have to use Windows computers, the PortableApps.org website has become a real find for me. Favorite Firefox and Pidgin, as well as some other programs firmly took their place on the protected section of the flash drive.

    Since the partition has the NTFS file system, I repeated the trick with no problem replacing the configuration files with links to the corresponding files in the saves folder .

    Data backup

    After such an aircraft is created, it is again a pity to lose it on the street. For backup (as well as synchronization), the good old Unison was connected , which can do everything if configured correctly.

    Three profiles were created for him:

    * Backup of all data on the insecure part of the flash drive (excluding the encrypted container file)
    * Backup of all protected data (excluding the saves folder )
    * Pidgin logs synchronization

    Since Unison does two-way synchronization, now to write a file to a USB flash drive, I first write it to the backup directory (paradoxically it sounds), and if necessary, I start the synchronization procedure.

    Combining all the actions into a single whole

    Now the simplest thing remains - to write a script that will combine all the steps together. Its algorithm is very simple:

    1. Mount the protected section of the flash drive
    2. Synchronize the Pidgin logs
    3. Synchronize the insecure part of the flash drive
    4. Synchronize the protected part of the flash drive
    5. Run prepare_saves for saves on the flash drive
    6. Commit the changes to saves on the flash drive
    7. Runprepare_saves for saves on disk.
    8. Make changes to saves on the disk and send them to the USB flash drive
    9. Unmount the protected section of the USB flash drive

    Now, when the USB flash drive is connected to the computer, just click on one tab, enter the password to access the protected container, enter, if necessary, a comment on the changes and ... understand that you spent a lot of time on things that other people don’t even think about :)

    If someone wants to look at the source codes of some of my scripts or just ask questions - welcome! I will only be happy to answer and help :)

    Also popular now: