Storage, backup and cataloging of photographs

    Here periodically write posts about how to store and backup their photos - well, just files. In the last such post I wrote a rather long comment, thought a little and decided to deploy it in the post nonetheless. Moreover, I changed the backup method to the cloud a little, it may be useful to someone.

    Home server, where a lot of the following happens:


    What should be kept?

    My most important and voluminous is photographs. Sometimes a video, but very rare - it takes up too much space and takes up too much time, because I don’t like it too much, I shoot only short videos that are lying in the same heap as the photos. Currently, the photo archive occupies about 1.6 terabytes and is growing somewhere around 200 gigabytes per year. Other important things are much less voluminous and there are fewer issues with them in terms of storage and backup, a dozen or two gigabytes can be crammed into a bunch of free or very cheap places, starting from DVDs and ending with flash drives and clouds.

    How is it stored and backed up?

    My entire photo archive is about 1.6 terabytes at the moment. The master copy is stored on a two-terabyte SSD in the home computer. On memory cards, I try not to keep photos longer than necessary, at the first opportunity I merge them onto a desktop or laptop (when on the road). Although I do not delete the flash drive, if there is still space. An extra copy never hurts. From the laptop upon arrival home, everything also drops off on the desktop.


    A daily copy of the folder with photos is made on the home server (with a mirror type based on Drivepool, where duplication of important folders is configured). By the way, I still recommend Drivepool - for all the years of using a single glitch. It just works. You don’t just have to look at its Russian interface, I sent the developers a more decent translation, but I don’t know when it will be implemented ( update: they said they’ll try in the next release). In the meantime, in Russian, this is a program for managing a pool (manage pool).


    You can, of course, make copies more often, if a lot of work has been done in a day, then I can forcibly run the task. Although now I’m still thinking about starting copying on changing files, I want to stop keeping the desktop turned on around the clock, let the server work more. The program is GoodSync.


    Until recently, files from the same GoodSync desktop were uploaded to the Onedrive cloud. Most of the files I have are not personal, so I uploaded it as is, without encryption. What was personal was uploaded by a separate task, with encryption.

    Onedrive was chosen because the Office 365 Home Premium subscription for 2000 per year yielded five (and now six) terabytes in the cloud. Even with pieces per terabyte. Now, however, the freebie has risen in price a little, but a few weeks ago there were still an option for 2600-2700 a year (you need to look for retailers). I foresaw this, when last year MS lifted prices, and even stopped selling the subscription on the site, because I activated the subscription for five years ahead, there were still 1800-2000 boxes on sale (of course, there were also a few boxes in reserve to take, but I did not dare to think so far).


    Download speed is the maximum for my tariff, 4-5 megabytes / sec., At night up to 10. I looked at crashplan at one time - it’s good there if megabytes per second were loaded.

    Lifetime 5TB for $ 2-3 with ebay - a very random thing. For the life span can be very short, so far three months is a record. It’s not a matter of backing up to a place that can be covered at any time. Even for a penny.


    But now, due to the fact that he decided to drag part of the tasks from the desktop to the server, copying to Onedrive transferred to Duplicati. Let it be beta, but I’ve been using it for several months and so far it has been working quite stably. Since Duplicati still stores its backups in archives, and not in bulk, I decided to encrypt everything downloaded with built-in tools. It’s all the same to restore, if that, through Duplicati it is necessary. So let everything encrypt.

    Given that I have terabytes in pieces, a backup to the cloud consists of several tasks. This is where the backup is being uploaded to the cloud. 2019 was flooded quickly - there were fifty photos in a couple of days, I have traveled a little so far, and 2018 is slowly pouring. The current download speed is not the maximum - day, channels are loaded and all that.


    In the cloud, the backup folder looks like this - there are a lot of zip archives, the size of the archive is adjusted when creating the task:


    About once a month I make a copy on an external drive, which is stored in the cabinet. I connect and run the task with my hands in the same GoodSync. Although, of course, it is possible to set up a launch upon connecting a disk - but I do not always need to make a copy when I connect the disk.

    In a good way, you would need another remote storage location - your own and not very cloudy. On my server, which is located on the provider’s platform, I have already prepared a disk for this business for a long time, but all hands do not reach. But since I took up dragging everything under duplicati, I think I’ll do it now, after I reload everything in onedrive.


    How is it cataloged?

    Here the question is divided into two - the level of the file system, where cataloging is at the folder level and logical cataloging for more parameters, because the folder tree is still limited in capabilities.

    Yes, I’m taking pictures in the rabies. Because you can make jpg from raw at any time, but not vice versa. Once I shot in raw + jpg, so that I could quickly upload photos to my phone and send them to the Internet (it was difficult to transfer raw to my phone). jpg then erased when copying to the desktop. But now the phone began to suit me in terms of photo quality (for uploading to the Internet), therefore I completely refused jpg on cameras. Remained either from those times when I did not have a mirrorless, or come from the phone.


    At the file system level, it looks something like this: at the top level of folders, the source. The names of photographers are usually.


    A lower level is the topic. All are more or less the same, there may be personal themes (for example, “Dogs”, there may not be any topics.


    Then, years. Inside the year, folders by day. There may be separate photo sessions in the folder if the photos are beaten up by day .

    As a result, the file path might look something like this: My \ Travel \ 2018 \ 11.04.2018 Berlin \ French station \ P4110029.ORF

    I take pictures with two cameras, usually in turn, but occasionally I take both with me - then I dump the photos from them into one folder. The main thing is that the time should be synchronized, otherwise then you have to calculate the difference and adjust the shooting date for all files (in the lightroom it is simple, but tedious to consider the time difference).

    For photos from the phone there is a separate folder on the second level, but if necessary, the photo can be sent to a thematic folder.

    Logical cataloging on top of folders - Adobe Lightroom . Of course, there are a lot of programs for cataloging and processing, but the lightroom suits me, it’s quite uplifting (and they even give Photoshop in the kit), and over the past couple of years it has also become less slow. Although, of course, there is also a complete transition to SSD helped.

    All photos live in one directory. The folder structure from the previous paragraph is used mainly, on top of it is EXIF ​​information, geo-tags, tags and color marks. You can also enable face recognition, but I do not use it.

    Based on all of the above, you can create "smart collections" - dynamic selections for specific file properties - from shooting parameters to text in comments.


    All tags are stored in files, the editing history is in XMP files next to the ditches. The lightroom catalog is backed up by the lightroom itself once a week to a specific folder, from where it then drops onto onedrive. Well, plus, through the veeam agent, the desktop disk of the desktop is poured onto the server every day - and the directory is stored on the system disk.

    And what about all the photos? What, there are no other types of files?

    There is, why not. Backup methods do not differ (if you need to backup at all), and cataloging methods depend on the type of content.

    Basically, sorting at the folder level is enough, tags are not needed. For movies and TV shows only, a separate cataloger is used. - Plex Media Server. He is also a media server, as the name implies. But there the horse didn’t lie around, it’s normally sorted well if a quarter of the film library, and the rest is lying in the "! To sort" folder.

    Also popular now: