Backing up your Photos – A Cautionary Tale

A recent article appeared on Petapixel regarding a Montreal photojournalist having all his photos stolen by burglars:

A photographer’s worst nightmare just happened to a well-known photographer: on Monday, Montreal-based photojournalist Jacques Nadeau returned home to find that burglars had stolen all the photos he has taken during his life and career.

CBC News reports that Nadeau, a photojournalist for the newspaper Le Devoir, walked into his home find that five of his hard drives had been stolen.

They contained an estimated 30,000 to 50,000 photos captured over the course of his 35 year photography career.

This is a terrible story, and absolutely devastating to the photographer.  My heart goes out to him.  But we can take a lesson from this…

Embrace the Paranoia.  Always ask “What if….”

Take a look around you.  At your life, at your belongings, at things you hold dear.  Ask yourself “What would happen if this were lost or destroyed?”  If the answer is “This is irreplaceable”, then move on to the next question “How can I protect these things in a way that makes sure they’re never lost?”

For anyone in the digital world, the answer is simple.  Backups.  There are myriad sites singing the song “Always do your backups!” and “Here’s how to back up your things!”  I won’t go into detail here.  But people should extend that idea to other things of value.  Important documents.  Printed photos.  Artwork.  That doll from your youth.  Look at these things of value and be a little paranoid.  “How could this be destroyed?”  Some china inherited from a relative – is it on a shelf that can be knocked over easily?  A doll you once cuddled as a child, perhaps putting it out of reach of the dog would be a good idea?

Yea yeah, okay.  So how do YOU do it?

I’m glad you asked!  This article happened to appear while I was in the middle of backing up my photo library!

Currently, I do all my photo work in Aperture.  Apple has announced that this product is being end of lifed, so no matter what, I’ll need to do a bunch of work migrating photos.  I keep my photo library on an external 1TB USB3 drive, and I’m acutely aware of how fragile that is.  Hard drives fail constantly, and having all my eggs in one basket is never a good idea.  The challenge is, photo libraries are BIG.  Hundreds of gigabytes of data.  If I were to try to back up my Aperture library onto DVD+R DS (the largest ‘consumer level’ long term storage medium available at 17G per disc),  I’d need 31 some odd discs.  That’s too many, and cumbersome as heck to work with.

I considered Dropbox, Box.net, Google drive, and Amazon Drive, but I feel these are targeted at a desktop user who just wants a drive out in the cloud.  While I use Dropbox extensively for making photos available to customers, it’s sync mechanism is quite tricky if what you’re storing on Dropbox is much larger than what you can store locally.  I’m also not confident these systems will last, unchanged and accessible, for the long term.  Google, in particular, has a dreadful record for keeping products and offerings available for the long run.

In the end I decided to use a pretty technical solution:  Amazon S3 storage.

Backing up to S3 and Glacier

Amazon has a bulk storage system called S3, coupled with a ‘long term storage’ system called Glacier.  S3 is in essence a big storage bucket where you can drop files and retrieve them at will.  Glacier allows you to take S3 elements and put them in, as you might guess from the name, ‘Cold storage’.  The costs for S3 storage is extremely low ($0.0240 per GB per month, or for my 600G of photo data, about $14/mo).  If I move those files into Glacier, it drops to $6/mo.  The difference is that restoring data from Glacier may not be immediate – it may take a few hours for your files to be available.  For this sort of long term storage, that’s fine by me!

This is not as cheap as current offerings from Amazon Prime (Unlimited storage free with Prime and Amazon Drive).  But I’m still very skeptical of the ‘drive’ offerings from the big players.  Everyone is trying to get into the “cloud drive” market with custom clients and apps.  My storage needs are exceedingly simple.  About 300 very large files (copies of each of my photo projects).  S3 is extremely well established, and used widely in the industry.

With S3, to back up my library, I go through these (for me) straightforward steps:

  • In Aperture, I select a project, and say “Export to library”.  I locate that library on my external drive.  This is an exact copy of my original masters / RAW images, as well as all the ‘versions’ I may have created (all in JPG form).  It’s also including metadata and Aperture edit notes.  While I know Aperture is not long for the world, I at least have things backed up.  This results in a directory that contains a mini ‘apilibrary’ containing all my files.
  • From the command line, I make a ‘tgz’ of that directory.  This compresses the directory down into a single file.  If I were so inclined, I could do this on the Mac just by selecting the directory and choosing ‘Compress’ – that will create a .zip file containing the entire library.
  • Next, I copy the file up to S3.  Because I’m a super-geek, I do this right on the command line using my Amazon credentials I created a while back.  If you’re a GUI person, you can use any number of S3 clients for the mac or PC.  For me, I do:
    aws s3 cp 2014-09-23\ CA\ Over\ 15k.aplibrary.tgz s3://daveshevettphotos/ --profile personal

After some time (some of the libraries are quite large.  A 25gig wedding archive took 85 minutes to upload) I have an offsite backup of that photo library!  Hurray!  At any point I can go to the Amazon S3 console and put these files into Glacier for long term storage, or download them as needed.

I realize this process is not for everyone.  I share here to simply raise awareness that in the modern age, many of our most important things are stored in an ephemeral, easily lost way.  Take the time to look around and see what you could lose if something were to happen.  Something as simple as your laptop being stolen,  a broken water pipe, or even a home fire.  Always ask.. “What if…”

primark

About

A wandering geek. Toys, shiny things, pursuits and distractions.

View all posts by

3 thoughts on “Backing up your Photos – A Cautionary Tale

  1. He kept his life’s work on hard drives that were *not backed up*? That’s — I have no words.

    I use S3 for backing up my blog. For everything else, I am leery of the cloud. Everything I care about is on a single hard drive, which gets cloned to one of three other drives, on a rotating basis, about once a month (so I always have a clone that’s about three months old, another that’s about two months old, and another that’s a month or less old). The three clones are stored upstairs, downstairs, and in a separate building on the other side of my patio. I also do daily backups to yet another drive.

    Aside from the fact that it might be better if the three clones were in three separate states, do you see any flaws in this plan?

    1. WillO, five drives were stolen, what’s to say every one wasn’t redundant? I have multiple copies of things on mulitple drives, but all of those drives are still in the same house.

    2. @WillO – It’s a good plan, though a little more dramatic on the local hard drives than I would have gone. But I’m also pretty well invested in Amazon, S3, and their storage services. S3 is not a ‘flash in the pan, also ran’ system, which is how I feel Amazon Cloud and others are. S3 was doing bulk storage around the time Dropbox and box.net started, so their market is really enterprise storage for apps, not “i back up my cat pictures for you”.

Leave a Reply

Your email address will not be published. Required fields are marked *