Project Description
Syncronise all files in a local folder with another drive, or with an Amazon S3 bucket. Can easily extend to support other Cloud storage providers (such as Google).

Motivation
Problem: 6years worth of photos and videos that we don't want to lose, so 75+Gb of data to backup securely, adding to regularly. Might need them one day if there's a 21st or funeral (served with mozarella cheese and son-died tomatoes).

Option #1: Burn 20+ DVDs, then do it again to be sure. Plus 2 more for every 4Gb of new data, careful not to miss any files.
Option #2: Write to portable hard-drive (x2). Add new files to both drives when you upload from the camera.

*These options both get stored in the bolted-down fireproof-grade gun safe (which looks like something worth stealing), and hopefully it doesn't get flooded (e.g. by heavy rain or the firemen). Or you can bury it in a watertight container (separately from any contraband), noting the GPS coordinates and any planned real estate developments.

Option #3: Cloud storage...
I wrote this tiny application (70kB) to regularly sync all my files to/from Amazon S3 cloud storage. Amazon keep 600+ copies distributed around the globe, so the files won't get lost or stolen. It's like storing your treasure in heaven, but you have to pay Amazon (or Google?). At around $15/month, over 10years it will cost about the same as that fireproof gun safe. I can sync all files between all our computers/laptops and Amazon (like Dropbox but bigger), and can make a file public if I need to share it. Initial upload takes about a week, after that about an hour.

Notes:
The same application can also be used for option #2, i.e. syncing directly between 2 drives to make sure nothing is missing.

It can be pre-configured with the Amazon settings, so user only needs to browse to the folder and hit go. I plan to add custom encryption options (Amazon also provide encryption, if you trust them with the keys). Similar apps exist, but this one is simple to use, and adds some basic metadata (computer/user name and file date) for more complex syncing.

I have released it on Codeplex as open source (GNU) freeware for transparency, and can also provide custom features to commercial users.

Screenshot This screenshot shows the use of config settings to lock down and hide the amazon keys, so user just needs to browse to a folder. Note also that the logic currently does the copy in 2 passes, uploading small files first (<5mb e.g. photos), then does the bigger files (video clips etc) in a second pass.
screenshot.png

Last edited Jan 7, 2012 at 10:09 AM by jeremyconnell, version 13