Rsync Your Site To Dropbox

Just want to share some rsync code snippets that I used to backup my data to either Dropbox or Jungle Disk on my Mac.

I have problems finding/Googling the proper flags to use for rsync for Dropbox and Jungle Disk when I first got started with rsync on the Internet, hopefully this blog post will be useful for someone.

As Dropbox is essentially a folder on your Mac, the rsync flags are pretty straight forward, however the same cannot be said for Jungle Disk as it is using WebDAV and storing the file over at Amazon S3 or Rackspace Cloud Files where file modification times will not be accurate.

• I am using rsync 3.0.8. You can compile rsync 3.0.8 on your Mac by following this tutorial. You just have to replace 3.0.7 with 3.0.8 in the commands.
• For Jungle Disk, I am using Rackspace Cloud Files instead of Amazon S3 because Rackspace does not charge for data transfer and hence it will be cheaper.
• I am doing an rsync from my remote host to my computer and any files not found on my remote host will be deleted on my computer.

/usr/local/bin/rsync -avz --delete --log-file='/path/to/Dropbox/logs/rsync/yourdomain.com.txt' --log-file-format='[%o] %f (%l bytes) (Last Modified: %M)' --exclude-from '/path/to/Dropbox/web/exclude_yourdomain.com.txt' -e ssh username@yourdomain.com:/home/username/public_html/ /path/to/Dropbox/web/yourdomain.com/

Rsync Flag Description

• -a aka --archive is a short hand for -r -l -p -t -g -o -D. Quick way of saying you want recursion and want to preserve almost everything

  • -r aka --recursive Tells rsync to copy directories recursively
  • -l aka --links When symlinks are encountered, recreate the symlink on the destination
  • -p aka --perms Causes the receiving rsync to set the destination permissions to be the same as the source permissions
  • -t aka --times Tells rsync to transfer modification times along with the files and update them on the remote system
  • -g aka --group Causes rsync to set the group of the destination file to be the same as the source file
  • -o aka --owner Causes rsync to set the owner of the destination file to be the same as the source file
  • -D aka --devices --specials Causes rsync to transfer character and block device files to the remote system to recreate these devices as well as special files such as named sockets and fifos

• -v aka --verbose Increases the amount of information you are given during the transfer

• -z aka --compress Compresses the file data as it is sent to the destination machine, which reduces the amount of data being transmitted

• --delete Tells rsync to delete extraneous files from the receiving side (ones that aren't on the sending side), but only for the directories that are being synchronized

• --log-file Tells rsync to log what it is doing to a file

• --log-file-format Specify exactly what per-update logging is put into the file

• --exclude-from Specifies a file that contains exclude patterns (one per line)

• -e Specifies the remote shell to use

Layman's Terms

• I am logging the rsync operation to the file /path/to/Dropbox/logs/rsync/yourdomain.com.txt with the log format [%o] %f (%l bytes) (Last Modified: %M) which translate to (as an example) 2011/07/14 08:00:00 [755] [recv] path/filename.txt (1024 bytes) (Last Modified: 2011/07/13-01:00:00).
• I am excluding all files and folders contained in this text file /path/to/Dropbox/web/exclude_yourdomain.com.txt. Each file or folder name you want to exclude should be on a new line in that text file.
• I am logging into yourdomain.com with the username username via SSH. You will be prompted to key in your password.
• Once my credentials are verified, I will be transferring all the files and folders in /home/username/public_html/ of the remote host to /path/to/Dropbox/web/yourdomain.com/ in my local computer.

Take a look at Rsync Manual Page for more detailed explanations.

I am up for discussions under the comments =)