Tuesday, January 31, 2012

Backing up through FTP on a Mac

It's been a little over half a year now that I made the transition to mac. I was more or less forced into it by Dell because they didn't make the 1920x1200 high-res laptops any more that I like working with.
But I have to say, the Apple hardware is second to none - I haven't experienced any mysterious crashes since.

However I had to find Mac alternatives for all the software that I was using, and I managed to find most of what I was looking for quite quickly...
  • For viewing zip archives I currently use Zipeg, it only does extraction but that's good enough more most cases.
  • I really like Breeze for keyboard-based window alignment, similar to what Windows 7 has.
  • And I started using BetterTouchTool to map a keystroke the mousebuttons (saves me from having to press the touchpad all the time, which I don't really like).
  • I'm still undecided about iTunes, I don't really like it (e.g. it's missing a feature that shows what you're playing in the dock or something like that) but it kinda does what it needs to do and there doesn't seem to be anything better on the Mac, strangely enough...
  • Most other applications that I had been using before have Mac versions, except for...
... a backup solution! I was always using Areca in the past which works great and had both Windows and Linux versions available. I guess everybody on Mac simply buys a Time Machine but I wanted to use my existing network storage drive for my backups.

I found iBackup and used it for about half a year. It was seemingly able to do the work over a Samba mount to my network drive, although I did notice that I was getting many failure runs, in fact over half of the runs were failures. It might have been caused by the SMB implementation on my NAS drive, I don't really know but I noticed that it was having issues with relatively large files (over a couple of megs).
I started looking at using FTP to do the backups instead (my NAS drive supports FTP) but iBackup doesn't support that.

After looking around for a while I couldn't find any freely available backup option for Mac that uses FTP. I might have missed one or two, but in the end I started writing a little shell script to do the job. One of the things that I was worried about was the backup speed. My script logs in and out for every directory recursively so I was concerned that it would take much longer to complete than the SMB option. I was very surprised that for my data it took 9 minutes to complete, where the SMB-based option took several hours if it completed at all.

So here it is, a simple little bash script to backup any directory on your mac using FTP, I invoke it like this
example..
ftp-bku.sh ~/bin ~/docs ~/etc ~/utils
Note that I have my FTP credential in my ~/.netrc file so that I don't need to provide them in the script itself...

For those who are interested, you can find the script below (btw one thing that isn't completely reliable yet is the return code of the ftp program. It's 0 in some cases when there actually is a problem. Not sure if this can be fixed, although I could grep the output for error messages...)

FTPURL=ftp://User@192.168.1.24:2121
FTPBASEDIR=MyDir
TIMESTAMP=$(date +"%d%m%y-%H%M%S")
BACKUPDIR=backup_`hostname`_$TIMESTAMP
# Perform the actual FTP backup
function backup {
    PNAME="$1"
    DNAME="$2"
    ftp -i -V $FTPURL<<EOF
    bin
    cd "$FTPBASEDIR"
    mkdir "$BACKUPDIR"
    cd "$BACKUPDIR"
    lcd "$PNAME"
    mkdir "$DNAME"
    cd "$DNAME"
    mput *
    quit
EOF
    if [ "$?" == "0" ]; then
        echo Backup succeeded: "$BACKUPDIR"
    else
        echo Backup failed: "$BACKUPDIR"
    fi
}
# Backup the directory specified as $2 recursively where the base directory is $1
# So to backup /Users/david/etc/tmp as etc/tmp you pass in 
#  /Users/david/etc/tmp etc/tmp
function backup-dir {
    cd "$1"
    if [ -d "$2" ]; then
        BASE="$1"
        DIR="$2"
        BASELEN="${#BASE}"
        SUBDIR="${DIR:BASELEN}"
        TARGET="`basename "$1"`$SUBDIR"
        backup "$2" "$TARGET"
    fi
    cd "$2"
    for dir in *
    do
        if [ -d "$2/$dir" ]; then
            backup-dir "$1" "$2/$dir"
        fi
    done
}
# Staring point, backup all directories specified on the command line
for directory in "$@"
do
    backup-dir "$directory" "$directory"
done

6 comments:

Ugo Sangiorgi said...

Nice, but why dont you use rsync instead? it will transmit only the modified files over time.

dave said...

I guess everybody on Mac simply buys a Time Machine but I wanted to use my existing network storage drive for my backups.

I think you meant "Time Capsule" -- Time Machine is the software, which comes built in to OS X.

Personally I just use a hard drive attached to my router with Time Machine, though it does have to be HFS+ formatted...

However, it is possible to use Time Machine with any network-attached drive. It's not simple, unfortunately, but it is possible:
http://code.stephenmorley.org/articles/time-machine-on-a-network-drive/

davidb said...

@Ugo - AFAIK rsync requires a daemon on the server side which my NAS box doesn't have, or do you know of a way to use it with FTP?

davidb said...

@dave - ah, I didn't know that you could use Time Machine with a NAS as well, thanks for the tip. However I guess I'll have to reformat my NAS drive for that to HFS+, which is not really an option for me...

Mazya uchapati (my low-tech efforts) said...

Hi David,

I have a NAS drive which has Time Machine support and it just allowed me to create a new volume out of current partition (without any new partition or reformat) I this what it does is creates a special sparse filesystem for backup. I have it running successfully on my LG N1A1 with Time Machine.

Mazya uchapati (my low-tech efforts) said...

Hi David,

I have a NAS drive which has Time Machine support and it just allowed me to create a new volume out of current partition (without any new partition or reformat) I this what it does is creates a special sparse filesystem for backup. I have it running successfully on my LG N1A1 with Time Machine.