[Techtalk] What is you backup strategy for home use?
conor.daly at cod.utvinternet.com
conor.daly at cod.utvinternet.com
Fri Feb 18 12:44:58 EST 2005
On Wed, Feb 16, 2005 at 10:23:20PM -0500 or so it is rumoured hereabouts,
Angelina Carlton thought:
>
> 1. I have several computers to backup, 3 main ones, 2 running debian
> unstable and 1 running debian woody. I have a windows machine I would
> like to re-deploy and do a fresh install of woody and use this as the target for
> all the other backups.
Likewise.
> 2. the only thing I back up is data, no system files.
I backup system files too. Made a big difference for me when my server
blew a disk. A minimal system install followed by a restore from backup
had me back in action in a few hours. The big thing it saved for me was
all of the configuration I've done over the last few years and forgotten.
> 3. all the target data is in /home on each client. /home is the only
> folder that will be backed up.
I export /home from the server by nfs so there's only one /home to backup.
> 4. At present, my biggest ~/ directory is 3.1 GB I cannot guess
> what size it will be in 1 year or 2, but I expect less than 5GB
> lets say.
As much space as you have available and add 10%!...
> good approach? I don't want to backup from one partition on host1 to
> another partition on host1 as a disc crash could wipe out all my data.
A machine at work backs up nightly to a second internal disk. This is
synced to an external disk weekly. Loss of a disk is not a problem, loss
of the machine means up to a week of data lost. Maybe I should do the
external sync twice a week...
> my feeling is that the likelyhood of both the server and the client
> having their disc crash on the same day is a reasonable risk to take.
> I do keep cdroms at my parents house and will continue to do so in the
> event of theft, flood, or fire.
The external backup is always a good thing. Keeping it up to date is an
even better thing.
> If there is a killer package to do all of this for me I'd love to hear
> about it, and if its available for Debian even better!
I use mondoarchive for the server at work. It produces a bootable CD set
that restores from bare metal. It's pretty versatile, using CDR{W}, DVD,
hard disk, network drives, tape as backup media so it will fit with pretty
much any data size. http://www.mondorescue.org and yes, debian is
supported.
For home, I use a 250Gb hard disk in a pluggable caddy {it's not
hot-pluggable, I have to power down to insert / remove the disk}. It
would be just as reasonable to use a USB / firewire external disk. I use
rsync and cp to keep mirrors of all machines running back in time. The
basic method is this:
o mount the external disk under /mnt/external
o create a directory /mnt/external/current/$HOST for each host on the
network.
o do an 'rsync -a --delete -e ssh -x $HOST:/ /mnt/external/current/$HOST'
for each host.
o umount the external disk, bring in to work and park in desk drawer.
Next time you do the backup {about weekly for me}:
o mount the disk
o move the /mnt/external/1 directory to /mnt/external/2 {subsequent
times...}
o do a 'cp -al /mnt/external/current /mnt/external/1' cp -l creates hard
links instead of copying files.
o do the 'rsync -a ....' thing again.
This time, rsync will remove files that have been deleted and will replace
files that have changed with new versions. Files that have not changed
will remain as links and will therefore use no extra space in your backup.
So, you end up with a backup disk that contains, for eg., 10 sequential
images of your systems in only 1 + {<size of changes> * 9} times the
space. The drawback of this system is that you have to get any one
damaged machine up to the point where it's on the network so you can
restore its image. To that end, you could put a minimal linux on the
backup disk so you can install it in the machine to be recovered, boot
from it and rsync the mirror back to the new disk. There's still the boot
loader to install and you're back in action.
My home backup lives in my work desk drawer..
Here's the script that does mine:
enjoy!
Conor
################## cut here #######################################
#!/bin/bash
##########################################
#
# sync_disks.sh
#
# A program to mirror a number of machines over the
# network. Uses rsync and cp.
#
# Author: Conor Daly <conor.daly at oceanfree.net>
# Date: 18-Feb-2005
# License: GPL <http://www.gnu.org/licenses/licenses.html#GPL>
#
# Description: uses rsync over ssh to mirror a machine
# on the network. Uses 'cp -al' to create linked
# copies for time series backups. rsync will then
# update those files that have changed, using the
# minimum possible disk space.
#
# Requirements: It is assumed that the target disk is not
# mounted. Each host to be backed up must have a copy
# of the backup host's root user's ssh public key in
# its root user's $HOME/.ssh/authorized_keys2 . This
# key may be used with a passphrase but you'll have to
# babysit the backup process if so. If you're not happy
# to keep passphraseless ssh keys around on your machines
# and do not wish to babysit the process, please refer to
# http://linuxmafia.com/faq/Security/ssh-publickey-process.html
# for how to lock down an ssh keypair to a single process.
#
##########################################
# First specify the backup device and mountpoint
DEVICE=/dev/hdd1
MOUNTPOINT=/mnt/dismount
LOG=/var/log/sync_disks.log
# This does the actual mirroring job
sync_host() {
HOST=$1
# We treat some hosts differently
if [ "$HOST" == "valkerie" ]; then
SWITCHES="-x " # Only the / filesystem and no ssh since this is the
USERHOST="/" # backup host
elif [ "$HOST" == "faenor" ]; then
SWITCHES="-x "
# This host didn't have ssh or rsync so we used rsh and tar
# to grab a copy first.
USERHOST="/mnt/dismount/${HOST}"
rsh ${HOST} 'tar -cf - /' | (cd ${USERHOST}; tar -xf -)
elif [ "$HOST" == "hobbiton" ]; then
SWITCHES="-e ssh" # This has multiple filesystems to mirror so no '-x' switch
USERHOST="root@${HOST}.cod.ie:/"
else
SWITCHES="-x -e ssh"
USERHOST="root@${HOST}.cod.ie:/"
fi
# Check if the host is there.
ping -c 1 $HOST >/dev/null
if [ $? -eq 0 ]; then
# do the rsync
echo "syncing $HOST.cod.ie"
rsync -a --delete -v $SWITCHES $USERHOST /mnt/dismount/current/$HOST/ >> $LOG 2>&1
rm -rf /mnt/dismount/current/$HOST/proc >> $LOG 2>&1
if [ "$HOST" == "fafnor" ]; then
# This host has a separate /boot partition to backup too.
rsync -a --delete -v $SWITCHES ${USERHOST}boot/ /mnt/dismount/current/$HOST/boot/ >> $LOG 2>&1
fi
else
echo "$HOST.cod.ie not responding, skipping..."
fi
echo
}
# This is to log output while displaying on stdout.
if [ "$1X" == "X" ]; then
exec $0 logged | tee $LOG
exit $?
fi
echo "--------------------------------"
# See if DEVICE is mounted.
mount | grep $MOUNTPOINT | grep $DEVICE >/dev/null
if [ $? -eq 0 ]; then
echo "$DEVICE mounted at $MOUNTPOINT"
umount $DEVICE
if [ $? -ne 0 ]; then
echo "Failed to unmount $DEVICE"
exit 1
fi
else
echo "$DEVICE not mounted"
fi
# before mounting DEVICE, fsck it.
fsck -a -C $DEVICE
if [ $? -ne 0 ]; then
echo "fsck problem..."
exit 1
fi
# Now mount DEVICE on MOUNTPOINT
echo "Mounting..."
mount /dev/hdd1 /mnt/dismount
if [ $? -ne 0 ]; then
echo "Failed to mount $DEVICE on $MOUNTPOINT"
exit 1
fi
# Set a timestamp for this backup
THISDATE=`date +%Y%m%d`
if [ -f ${MOUNTPOINT}/timestamp ]; then
DATESTAMP=`cat ${MOUNTPOINT}/timestamp`
else
echo $THISDATE > ${MOUNTPOINT}/timestamp
fi
# Now move all directories up one number
if [ "X$DATESTAMP" == "X$THISDATE" ]; then
# but only if not already done today
echo "You did this already today. Not linking..."
else
for num in `seq 100 -1 2`; do
next=$(($num - 1))
# echo "moving ${MOUNTPOINT}/${next} to ${MOUNTPOINT}/${num}"
mv ${MOUNTPOINT}/${next} ${MOUNTPOINT}/${num} 2> /dev/null
done
# Link MOUNTPOINT/current to MOUNTPOINT/1
echo "linking ${MOUNTPOINT}/current to ${MOUNTPOINT}/1"
cp -al ${MOUNTPOINT}/current ${MOUNTPOINT}/1
# and set a timestamp
echo $THISDATE > ${MOUNTPOINT}/timestamp
touch ${MOUNTPOINT}/current
fi
# Now we mirror each host in turn.
echo
echo "Starting"
df /mnt/dismount
df -h /mnt/dismount
date
sync_host fafnor
date
sync_host knoppix
date
sync_host webserver
date
sync_host valkerie
date
sync_host hobbiton
date
df /mnt/dismount
df -h /mnt/dismount
echo "Finished"
umount $DEVICE
exit 0
################### cut here ######################################
--
Conor Daly <conor.daly at oceanfree.net>
Domestic Sysadmin :-)
---------------------
Hobbiton.cod.ie
00:51:24 up 10 days, 12:11, 1 user, load average: 0.00, 0.01, 0.00
More information about the Techtalk
mailing list