[techtalk] RE:backup newbie

[Brian Sweeney]

>Message: 12
>Date: Wed, 29 Nov 2000 18:40:27 -0800 (PST)
>From: Eric Richard Turner <turnere at cc.wwu.edu>
>To: Conor Daly <conor.daly at oceanfree.net>
>cc: techtalk at linuxchix.org
>Subject: Re: [techtalk] RE: backup newbie

>On Wed, 29 Nov 2000, Conor Daly wrote:

> but can I do the following?
>
> 1st of Month:          Full backup
> Sunday:                Differential Backup from Monthly
> Daily:                 Differential Backup from Sunday
>

>Good question. I suspect it depends on the backup utility and/or operating
>system. I know that NTBACKUP on Windows NT uses the archive bit on each
>file. During a full-system backup it clears the archive bit on all files.
>When a file is changed, the archive bit is set. Differential and
>incremental backups save only those files for which the archive bit is
>set. The difference between the two is that during differential backups,
>the archive bit is not cleared whereas during incremental backups the
>archive bit is cleared.
>So, using NTBACKUP on Windows NT, the answer to your question is no.
>However, it seems reasonable that one could write a backup utility that
>would catalogue when each file has been saved, perhaps using a checksum,
>CRC or last-modified date to determine when a file has been modified. Then
>the backup utility could do all sorts of interesting backup scenarios
>based on the combination of the catalogue and the list of modified files.
>Whether or not this sort of thing exists is beyond the scope of my
>experience. I've never had to set up incremental or differential backups
>for Linux, so I'm not sure what can and cannot be done. I recently
>received the latest Linux Journal, which has an article in it about
>automated backups in Linux. Now I'm curious, so I guess I'll just have to
>read it :-)>>>

>Eric R. Turner

I'm not sure that's completely correct; with Linux's dump, for instance, you
specify the level of the backup (0-9) when creating an incremental backup,
so it's not neccesarily what's changed since the day before.  With dump, as
with many archive utilities, , the difference between an incremental and
differential backup is that a differential backs up everything changed since
the last FULL BACKUP, whereas an incremental backs up everything changed
since the most recent lower-number backup. So, a differential would look
like this:

Sun-full backup
Mon-All files changed since sunday
Tue-All files changed since sunday (note: this will be re-backing up some
files that were backed up monday; that's the disadvantage to differentials)
Wed-All files changed since sunday
etc...

A simple incremental would look like this:
Sun-level 0 (full backup)
Mon-level 1 (all files changed since sun)
Tue-level 2 (all files changed since mon)
etc...

The disadvantage here is that if things go boom, you need the last full
backup and all incremental backups since then.  If this were to happen the
day before you did a full backup, that's a lot of tapes!  Now with a
differential, you would only need the last full backup, and the last
differential backup, MUCH easier to deal with. However, differentials are
not very efficient; many files are being  backed up every day. (NOTE: a
differential is really an incremental with one day being a level 0 backup,
and all other days being level 1's)

Now if you've got a large server environment, or someplace where most of
your files change often, you might consider doing a more "clever"
incremental backup system, as follows:

Once a month (sunday)-level 0
Once a week (sunday) - level 1
Daily schedule -
	mon-level 3
	tue-level 2
	wed-level 5
	thu-level 4
	fri-level 7
	sat-level 6

This is called the "towers of hanoi" method, and is, from what I've read,
the most efficient system for doing backups (I believe this is what Conor
was referring to, though I admit I've never implemented it).  In this case,
most files are backed up twice but only twice, so it's a nice compromise
between a differential and a simple incremental.  On the downside, it still
requires you to have more than 2 tape sets in the event of a complete
restore.

On another note, one thing that I think confuses a lot of people is that
different pieces of software use the words "incremental" and "differential"
to mean different things.  I've defined them as I have because I believe
that's how most Linux utilities I'm aware of define them; however, a novell
backup utility I'm running (Arcserve) defines differential the same way, but
incremental as what I've described as a "simple" incremental above.  And
I've heard that some utilities switch the terms around altogether :(.

In my company, we don't have too much to backup (~250G), and we have access
to a 22 tape DLT jukebox, so I opted to go for ease-of-restore over
efficiency-of-backup, and I use a differential system using archive bits
(you can also usually do differentials by date).

I hate to repeat myself, as I've said this in previous posts, but I will
anyway ;-): O'reilly's "Unix Backup & Restore" book is a really good
reference for a lot of this stuff.  I ended up doing a lot of research
online that could've been avoided if I got that book sooner.  Armed with it
and the manual for a backup utility, I'd wager you can conquer any ugly
backup task.  Just my 2 cents, though...

Good luck to all...
-Brian