[Techtalk] Different usage betweed df and du

[Mary]

On Wed, Jul 13, 2005, Sue Stones wrote:
> No, although I deleted some files that I wasn't sure what they were,
> after the fact - to make space to save what I was working on.  (they
> were things in /tmp and I think they were download log files or
> something like that).  I assumed that they would not allow me to
> delete them if they were needed, ie I assumed that they would have
> root privileges if they were essential.

While that's probably true, the missing link here is that 'deleting' a
file on UNIX does not always remove that file from the disk. A scenario:

Long running process foo is keeping /tmp/data open for reading. You type
'rm /tmp/foo'. This will remove the directory entry for /tmp/foo so that
it is not visible to ls (or du, or anything else that reads the
directory structure from disk). Intuitively, you'd expect process foo to
immediately blow up because /tmp/data has disappeared. But this is not
so! While process foo keeps that file open, the kernel does not free
the space on disk and allows foo to read for as long as it likes.

This is even true if you create another file with the same name. If you
delete /tmp/data and make a new /tmp/data, process foo will still be
reading from the old one until it closes it.

This is actually rather useful. It allows you to do really quite
fundamental things like upgrading libc, which practically everything
uses, *without rebooting*. Things that are using the existing libc just
keep on using it, new things starting up start using the new one.
(Windows filesystems -- at least FAT -- don't have this feature, which
is part of the reason why it insists you reboot after major upgrades.
Any replaced system libraries are well and truly replaced!)

But it does have some unexpected side-effects:

 1. if you accidently delete something and put it back, long running
    processes with open file handles will be using the *old* one and
    newly started processes will use the *new* one. This can cause
    inconsistencies between the processes that can seem quite
    mysterious. ("I updated /tmp/data, why is process foo not
    noticing?!")

 2. in your situation, it can mean that deleting something doesn't free
    up disk space, because a long running process still has the deleted
    version open. This is Conor's theory. It's quite easily testable if
    you reboot: rebooting will force all processes to stop. If you
    reboot and df and du still disagree, Conor's theory has been
    falsified :)

-Mary