[comp.unix.questions] Where has all my disk space gone?

peter@aucs.AcadiaU.ca (Peter Steele) (04/24/91)

Could someone please explain exactly what is taking up all our disk
space. We have these advertised 1.2 Gig drives on our Unix box (a CDC/MIPS
creation, basically System V). However, we don't seem to be getting as
much capacity out of these drives as we'd like. For example, on one of
the drives, df reports the following:

Filesystem                 Type  kbytes     use   avail %use  Mounted on
/dev/dsk/ipc1d2s3           ffs  945670  589136  356534  62%  /u2

According to this, we have 589136 kbytes in use on this drive. However,
du does not seem to agree with df:

828158  /u2/oracle/srls
4122    /u2/oracle/srld
1       /u2/oracle/adms
161980  /u2/oracle/admd
5       /u2/oracle/temparch
994267  /u2/oracle
994268  /u2

The total of 994268 blocks in use converts to 497134 kbytes, a difference
of 92002 kbytes--not a small amount. The value reported by du appears to
be the value that is limiting us. We need to increase the size of our data
base by 400 megs but we cannot, even though we feel the space should be
available, knowing how much we really do have in use simply by adding up
the sizes of the individual files, as follows:

$:/dir /u2
total 1
drwx------   7 oracle   other        512 Nov  9 10:57 oracle

$:/dir /u2/oracle               -- this is the only directory in /u2
total 9
drwx------   2 oracle   other        512 Jan 17 11:18 admd
drwx------   2 oracle   other        512 Aug 23  1990 adms
drwx------   2 oracle   other        512 Nov 20 09:43 srld
drwx------   2 oracle   other        512 Oct 29 14:04 srls
drwx------   2 root     other       2560 Nov  9 14:23 temparch

$:/dir /u2/oracle/admd
total 162306
-rw-------   1 oracle   dba      40968192 Apr 23 08:57 admd.001
-rw-------   1 oracle   dba      40968192 Apr 23 06:35 admp.001
-rw-------   1 oracle   dba       512512 Apr 23 08:57 log2admd.dbf

$:/dir /u2/oracle/adms
total 0

$:/dir /u2/oracle/srld
total 4130
-rw-------   1 oracle   dba      2097664 Apr 23 07:57 log2srld.dbf

$:/dir /u2/oracle/srls
total 829816
-rw-------   1 oracle   dba      2097664 Apr 23 04:00 log2srls.dbf
-rw-------   1 oracle   dba      209723392 Apr 23 11:37 srls.001
-rw-------   1 oracle   dba      209723392 Apr 23 11:30 srls.002

$:/dir /u2/oracle/temparch
total 0

If I add up the values reported by each of these "dir" commands via
the "total xxx", I get 996262 blocks or 498131 kbytes, assuming my
arithmetic is correct. This is not the same value reported by du nor
by df.

Finally, if I add up the sizes of the individual files reported above,
the value is 494235 kbytes. So, we end up with 4 different values
for the amount of space in use on /u2:

    df -       589136 kbytes
    du -       497134
    dir, total 498131
    dir, files 494235

Which is correct? I know I am missing something vital, but I'd really
like to know where the 90 megs or so extra reported by df is being used.
Perhaps the space is being taken up by hidden directory entries, but
90 megs seems like a lot of directory entries!

Would anyone care to shed some light on this matter? Thanks in advance
for your comments.

-- 
Peter Steele   Postmaster   peter@auad.acadiau.ca   Tel: 902-542-2201
Software Analyst, Acadia University, Wolfville, NS  Fax: 902-542-7224

barmar@think.com (Barry Margolin) (04/24/91)

In article <1991Apr23.174228.11278@aucs.AcadiaU.ca> peter@aucs.acadiau.ca (Peter Steele) writes:
>df reports the following:
>
>Filesystem                 Type  kbytes     use   avail %use  Mounted on
>/dev/dsk/ipc1d2s3           ffs  945670  589136  356534  62%  /u2
>
>According to this, we have 589136 kbytes in use on this drive. However,
>du does not seem to agree with df:
>994268  /u2
>
>The total of 994268 blocks in use converts to 497134 kbytes, a difference
>of 92002 kbytes--not a small amount.

If a process opens a file and then deletes it, the file still exists on
disk until all the processes that have it open either close it or die.
This is probably the most common cause of missing disk space.  It can
happen if someone deletes a log file, not realizing that there's still a
process writing to it; the log will continue to grow, and there will be no
way to stop it.

It's also possible that a file system error resulted in orphan files.

The easiest way to recover that space in either case is to shut down and
fsck.  In the first case, a less drastic measure requires finding the
process(es) that have the deleted file(s) open and kill them.

--
Barry Margolin, Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar

subbarao@phoenix.Princeton.EDU (Kartik Subbarao) (04/24/91)

In article <1991Apr23.194511.29646@Think.COM> barmar@think.com (Barry Margolin) writes:
>In article <1991Apr23.174228.11278@aucs.AcadiaU.ca> peter@aucs.acadiau.ca (Peter Steele) writes:
>>df reports the following:
>>
>>Filesystem                 Type  kbytes     use   avail %use  Mounted on
>>/dev/dsk/ipc1d2s3           ffs  945670  589136  356534  62%  /u2
>>
>>According to this, we have 589136 kbytes in use on this drive. However,
>>du does not seem to agree with df:
>>994268  /u2
>>
>>The total of 994268 blocks in use converts to 497134 kbytes, a difference
>>of 92002 kbytes--not a small amount.
>If a process opens a file and then deletes it, the file still exists on
>disk until all the processes that have it open either close it or die.
>This is probably the most common cause of missing disk space.  It can
>happen if someone deletes a log file, not realizing that there's still a
>process writing to it; the log will continue to grow, and there will be no
>way to stop it.

Yes, but also note that du reports in terms of Kbytes, not blocks, so there
is no 400 meg discrepancy. Note that directories take up space on the disk,
as does creating the file system itself.


		-Kartik


--
internet# rm `df | tail +2 | awk '{ printf "%s/quotas\n",$6}'`

subbarao@phoenix.Princeton.EDU -| Internet
kartik@silvertone.Princeton.EDU (NeXT mail)  
SUBBARAO@PUCC.BITNET			          - Bitnet

barmar@think.com (Barry Margolin) (04/25/91)

In article <azLEBNf8XKcms@idunno.Princeton.EDU> subbarao@phoenix (Kartik Subbarao) writes:
>In article <1991Apr23.194511.29646@Think.COM> barmar@think.com (Barry Margolin) writes:
>>In article <1991Apr23.174228.11278@aucs.AcadiaU.ca> peter@aucs.acadiau.ca (Peter Steele) writes:
>>>df reports the following:
>>>
>>>Filesystem                 Type  kbytes     use   avail %use  Mounted on
>>>/dev/dsk/ipc1d2s3           ffs  945670  589136  356534  62%  /u2
>>>du does not seem to agree with df:
>>>994268  /u2
>Yes, but also note that du reports in terms of Kbytes, not blocks, so there
>is no 400 meg discrepancy. 

I think you're wrong.  If du's number is Kbytes, then it claims that /u2 is
taking up 50MB *more* than the total size of the file system, even though
df claims that 356MB is still available.  Du and df are almost consistent
if du's output is in in 512Kb blocks.

>			    Note that directories take up space on the disk,
>as does creating the file system itself.

Du includes directories in its totals.  File system overhead is only a few
blocks, so shouldn't be significant to this problem.  I'm not sure, but I
think the total size reported by df may be *after* the overhead is taken
into account.
--
Barry Margolin, Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar