gorged harddrive

[Todd Denniston]

Ed Greshko wrote, On 03/31/2008 10:57 PM:
> charles f. zeitler wrote:
>> --- Ed Greshko <Ed.Greshko at greshko.com> wrote:
>>
>>> charles f. zeitler wrote:
>>>> i've been pruning my "downloads" disk,
>>>> rather drastically, and not making a dent.
>>>>
>>>> today some more, less drastic but still
>>>> hefty, same result.
>>>>
>>>> revisited du- checked it twice - three
>>>> times- yup, it reports one directory at
>>>> 800+ gb- on a 400gb disk!
>>>>
>>>> fsck (forced) failed to report any problems,
>>>> there don't seem to be any symlinks,
>>>> and the sub-direcory sizes are sane...
>>>>
>>>> any ideas welcome, and appreciated.
>>> Instead of telling people what you are seeing it would be better to 
>>> show the actual commands and output.
>>>
>> good point.
>>
>>
>> [fedora_8 at Nyarlethotep ~]$ df
>> Filesystem           1K-blocks      Used Available Use% Mounted on
>> /dev/sdb1            384578164 330445976  34596748  91% 
>> /home/fedora_8/torrents_isos
>>
>> /dev/sdb1 is the drive under discussion.
>>
>>
>> [fedora_8 at Nyarlethotep ~]$ du -sb t*s/*
>> 34256010522     torrents_isos/backup
>> 883393808812    torrents_isos/data
>> 58352749159     torrents_isos/finished
<snip>
>> somethings wrong with t*s/data  ....
>>
> 
> OK....  I believe I know what the problem is.  The torrents_isos/porn 
> directory makes things seem larger than what they really are....
> 
> No, just kidding.....
> 
> I believe you may have a bunch of non-completed torrent downloads.  When 
> you start a torrent download the client will reserve the space and it 
> will be reflected in the output of "du" but *not* in the output of 
> "df".  Thus with "du" you can have a situation where it "thinks" more 
> disk space is being used than it actually is.  FWIW, this is normal.
> 

I think Ed is on the right track, but the strange thing is the reserve would 
happen sparse...
cd /dev/shm/
mkdir normal sparse
dd if=/dev/zero of=normal/bunchozeros bs=1M count=20
cp --sparse=always normal/bunchozeros sparse/

ls -l normal/* sparse/*
-rw-r--r-- 1 root root 20971520 2008-04-01 09:28 normal/bunchozeros
-rw-r--r-- 1 root root 20971520 2008-04-01 09:30 sparse/bunchozeros

du -sb *
20971580        normal
20971580        sparse

du -s *
20504   normal
0       sparse

df -h .
Filesystem            Size  Used Avail Use% Mounted on
tmpfs                1014M   21M  994M   2% /dev/shm

OK!!!
df is showing how much disk is used/free based on inodes and filesizes (you 
can run out of space if you have no inodes left).
du -s, du -sm show how much disk space is REALLY used on disk by the files.
du -b implies --apparent-size
du --apparent-size "print apparent sizes,  rather  than  disk  usage"


but you are probably looking to figure out WHAT for sure is causing this...
suggestion:
cd /home/fedora_8/torrents_isos/data
BIGONE=`du -sb * |sort -n |tail -1|awk '{print $2}'`
du -sb * |sort -n |tail -1
echo $BIGONE
cd $BIGONE
BIGONE=`du -sb * |sort -n |tail -1|awk '{print $2}'`
du -sb * |sort -n |tail -1
echo $BIGONE
cd $BIGONE
...
until the size seems to get reasonable, now you have probably located the 
file/directory with the unreasonable file/directory.

knowing which file/directory is giving you strange results, you can look back 
(in your mind or command history) to figure out what tried to create it, and 
better yet you might be able to get rid of it, if you desire.
BTW I usually use `df -h` and `du -sm` when tracking these, as the numbers you 
are getting are of the size that the human brain starts thinking 'was that as 
many chars as last time???' instead of 'OK this is a reasonable number.'

-- 
Todd Denniston
Crane Division, Naval Surface Warfare Center (NSWC Crane)
Harnessing the Power of Technology for the Warfighter