On 03/13/2018 04:59 PM, Rick Stevens wrote:
On 03/13/2018 02:21 PM, Stephen Morris wrote:
On 13/3/18 8:46 pm, Patrick O'Callaghan wrote:
On Mon, 2018-03-12 at 18:25 -0700, Rick Stevens wrote:
On 03/12/2018 03:37 PM, Patrick O'Callaghan wrote:
On Tue, 2018-03-13 at 07:26 +1100, Stephen Morris wrote:
> 'du' with no parameters recursively lists all the subdirectories and > their sizes, along with the grand total. When applied to my home > directory, I get over 30,000 lines of output. That's almost never > what > I want. My usual call is 'du -hs'. > > poc Thanks Patrick, taking this a step further, it seems to me that the only parameter for du that, to me, provides the correct file size is -b as shown below.
I think you have a misconception here. 'du' does not give file sizes, it gives disk usage. A 1-byte file takes up at least 1 disk block, so that's the size 'du' will give. I seem to remember that it also counts indirect blocks and other housekeeping that corresponds to the file without being included in the file's content, but I could be mistaken (though I'm fairly sure early versions did do that).
du (with no flags) gives disk usage. As Patrick says, a 1-byte file uses one disk block (which is generally 4KiB) and that's what du is reporting (after all, "du" means "disk usage"). The "-b" flag means "set the block size to 1 byte and show the apparent size", which is what "ls -l" would report (there may be differences between du and ls if sparse files are involved).
Good point about sparse files, which I'd forgotten in this context. A sparse file can be 10TB in size yet occupy only 1 disk block. 'du' will give the usage as 1 block and 'ls' will say it's 10TB.
Also, du walks down the entire current directory unless you give it arguments to tell it what to look at. Note that the arguments you pass it are interpreted by the _shell_, not "du" (even the man page says "PATTERN is a shell pattern (not a regular expression)").
This is a common confusion point with many people. Unless you enclose shell metacharacters in quotes (and dots and splats are metacharacters), the shell WILL interpret them--sometimes in ways you aren't expecting! By default, shell globbing does NOT expand filenames that start with a dot (to the shell, a dot means "current directory").
Slight nit: '.' (on its own) means 'current directory' to the kernel, not to the Shell, i.e. it's wired into the system and doesn't depend on the command interpreter (in the same way that '/' is the path separator and cannot be changed). The use of dot-files as a way of hiding certain names is IIRC originally just a convention of 'ls' which the Shell adopted for consistency.
poc
Thanks guys, I fully understood the -b parameter was giving an apparent size of the file (which approximately matches what ls -l provides), that sparse files can impact the apparent size, and that the physical size of a file is governed by the sector size, but my usage of du, especially with the -s parameter, is the amount of "apparent" space occupied by files and directories (where possible I usually use a sector size of 512) to identify space hogs. I'm also fully aware of the sector size trade off between efficient space utilization and efficient I/O and the apparent space used by files.
What I hadn't realized until Patrick mentioned it, was the significance of the * in the path specification. I hadn't realized that with using the * not only did it cause files prefixed with a '.' to be ignored but it also causes directories prefixed with a '.' to be ignored also.
No. What you have to get straight is that when you use "*", you're asking the shell to create a list of files and return it, and by default the shell doesn't list files starting with a dot.
To clarify, assume you have a directory containing the files "file1", "file2", "file3" and ".dotfile". If you "cd" to it and do an "ls *" (asking the shell to expand the glob), you'll get a list of "file1", "file2" and "file3". The same is true if the command was "du" and not "ls". So the command:
du -a *
is exactly equivalent to:
du -a file1 file2 file3
If you simply do:
du -a
then du will walk down the _directory_ you're in and display all of the files, including ".dotfile", since du does not ignore files starting with a dot.
Here's an example:
[root@golem4 ~]# mkdir /tmp/testdir [root@golem4 ~]# cd /tmp/testdir [root@golem4 testdir]# touch file1 file2 file3 .dotfile [root@golem4 testdir]# echo "Without shopt:";ls *;echo "With shopt:";shopt -s dotglob;ls *;echo "Without shopt again:";shopt -u dotglob;ls * Without shopt: file1 file2 file3 With shopt: .dotfile file1 file2 file3 Without shopt again: file1 file2 file3 [root@golem4 testdir]# du * 0 file1 0 file2 0 file3 [root@golem4 testdir]# du -a * 0 file1 0 file2 0 file3 [root@golem4 testdir]# du -a 0 ./file2 0 ./file3 0 ./.dotfile 0 ./file1 4 .
Hope that clarifies it a bit.
And when I say "files that start with a dot", that includes directories, devices, whatever (remember, essentially in Unixish systems, EVERYTHING is some type of a file). ---------------------------------------------------------------------- - Rick Stevens, Systems Engineer, AllDigital ricks@alldigital.com - - AIM/Skype: therps2 ICQ: 22643734 Yahoo: origrps2 - - - - To err is human, to moo bovine. - ----------------------------------------------------------------------