Working with large directories

[fred smith]

On Fri, Mar 16, 2012 at 09:39:03PM -0430, Patrick O'Callaghan wrote:
> On Fri, 2012-03-16 at 19:25 -0400, fred smith wrote:
> > On Fri, Mar 16, 2012 at 11:01:03PM +1030, Tim wrote:
> > > On Thu, 2012-03-15 at 23:01 -0700, Robert Moskowitz wrote:
> > > > Nautilus takes forever to list these directories and at times 
> > > > I just want to look for a particular file by string in the name.  Is 
> > > > there a fast graphic tool for this?  Then when I find the desired
> > > > file, I typically open it in Firefox.
> > > 
> > > Nautilus seems to sniff the files to discover their types, and a plug-in
> > > tries to generate a thumbnail image for the file.  Both these features
> > > are painfully slow with moderately largish directories.  You can turn
> > > off the inclusion of the filetype in the list details, but not the image
> > > generation.  Perhaps the plug-in could be manually replaced with
> > > something which worked faster, or instantly "did nothing."
> > > 
> > > I've used the emelFM2 file manager as an alternative, it doesn't show
> > > thumbnails of files.  And it does let you do some wildcarding to
> > > show/hide files in the lister gadget.
> > 
> > I think it's most likely not that nautilus, etc., are slow in 
> > large directories, it is that large directories take a long time
> > to search for files, making any action on those directories much
> > slower than normal. A "problem" of long-standing on pretty much
> > all Unix(-ish) file systems (and I'm not saying it won't happen on
> > other systems, I don't really know, but suspect it does.)
> 
> If that were so, 'ls' would take as long as Nautilus (or Dolphin or
> whatever) to list a large directory. I don't have any huge directories
> to test, but I'm sceptical.

Hmm. you do seem to be correct:

	time ls | wc -l
	105612

	real    0m2.582s
	user    0m2.429s
	sys     0m0.163s

I know that on (much) older systems, large directories were inherently
slow to traverse. I guess I shouldn't assume that is still the case.

-- 
---- Fred Smith -- fredex at fcshome.stoneham.ma.us -----------------------------
                       I can do all things through Christ 
                              who strengthens me.
------------------------------ Philippians 4:13 -------------------------------