Disk IO issues

[Mike McGrath]

On Fri, 2 Jan 2009, James Antill wrote:

> On Fri, 2009-01-02 at 11:57 -0600, Mike McGrath wrote:
> > On Fri, 2 Jan 2009, Sascha Thomas Spreitzer wrote:
> >
> > > Hello again,
> > >
> > > this line looks suspicious to me:
> > >
> > > # name            <active_objs> <num_objs> <objsize> <objperslab>
> > > <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> :
> > > slabdata <active_slabs> <num_slabs> <sharedavail>
> > > ext3_inode_cache   98472 150260    760    5    1 : tunables   54   27
> > >   8 : slabdata  30052  30052    189
> > >
> > > Is it 1 big filesystem with about 1,342,177,280 inodes. Has this
> > > amount ever be tested in the wild?
> >
> > Not sure if it has been tested in the wild or not but the filesystem
> > itself contains a _TON_ of hardlinks.  Creation of hardlinks is one of the
> > big purposes of this filesystem.
>
>  Ah ha ... I bet that you'll find tar/cp-a/whatever is having a major
> problem keeping tabs on which inodes it's "seen", so it doesn't copy the
> same data N times. Try running: cp -a --no-preserve=links ... and see if
> that is much faster?
>

Naw, I've been testing on the non-link portions.  Dennis, Jesse, etc,
correct me if I'm wrong on this:

We've got a dir /mnt/koji/packages/ that contains all of the packages.
You can actually view this dir yourself at:

http://kojipkgs.fedoraproject.org/packages/glibc/

There are other directories at /mnt/koji/static-repos/.  A directory like
static-repos contains almost exclusively hardlinks to those packages.

Since many of those hardlink oriented directories can be recreated, we
don't bother backing them up so I haven't been testing with them.

One thing I'm going to try to do is re-index the filesystem (e2fsck -D).
I figure its a worthwhile thing to do.  I'm testing on a snapshot first.

	-Mike