Oprofiling Eclipse CVS checkout

[Andrew Overholt]

Hi,

I'm trying to track down the cause of [1], so it was suggested to me that I
run oprofile and see what's going on.  I've done so and attached the
output.  I don't really know what I'm supposed to be seeing in this, so if
someone who knows more could help me out, I'd really appreciate it.

What I did:

1. sudo opcontrol --reset
2. sudo opcontrol --start --no-vmlinux
3. started eclipse and tried to check out GNU-Crypto [2]

   While this was going, I checked on gnu-crypto on the command line (on a
   different machine .. it's the same if I do it on the same machine as the
   one running Eclipse).  The command line checkout started when the
   Eclipse checkout was about 20% or 30% completed and finished before the
   Eclipse checkout made it to 40% (these percentages are just rough
   estimates based upon the progress bars).

4. stopped the Eclipse checkout while it was still less than 50% completed
   and cleanly exited Eclipse
5. sudo opcontrol --stop
6. sudo opreport image-exclude:/no-vmlinux

Thanks,

Andrew

[1]
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=151832

[2]
cvs -d :ext:[e-mail]/cvsroot/gnu-crypto co gnu-crypto

[Andrew Haley]

Andrew Overholt writes:
>  Hi,
>  
>  I'm trying to track down the cause of [1], so it was suggested to me that I
>  run oprofile and see what's going on.  I've done so and attached the
>  output.  I don't really know what I'm supposed to be seeing in this, so if
>  someone who knows more could help me out, I'd really appreciate it.
>  
>  What I did:
>  
>  1. sudo opcontrol --reset
>  2. sudo opcontrol --start --no-vmlinux
>  3. started eclipse and tried to check out GNU-Crypto [2]
>  
>     While this was going, I checked on gnu-crypto on the command line (on a
>     different machine .. it's the same if I do it on the same machine as the
>     one running Eclipse).  The command line checkout started when the
>     Eclipse checkout was about 20% or 30% completed and finished before the
>     Eclipse checkout made it to 40% (these percentages are just rough
>     estimates based upon the progress bars).
>  
>  4. stopped the Eclipse checkout while it was still less than 50% completed
>     and cleanly exited Eclipse
>  5. sudo opcontrol --stop
>  6. sudo opreport image-exclude:/no-vmlinux 
You need the syms.  opreport -l.

But I can tell imediately from the 

>       8530 18.2055 libgcc_s-4.0.0-20050428.so.1 
 
what is wrong.  You're throwning and catching millions of exceptions.

Andrew.

[Andrew Overholt]

* Andrew Haley <aph(a)redhat.com&gt; [2005-05-06 10:54]:
>  You need the syms.  opreport -l. 
Attached.

>  But I can tell imediately from the 
>  
>   >      8530 18.2055 libgcc_s-4.0.0-20050428.so.1
>   
>  what is wrong.  You're throwning and catching millions of exceptions. 
Hmm.  I guess I'll have to start digging to see where we're doing this.
Good times.

Andrew

[Andrew Haley]

Andrew Overholt writes:
>  * Andrew Haley <aph(a)redhat.com&gt; [2005-05-06 10:54]:
>  > You need the syms.  opreport -l.
>  
>  Attached.
>  
>  > But I can tell imediately from the 
>  > 
>  >  >      8530 18.2055 libgcc_s-4.0.0-20050428.so.1
>  >  
>  > what is wrong.  You're throwning and catching millions of exceptions.
>  
>  Hmm.  I guess I'll have to start digging to see where we're doing this.
>  Good times. 
No use, I'm afraid.  Your libgcj has been stripped of debug info.
With a bit of debug info, I could tell you what exceptions they were.

Andrew.

[Ziga Mahkovec]

On Fri, 2005-05-06 at 16:11 +0100, Andrew Haley wrote: 
>  Andrew Overholt writes:
>   > Hmm.  I guess I'll have to start digging to see where we're doing this.
>   > Good times.
>  
>  No use, I'm afraid.  Your libgcj has been stripped of debug info.
>  With a bit of debug info, I could tell you what exceptions they were. 
# yum install kernel-debuginfo gcc-debuginfo
# opcontrol --vmlinux=/usr/lib/debug/lib/modules/2.6.11-1.1240_FC4/vmlinux

This should do, right?
-- 
Ziga

[Andrew Haley]

Ziga Mahkovec writes:
>  On Fri, 2005-05-06 at 16:11 +0100, Andrew Haley wrote: 
>  > Andrew Overholt writes:
>  >  > Hmm.  I guess I'll have to start digging to see where we're doing
> this.
>  >  > Good times.
>  > 
>  > No use, I'm afraid.  Your libgcj has been stripped of debug info.
>  > With a bit of debug info, I could tell you what exceptions they were.
>  
>  # yum install kernel-debuginfo gcc-debuginfo
>  # opcontrol --vmlinux=/usr/lib/debug/lib/modules/2.6.11-1.1240_FC4/vmlinux
>  
>  This should do, right? 
I don't know if oprofile uses debuginfo packages.

Andrew.

[Keith Seitz]

On Fri, 2005-05-06 at 16:48 +0100, Andrew Haley wrote:
>   > This should do, right?
>  
>  I don't know if oprofile uses debuginfo packages. 
It does since 0.8.

Keith

[Robin Green]

On Fri, 06 May 2005 17:38:54 +0200, Ziga Mahkovec wrote:

>  On Fri, 2005-05-06 at 16:11 +0100, Andrew Haley wrote: 
> > Andrew Overholt writes:
> >  > Hmm.  I guess I'll have to start digging to see where we're doing
> this.
> >  > Good times.
> > 
> > No use, I'm afraid.  Your libgcj has been stripped of debug info.
> > With a bit of debug info, I could tell you what exceptions they were.
>  
>  # yum install kernel-debuginfo gcc-debuginfo
>  # opcontrol --vmlinux=/usr/lib/debug/lib/modules/2.6.11-1.1240_FC4/vmlinux
>  
>  This should do, right? 
I suggest you also use oprofile from CVS, which has some fixes for a
symbol resolution bug that I found, which affects resolving symbols in
libgcj.

-- 
Robin

[Andrew Overholt]

* Robin Green <greenrd(a)presidium.org&gt; [2005-05-07 19:04]:
>  On Fri, 06 May 2005 17:38:54 +0200, Ziga Mahkovec wrote:
>  
>  > On Fri, 2005-05-06 at 16:11 +0100, Andrew Haley wrote: 
>  >> Andrew Overholt writes:
>  >>  > Hmm.  I guess I'll have to start digging to see where we're doing
> this.
>  >>  > Good times.
>  >> 
>  >> No use, I'm afraid.  Your libgcj has been stripped of debug info.
>  >> With a bit of debug info, I could tell you what exceptions they were.
>  > 
>  > # yum install kernel-debuginfo gcc-debuginfo
>  > # opcontrol --vmlinux=/usr/lib/debug/lib/modules/2.6.11-1.1240_FC4/vmlinux
>  > 
>  > This should do, right?
>  
>  I suggest you also use oprofile from CVS, which has some fixes for a
>  symbol resolution bug that I found, which affects resolving symbols in
>  libgcj. 
Thanks for everyone's suggestions.  I've been running into the CVS slowdown
issue while preparing our talks for GUADEC so I spent a bit more time
profiling this afternoon.  I've got two reports (from two independent runs
of checking out gnu-crypto) that are pretty similar.  I've attached bzipped
versions of them.

Any help appreciated.

Andrew

[Tom Tromey]

Andrew> Thanks for everyone's suggestions.  I've been running into the
Andrew> CVS slowdown issue while preparing our talks for GUADEC so I
Andrew> spent a bit more time profiling this afternoon.  I've got two
Andrew> reports (from two independent runs of checking out gnu-crypto)
Andrew> that are pretty similar.  I've attached bzipped versions of
Andrew> them.

These look a bit weird, e.g.:

5384      7.8072  libgcj.so.6.0.0          java::lang::Compiler::Compiler()
5358      7.7695  libgcj.so.6.0.0          __do_global_ctors_aux

But Compiler's constructor is private and, afaik, never called.
And __do_global_ctors_aux should barely show up, I think.


Anthony ran into some problem where oprofile was finding the wrong
symbols.  He upgraded and this was fixed.  I don't know what versions
are affected.

Tom

[Andrew Overholt]

* Tom Tromey <tromey(a)redhat.com&gt; [2005-05-18 17:54]:
>  Anthony ran into some problem where oprofile was finding the wrong
>  symbols.  He upgraded and this was fixed.  I don't know what versions
>  are affected. 
I ran CVS head opreport as Anthony suggested (he said opcontrol was fine to
use with the binary in FC4's package).

Andrew

[Anthony Green]

On Wed, 2005-05-18 at 17:57 -0400, Andrew Overholt wrote:
>  * Tom Tromey <tromey(a)redhat.com&gt; [2005-05-18 17:54]:
>  > Anthony ran into some problem where oprofile was finding the wrong
>  > symbols.  He upgraded and this was fixed.  I don't know what versions
>  > are affected.
>  
>  I ran CVS head opreport as Anthony suggested (he said opcontrol was fine to
>  use with the binary in FC4's package). 
Can you triple check that?  When I run opreport from FC4 I get the same
goofy looking report you do for this task (with
java.lang.Compiler.Compiler, etc).   But when I run the opreport I built
from oprofile cvs sources I get a really believable report where
everything appears to make sense.

AG

[Andrew Haley]

Tom Tromey writes:
>  Andrew> Thanks for everyone's suggestions.  I've been
> running into the
>  Andrew> CVS slowdown issue while preparing our talks for GUADEC so I
>  Andrew> spent a bit more time profiling this afternoon.  I've got two
>  Andrew> reports (from two independent runs of checking out gnu-crypto)
>  Andrew> that are pretty similar.  I've attached bzipped versions of
>  Andrew> them.
>  
>  These look a bit weird, e.g.:
>  
>  5384      7.8072  libgcj.so.6.0.0          java::lang::Compiler::Compiler()
>  5358      7.7695  libgcj.so.6.0.0          __do_global_ctors_aux
>  
>  But Compiler's constructor is private and, afaik, never called.
>  And __do_global_ctors_aux should barely show up, I think.
>  
>  
>  Anthony ran into some problem where oprofile was finding the wrong
>  symbols.  He upgraded and this was fixed.  I don't know what versions
>  are affected. 
OK, so I'm not going to investigate this further.

FWIW, my oprofile results have been accurate AFAICS.  Oh, and Will
Cohen is extremely helpful -- we should use him if we have oprofile
problems.

Andrew.

[Anthony Green]

On Thu, 2005-05-19 at 10:58 +0100, Andrew Haley wrote:
>  FWIW, my oprofile results have been accurate AFAICS.  Oh, and Will
>  Cohen is extremely helpful -- we should use him if we have oprofile
>  problems. 
FWIW, I filed this opreport problem in bugzilla a few days ago and it is
assigned to him.

AG

[Andrew Overholt]

* Anthony Green <green(a)redhat.com&gt; [2005-05-19 10:27]:
>  On Wed, 2005-05-18 at 17:57 -0400, Andrew Overholt wrote:
>  > * Tom Tromey <tromey(a)redhat.com&gt; [2005-05-18 17:54]:
>  > > Anthony ran into some problem where oprofile was finding the wrong
>  > > symbols.  He upgraded and this was fixed.  I don't know what versions
>  > > are affected.
>  > 
>  > I ran CVS head opreport as Anthony suggested (he said opcontrol was fine to
>  > use with the binary in FC4's package).
>  
>  Can you triple check that?  When I run opreport from FC4 I get the same
>  goofy looking report you do for this task (with
>  java.lang.Compiler.Compiler, etc).   But when I run the opreport I built
>  from oprofile cvs sources I get a really believable report where
>  everything appears to make sense. 
I don't have time to re-test, but I don't know if I mentioned before that
setting my CVS compression to 3 helped immensely.

Andrew