yum very slow

[Richard Hally]

Yum seems to be very slow in resolving dependencies. I'm sitting here 
watching my rawhide test box take over an hour to do the dep resolution 
for todays updates from rawhide. This is on a box that has a 2.5ghz P4 
and nothing else running on the machine. this has been occurring for at 
least a few weeks(or longer). What should I do to get useful information 
concerning this problem?

Thanks
Richard Hally

[seth vidal]

On Wed, 2005-05-11 at 02:40 -0400, Richard Hally wrote:
>  Yum seems to be very slow in resolving dependencies. I'm sitting
> here 
>  watching my rawhide test box take over an hour to do the dep resolution 
>  for todays updates from rawhide. This is on a box that has a 2.5ghz P4 
>  and nothing else running on the machine. this has been occurring for at 
>  least a few weeks(or longer). What should I do to get useful information 
>  concerning this problem?
>   
are you sure it's the dep resolution that is slow or is it slowing down
when it does the header download DURING the dep resolution. I'm making
that distinction b/c one is a function of code and processor and the
other is a function of mirror availability and bandwidth.

-sv

[Richard Hally]

seth vidal wrote:

> On Wed, 2005-05-11 at 02:40 -0400, Richard Hally wrote:
>   
>
> >Yum seems to be very slow in resolving dependencies. I'm sitting here 
> >watching my rawhide test box take over an hour to do the dep resolution 
> >for todays updates from rawhide. This is on a box that has a 2.5ghz P4 
> >and nothing else running on the machine. this has been occurring for at 
> >least a few weeks(or longer). What should I do to get useful information 
> >concerning this problem?
> >
> >    
> >
>
> are you sure it's the dep resolution that is slow or is it slowing down
> when it does the header download DURING the dep resolution. I'm making
> that distinction b/c one is a function of code and processor and the
> other is a function of mirror availability and bandwidth.
>
> -sv
>
>
>   
>  
here is what it says:
Excluding Packages in global exclude list
Finished
(wait an hour + )
Resolving Dependencies
--> Populating transaction set with selected packages..

During the wait, there is continuous disk activity.  and no network 
activity.
This has been going on for a while (it didn't just start today)
I did a yum clean all to see if it would help but it did not.

[Yuan Yijun]

I always set vorbose message level to 10 instead of default value 3 :)

2005/5/11, Richard Hally <rhally(a)mindspring.com&gt;:
>  seth vidal wrote:
>  
>  >On Wed, 2005-05-11 at 02:40 -0400, Richard Hally wrote:
>  >
>  >
>  >>Yum seems to be very slow in resolving dependencies. I'm sitting here
>  >>watching my rawhide test box take over an hour to do the dep resolution
>  >>for todays updates from rawhide. This is on a box that has a 2.5ghz P4
>  >>and nothing else running on the machine. this has been occurring for at
>  >>least a few weeks(or longer). What should I do to get useful information
>  >>concerning this problem?
>  >>
>  >>
>  >>
>  >
>  >are you sure it's the dep resolution that is slow or is it slowing down
>  >when it does the header download DURING the dep resolution. I'm making
>  >that distinction b/c one is a function of code and processor and the
>  >other is a function of mirror availability and bandwidth.
>  >
>  >-sv
>  >
>  >
>  >
>  >
>  here is what it says:
>  Excluding Packages in global exclude list
>  Finished
>  (wait an hour + )
>  Resolving Dependencies
>  --> Populating transaction set with selected packages..
>  
>  During the wait, there is continuous disk activity.  and no network
>  activity.
>  This has been going on for a while (it didn't just start today)
>  I did a yum clean all to see if it would help but it did not.
>  
>  
>  --
>  fedora-devel-list mailing list
>  fedora-devel-list(a)redhat.com
>  http://www.redhat.com/mailman/listinfo/fedora-devel-list
>

[Jef Spaleta]

On 5/11/05, Richard Hally <rhally(a)mindspring.com&gt; wrote:
>  Excluding Packages in global exclude list
>  Finished
>  (wait an hour + )
>  Resolving Dependencies
>  --> Populating transaction set with selected packages..
>  
>  During the wait, there is continuous disk activity.  and no network
>  activity.
>  This has been going on for a while (it didn't just start today)
>  I did a yum clean all to see if it would help but it did not. 
that's fascinatingly different than what I'm seeing. How much memory
do you have.. and how much memory is the yum process using during that
(wait an hour+)  I'm definitely not experiencing this on my rawhide
box.

-jef

[Warren Togami]

Jeff Spaleta wrote:
>  On 5/11/05, Richard Hally <rhally(a)mindspring.com&gt; wrote:
>  
> >Excluding Packages in global exclude list
> >Finished
> >(wait an hour + )
> >Resolving Dependencies
> >--> Populating transaction set with selected packages..
> >
> >During the wait, there is continuous disk activity.  and no network
> >activity.
> >This has been going on for a while (it didn't just start today)
> >I did a yum clean all to see if it would help but it did not.
>  
>  
>  that's fascinatingly different than what I'm seeing. How much memory
>  do you have.. and how much memory is the yum process using during that
>  (wait an hour+)  I'm definitely not experiencing this on my rawhide
>  box.
>   
developmen: ################################################## 3770/3770
Added 135 new packages, deleted 137 old in 8.43 seconds
(wait about 2 minutes with 100% CPU and some disk activity)
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for kernel-devel to pack into transaction set.

Athlon64 3200+ x86_64 1GB RAM

Not quite an hour, but that's a long time to get stuck with no visible 
indication of what's happening.

Warren Togami
wtogami(a)redhat.com

[seth vidal]

>  > 
>  
>  developmen: ################################################## 3770/3770
>  Added 135 new packages, deleted 137 old in 8.43 seconds
>  (wait about 2 minutes with 100% CPU and some disk activity)
>  Resolving Dependencies
>  --> Populating transaction set with selected packages. Please wait.
>  ---> Downloading header for kernel-devel to pack into transaction set.
>  
>  Athlon64 3200+ x86_64 1GB RAM
>  
>  Not quite an hour, but that's a long time to get stuck with no visible 
>  indication of what's happening.
>   
we've done this before on your system, warren, disable ipv6 support and
see if it miraculously speeds up.

-sv

[Warren Togami]

seth vidal wrote:
> >developmen: ##################################################
> 3770/3770
> >Added 135 new packages, deleted 137 old in 8.43 seconds
> >(wait about 2 minutes with 100% CPU and some disk activity)
> >Resolving Dependencies
> >--> Populating transaction set with selected packages. Please wait.
> >---> Downloading header for kernel-devel to pack into transaction set.
> >
> >Athlon64 3200+ x86_64 1GB RAM
> >
> >Not quite an hour, but that's a long time to get stuck with no visible 
> >indication of what's happening.
> >
>  
>  
>  we've done this before on your system, warren, disable ipv6 support and
>  see if it miraculously speeds up.
>   
It doesn't, this is a different issue.  That point where it stops with 
100% CPU usage and disk usage for 2 minutes is apparently the only slow 
point now, everything else is very fast.

Warren Togami
wtogami(a)redhat.com

[Richard Hally]

Jeff Spaleta wrote:

> On 5/11/05, Richard Hally <rhally(a)mindspring.com&gt; wrote:
>   
>
> >Excluding Packages in global exclude list
> >Finished
> >(wait an hour + )
> >Resolving Dependencies
> >--> Populating transaction set with selected packages..
> >
> >During the wait, there is continuous disk activity.  and no network
> >activity.
> >This has been going on for a while (it didn't just start today)
> >I did a yum clean all to see if it would help but it did not.
> >    
> >
>
> that's fascinatingly different than what I'm seeing. How much memory
> do you have.. and how much memory is the yum process using during that
> (wait an hour+)  I'm definitely not experiencing this on my rawhide
> box.
>
> -jef
>
>   
>  
Below is some of the output with -d 10 option specified. I may have been 
mistaken when I originally said the wait was while resolving 
dependencies. The long wait occurs immediately *before* the "Resolving 
Dependencies" is printed.  Once the "Resolving Dependencies" is printed, 
the header downloads and the rest of the update continue normally.

--------------------
Excluding dlm-debuginfo - 1.0-0.pre21.8.i386
Excluding dvgrab-debuginfo - 1.7-3.i386
Finished
Excluding Incompatible Archs
Finished
Reading Local RPMDB
Building updates object
putting mozilla-dom-inspector in simple update
putting procps in simple update
putting elfutils in simple update
putting glibc in complex update
putting mod_auth_kerb in simple update
putting sysreport in simple update
putting libwnck in simple update
putting gtk2-devel in simple update
putting php-pear in simple update
putting mozilla-nspr in simple update
putting system-config-printer-gui in simple update
putting php-odbc in simple update
putting postgresql-docs in simple update
putting statserial in simple update
putting php-pgsql in simple update
putting system-config-printer in simple update
putting postgresql-tcl in simple update
putting openssl in complex update
putting elfutils-libelf-devel in simple update
putting postgresql in simple update
putting postgresql-contrib in simple update
putting mozilla in simple update
putting postgresql-devel in simple update
putting vlock in simple update
putting gtk2 in simple update
putting e2fsprogs in simple update
putting bash in simple update
putting mozilla-mail in simple update
putting ORBit2 in simple update
putting fedora-release in simple update
putting php-ncurses in simple update
putting mozilla-js-debugger in simple update
putting lsof in simple update
putting php in simple update
putting procinfo in simple update
putting php-xmlrpc in simple update
putting ORBit2-devel in simple update
putting postgresql-test in simple update
putting mozilla-nspr-devel in simple update
putting php-imap in simple update
putting kernel in complex update
putting php-mysql in simple update
putting elinks in simple update
putting mozilla-devel in simple update
putting postgresql-pl in simple update
putting kernel-doc in simple update
putting php-mbstring in simple update
putting mozilla-nss-devel in simple update
putting system-config-bind in simple update
putting mozilla-nss in simple update
putting php-snmp in simple update
putting libwnck-devel in simple update
putting e2fsprogs-devel in simple update
putting php-gd in simple update
putting postgresql-jdbc in simple update
putting postgresql-server in simple update
putting elfutils-devel in simple update
putting mozilla-chat in simple update
putting kernel in complex update
putting php-devel in simple update
putting postgresql-libs in simple update
putting php-xml in simple update
putting php-ldap in simple update
putting elfutils-libelf in simple update
putting postgresql-python in simple update
processing glibc.i686
processing openssl
processing kernel.i686
processing kernel.i686


(waiting here...)
(then)

Resolving Dependencies

(then the rest of the update)

while the "wait" is happening, here is what top looks like

top - 15:56:04 up  3:23,  3 users,  load average: 2.06, 1.97, 1.24
Tasks: 105 total,   2 running, 103 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.7% us,  2.7% sy,  0.0% ni,  0.0% id, 94.0% wa,  0.7% hi,  0.0% si
Mem:    223316k total,   220440k used,     2876k free,      308k buffers
Swap:   458744k total,      860k used,   457884k free,    61788k cached

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3979 root      18   0 32876  23m 6744 D  2.7 10.6   0:23.48 yum
2915 root      15   0 56632  18m 5448 S  1.3  8.3   2:51.02 X
3154 richard   15   0 38200  13m 8624 S  0.7  6.2   0:04.94 gnome-terminal
4118 root      16   0  2028  988  776 R  0.7  0.4   0:00.17 top
3097 richard   16   0 21820 9428 7332 S  0.3  4.2   0:00.52 wnck-applet
   1 root      16   0  1748  560  480 S  0.0  0.3   0:01.85 init
   2 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0
   3 root      10  -5     0    0    0 S  0.0  0.0   0:00.22 events/0
   4 root      12  -5     0    0    0 S  0.0  0.0   0:00.02 khelper
   5 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
   7 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid
  86 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/0
  89 root      15   0     0    0    0 S  0.0  0.0   0:00.00 khubd
 135 root      15   0     0    0    0 S  0.0  0.0   0:00.00 pdflush
 136 root      15   0     0    0    0 S  0.0  0.0   0:00.03 pdflush
 138 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0
 137 root      15   0     0    0    0 D  0.0  0.0   0:00.05 kswapd0

[Richard Hally]

sorry for the reply to myself.
for the record, Problem solved:  see below.
Richard Hally wrote:

>  Jeff Spaleta wrote:
>
> > On 5/11/05, Richard Hally <rhally(a)mindspring.com&gt; wrote:
> >  
> >
> >> Excluding Packages in global exclude list
> >> Finished
> >> (wait an hour + )
> >> Resolving Dependencies
> >> --> Populating transaction set with selected packages..
> >>
> >> During the wait, there is continuous disk activity.  and no network
> >> activity.
> >> This has been going on for a while (it didn't just start today)
> >> I did a yum clean all to see if it would help but it did not.
> >>   
> >
> >
> > that's fascinatingly different than what I'm seeing. How much memory
> > do you have.. and how much memory is the yum process using during that
> > (wait an hour+)  I'm definitely not experiencing this on my rawhide
> > box.
> >
> > -jef
> >
> >  
> >
>  Below is some of the output with -d 10 option specified. I may have 
>  been mistaken when I originally said the wait was while resolving 
>  dependencies. The long wait occurs immediately *before* the "Resolving 
>  Dependencies" is printed.  Once the "Resolving Dependencies" is 
>  printed, the header downloads and the rest of the update continue 
>  normally.
>  
<snip>

after removing primary.xml.gz, primary.xml.gz.sqlite and repomd.xml from 
/var/cache/yum/development the problem has been solved. It looks like 
the new  primary.xml.gz.sqlite file that was created is substantially 
small (6.6mb vs 7.2mb) while the other recreated files are exactly the 
same size.

thanks for all the help
Richard Hally

[Jef Spaleta]

On 5/12/05, Richard Hally <rhally(a)mindspring.com&gt; wrote:
>  after removing primary.xml.gz, primary.xml.gz.sqlite and repomd.xml
> from
>  /var/cache/yum/development the problem has been solved. It looks like
>  the new  primary.xml.gz.sqlite file that was created is substantially
>  small (6.6mb vs 7.2mb) while the other recreated files are exactly the
>  same size. 
Ah, man .... you didn't keep the old version of that
primary.xml.gz.sqlite file did you?
Now that you have narrowed down the problem.. it would have been
interesting to poke at that file to see what was going wrong.

-jef

[Richard Hally]

Jeff Spaleta wrote:

> On 5/12/05, Richard Hally <rhally(a)mindspring.com&gt; wrote:
>   
>
> >after removing primary.xml.gz, primary.xml.gz.sqlite and repomd.xml from
> >/var/cache/yum/development the problem has been solved. It looks like
> >the new  primary.xml.gz.sqlite file that was created is substantially
> >small (6.6mb vs 7.2mb) while the other recreated files are exactly the
> >same size.
> >    
> >
>
> Ah, man .... you didn't keep the old version of that
> primary.xml.gz.sqlite file did you?
> Now that you have narrowed down the problem.. it would have been
> interesting to poke at that file to see what was going wrong.
>
> -jef
>
>   
>  
yup,  I kept it   = 8-)
it is 7218176 bytes.

Richard