yum annoyances

[Gene C.]

I call the following annoyances rather than bugs because yum is working as 
designed.  What I really wondering is if an enhancement to yum is 
appropriate.  While it may be appropriate, these cases may also be in the 
"too hard" or "not worth the effort" category.

I will also caveat this by saying that I have only seen these cases working 
with the development tree and not with "production" version.  I will also say 
that a "fresh install" will not have the problem but, even in development, I 
prefer updating rather than installing unless I am testing the installer.

Case 1: A package is install but, now, has been deleted from the 
development/update tree and the installed package blocks updates.  The 
current answer is to manually remove the package (yum remove or rpm -e) if, 
and only if, you know that package was deleted .. the recent case of dbus-qt.

Would it be possible to have some metadata which would flag to yum that the 
package needs to be removed.

Case 2:  This is similar to case 1.  In this case, an older version of a 
package is in the development/update tree,  The currently installed packages 
include one or more packages which are a "more recent version" but one that 
has problems ... therefore, we need to "roll back" the package.  Again, this 
can be handled manually via yum remove/install or using rpm --oldpackage if 
you know that the situation exists.
-- 
Gene

[Ralf Ertzinger]

On Tue, Dec 06, 2005 at 10:58:38AM -0500, Gene C. wrote:

>  Would it be possible to have some metadata which would flag to yum
> that the 
>  package needs to be removed. 
yum does not know if you want the package removed or not. But it could
indicate that none of the repos used for this resolver run provides
this package or a package which obsoletes it.

>  Case 2:  This is similar to case 1.  In this case, an older version
> of a 
>  package is in the development/update tree,  The currently installed packages 
>  include one or more packages which are a "more recent version" but one that 
>  has problems ... therefore, we need to "roll back" the package.  Again, this 
>  can be handled manually via yum remove/install or using rpm --oldpackage if 
>  you know that the situation exists. 
Multiple versions of one package in a repo is not the usual case.

[seth vidal]

On Tue, 2005-12-06 at 17:09 +0100, Ralf Ertzinger wrote:
>  On Tue, Dec 06, 2005 at 10:58:38AM -0500, Gene C. wrote:
>  
>  > Would it be possible to have some metadata which would flag to yum that the 
>  > package needs to be removed.
>  
>  yum does not know if you want the package removed or not. But it could
>  indicate that none of the repos used for this resolver run provides
>  this package or a package which obsoletes it. 

yum list extras.

-sv

[Ralf Ertzinger]

On Tue, Dec 06, 2005 at 11:12:48AM -0500, seth vidal wrote:

>  > yum does not know if you want the package removed or not. But it
> could
>  > indicate that none of the repos used for this resolver run provides
>  > this package or a package which obsoletes it.
>  
>  
>  yum list extras. 
Not quite. Let me reword the above:
But it could indicate that none of the repos used for this resolver run provides
this package, a package which upgrades it or a package which obsoletes it.

[Brian Gerst]

seth vidal wrote:
>  On Tue, 2005-12-06 at 17:09 +0100, Ralf Ertzinger wrote:
>  
> >On Tue, Dec 06, 2005 at 10:58:38AM -0500, Gene C. wrote:
> >
> >
> >>Would it be possible to have some metadata which would flag to yum that the 
> >>package needs to be removed.
> >
> >yum does not know if you want the package removed or not. But it could
> >indicate that none of the repos used for this resolver run provides
> >this package or a package which obsoletes it.
>  
>  
>  
>  yum list extras. 
The problem with list extras is that it doesn't differentiate when a 
package is extra because a newer version exists or if the package 
doesn't exist in the repo.

--
				Brian Gerst

[Hans de Goede]

seth vidal wrote:
>  On Tue, 2005-12-06 at 17:09 +0100, Ralf Ertzinger wrote:
> > On Tue, Dec 06, 2005 at 10:58:38AM -0500, Gene C. wrote:
> >
> >> Would it be possible to have some metadata which would flag to yum that the 
> >> package needs to be removed.
> > yum does not know if you want the package removed or not. But it could
> > indicate that none of the repos used for this resolver run provides
> > this package or a package which obsoletes it.
>  
>  
>  yum list extras.
>  
>  -sv
>  
>   
I think that what Gene / Ralf mean/want (at least this is what I would 
like) is that if:
-this is a default yum upgrade run (iow no --enablerepo or disable)
-none of the repos used for this resolver run provides this package
-this package causes a conflict, for example has dependcies which were
  met before the update but not after if we would continue.
That then and only then, (normal upgrade, troubel package not in any
repo) yum would suggest/ask the user to remove this package. This 
question should ignore -y, and be skipped when run non interactivly!


Maybe this can be done in a plugin?

Regards,

Hans


p.s.
I think we can come up with more cases where it would be nice if yum 
would suggest a fix. Note though that the current default behavior of 
bombing out of anything is wrong is excellent, yum should _never_ try to 
fix things automatically like smartpm tries to.

[seth vidal]

>  Case 1: A package is install but, now, has been deleted from the 
>  development/update tree and the installed package blocks updates.  The 
>  current answer is to manually remove the package (yum remove or rpm -e) if, 
>  and only if, you know that package was deleted .. the recent case of dbus-qt. 
>  Would it be possible to have some metadata which would flag to yum
> that the 
>  package needs to be removed. 
We have that, it is called an obsolete. However, if nothing else
obsoletes the package there is nothing we could or should do. To have an
arbitrary 'remove this' element in the metadata would just be
explosively dangerous. I can see the typo of glibc from glib. It would
be like a million machines cried out and then were instantly silenced.



>  Case 2:  This is similar to case 1.  In this case, an older version
> of a 
>  package is in the development/update tree,  The currently installed packages 
>  include one or more packages which are a "more recent version" but one that 
>  has problems ... therefore, we need to "roll back" the package.  Again, this 
>  can be handled manually via yum remove/install or using rpm --oldpackage if 
>  you know that the situation exists. 
I've never seen that happen in the update tree. 

In the development tree, sure - that's what happens in rawhide. Shit
breaks.

-sv

[Matthew Miller]

On Tue, Dec 06, 2005 at 10:58:38AM -0500, Gene C. wrote:
>  Case 1: A package is install but, now, has been deleted from the
>  development/update tree and the installed package blocks updates. The
>  current answer is to manually remove the package (yum remove or rpm -e)
>  if, and only if, you know that package was deleted .. the recent case of
>  dbus-qt. 
This could be worked around with an "obsolete-packages" rpm in the
repository, which would mark as Obsolete all such known packages.

[...]
>  Case 2:  This is similar to case 1.  In this case, an older version
> of a
>  package is in the development/update tree, The currently installed
>  packages include one or more packages which are a "more recent version"
>  but one that has problems ... therefore, we need to "roll back" the
>  package. Again, this can be handled manually via yum remove/install or
>  using rpm --oldpackage if you know that the situation exists. 
In the non-devel tree, this is "solved" with Epochs. In the devel tree, I'd
say the best thing to do is watch closely, because you're using the
development tree.

-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>
Boston University Linux      ------>              <http://linux.bu.edu/>

[Jef Spaleta]

On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  This could be worked around with an "obsolete-packages" rpm
> in the
>  repository, which would mark as Obsolete all such known packages. 
Is that really appropriate? To unleash a package in a repository that
will remove functionality from a client system?  What happens to user
who were relying on that functionality?

Its one thing to obsolete because of a package name change where the
new package provides similar functionality. Its quite another when
obsoleting is used in a blank payload package to delibrately remove
another package without actually providing similar functionality.

-jef

[Steve G]

On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  This could be worked around with an "obsolete-packages" rpm
> in the
>  repository, which would mark as Obsolete all such known packages. 
Seems like an answer might be to have a "check-retired" command. When packages
are removed from the repo, they are moved to an area where the retired packages
live. Meta-data is built from that directory so there are no typos.
Check-retired would not automatically delete packages just like update doesn't
automatically update packages. It should print the list for approval and proceed
only if the user tells it to.

-Steve


		
__________________________________________ 
Yahoo! DSL – Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com

[seth vidal]

On Tue, 2005-12-06 at 09:30 -0800, Steve G wrote:
>  On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  > This could be worked around with an "obsolete-packages" rpm in the
>  > repository, which would mark as Obsolete all such known packages.
>  
>  Seems like an answer might be to have a "check-retired" command. When packages
>  are removed from the repo, they are moved to an area where the retired packages
>  live. Meta-data is built from that directory so there are no typos.
>  Check-retired would not automatically delete packages just like update doesn't
>  automatically update packages. It should print the list for approval and proceed
>  only if the user tells it to.
>   
sounds like something worth adding to package-cleanup in yum-utils.

Care to hack something together?

-sv

[Matthew Miller]

On Tue, Dec 06, 2005 at 01:18:41PM -0500, seth vidal wrote:
>  > Seems like an answer might be to have a
> "check-retired" command. When
>  > packages are removed from the repo, they are moved to an area where the
>  > retired packages live. Meta-data is built from that directory so there
>  > are no typos. Check-retired would not automatically delete packages just
>  > like update doesn't automatically update packages. It should print the
>  > list for approval and proceed only if the user tells it to.
>  sounds like something worth adding to package-cleanup in yum-utils. 
Hmmm. Also, this'd be a case of needing to tag a repo as being "special" in
some way. Kind of like the thing with source packages....

-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>
Boston University Linux      ------>              <http://linux.bu.edu/>

[Jef Spaleta]

On 12/6/05, Steve G <linux_4ever(a)yahoo.com&gt; wrote:
>  Seems like an answer might be to have a "check-retired"
> command. When packages
>  are removed from the repo, they are moved to an area where the retired packages
>  live. Meta-data is built from that directory so there are no typos. 
They are moved to an area where retired packages live?  So the master
repository server could never actually fully delete the last version
of a particular package?

-jef

[Matthew Miller]

On Tue, Dec 06, 2005 at 01:22:50PM -0500, Jeff Spaleta wrote:
>  > Seems like an answer might be to have a
> "check-retired" command. When
>  > packages are removed from the repo, they are moved to an area where the
>  > retired packages live. Meta-data is built from that directory so there
>  > are no typos.
>  They are moved to an area where retired packages live?  So the master
>  repository server could never actually fully delete the last version
>  of a particular package? 
I guess theoretically you could just keep the metadata.


-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>
Boston University Linux      ------>              <http://linux.bu.edu/>

[Matthew Miller]

On Tue, Dec 06, 2005 at 12:22:22PM -0500, Jeff Spaleta wrote:
>  > This could be worked around with an
> "obsolete-packages" rpm in the
>  > repository, which would mark as Obsolete all such known packages.
>  Is that really appropriate? To unleash a package in a repository that
>  will remove functionality from a client system?  What happens to user
>  who were relying on that functionality? 
Well, if they're using Rawhide, they should by definition not be relying on
anything. And in a stable release, this shouldn't happen except between
releases (where it often does).

>  Its one thing to obsolete because of a package name change where the
>  new package provides similar functionality. Its quite another when
>  obsoleting is used in a blank payload package to delibrately remove
>  another package without actually providing similar functionality. 
Also, the obsoletes tags in such a package would need to precise, so they
match the version-release of the latest Core/Extras package but not beyond
that, so that there wouldn't be a problem with other repos providing the
same package.

-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>
Boston University Linux      ------>              <http://linux.bu.edu/>

[Jef Spaleta]

On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  Also, the obsoletes tags in such a package would need to precise, so
> they
>  match the version-release of the latest Core/Extras package but not beyond
>  that, so that there wouldn't be a problem with other repos providing the
>  same package. 
I think you missed my point.  Is it appropriate to make functionality
"disappear" on a client system via update packages that serve no
purpose other than to obsolete other packages?
Package foo in core provides /usr/bin/foo which can be scripted in
bash scripts. Core decides to remove package foo in fc5 and it has not
been added to Extras yet. The foo package from fc4 has no unfullfilled
deps which prevent it from installing and running on an fc5 system.
Why should an fc4 user who is relying on the functionality provided by
package foo be forced to remove that package via the payload-less
obsoletes package? We aren't talking about packages that have changed
names or sub-packages that have been re-structured. We are talking
about packages that have no functional replacement yet in the
available Fedora repos. Why should an update/upgrade force the removal
of such packages and functionality on client systems?  Isn't this
situation a case-by-case determination for the local admin?

-jef

[Matthew Miller]

On Tue, Dec 06, 2005 at 02:23:29PM -0500, Jeff Spaleta wrote:
>  > Also, the obsoletes tags in such a package would need to
> precise, so they
>  > match the version-release of the latest Core/Extras package but not beyond
>  > that, so that there wouldn't be a problem with other repos providing the
>  > same package.
>  I think you missed my point.  Is it appropriate to make functionality
>  "disappear" on a client system via update packages that serve no
>  purpose other than to obsolete other packages? 
I agree with you that this shouldn't happen in updates to released versions
of Fedora.

>  Package foo in core provides /usr/bin/foo which can be scripted in
>  bash scripts. Core decides to remove package foo in fc5 and it has not
>  been added to Extras yet.  
If it's known that package is going to be added to extras at some point
soon, that package probably shouldn't be added to the obsoleted-packages
list.


>                            The foo package from fc4 has no
> unfullfilled
>  deps which prevent it from installing and running on an fc5 system. 
I'm more concerned about the ones that do have broken deps, which seems to
be a lot of the time in cases like this.

>  Why should an fc4 user who is relying on the functionality provided
> by
>  package foo be forced to remove that package via the payload-less
>  obsoletes package? We aren't talking about packages that have changed
>  names or sub-packages that have been re-structured. We are talking
>  about packages that have no functional replacement yet in the
>  available Fedora repos. Why should an update/upgrade force the removal
>  of such packages and functionality on client systems?  Isn't this
>  situation a case-by-case determination for the local admin? 
It still could be; the obsoletes-packages RPM would just be a tool to help
if you decide to go that way.

-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>
Boston University Linux      ------>              <http://linux.bu.edu/>

[Jef Spaleta]

On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  I agree with you that this shouldn't happen in updates to
> released versions
>  of Fedora. 


>  If it's known that package is going to be added to extras at some
> point
>  soon, that package probably shouldn't be added to the obsoleted-packages
>  list. 
As soon as we get that read-the-future plugin into the Extras
buildsystem. As an exercise for the reader... how many dropped
packages in FC4 that were in FC3 were submitted for review before FC4
release? How many were submitted after FC4 release?


>  I'm more concerned about the ones that do have broken deps, which
> seems to
>  be a lot of the time in cases like this. 
If installed packages have broken deps.... hasn't anaconda in the past
advised you of this as part of the install? fc5t1 anaconda doesn't
count since its clearly not representative of full anaconda
functionality. I don't see why this is a concern unless anaconda is
going to start handling upgrade situation differently.

>  It still could be; the obsoletes-packages RPM would just be a tool to
> help
>  if you decide to go that way. 
If you install obsoletes-packages... there is no decision point. If
obsoleteing is active in the transaction,  it obsoletes all the
packages that is set to obsolete.. like all other obsoletes in all
other packages. For this package to be a tool to help you "decide", it
would have to be treated differently than all other obsoletes in all
other packages which obsolete without user interaction. Are you know
suggesting that this mythical obsoletes-package be treated specially
by the tools?

-jef

[Matthew Miller]

On Tue, Dec 06, 2005 at 03:42:07PM -0500, Jeff Spaleta wrote:
>  > If it's known that package is going to be added to extras at
> some point
>  > soon, that package probably shouldn't be added to the obsoleted-packages
>  > list.
>  As soon as we get that read-the-future plugin into the Extras
>  buildsystem. As an exercise for the reader... how many dropped 
Either that or the "better coordination between core and extras" plugin.

>  packages in FC4 that were in FC3 were submitted for review before
> FC4
>  release? How many were submitted after FC4 release? 
Early-growth time for extras; not really a benchmark for normal future
operation -- or at least, I hope not. 


>  > I'm more concerned about the ones that do have broken deps,
> which seems to
>  > be a lot of the time in cases like this.
>  If installed packages have broken deps.... hasn't anaconda in the past
>  advised you of this as part of the install? fc5t1 anaconda doesn't 
If I remember right, the pre-anaconda installer did, and would make you pick
"proceed anyway". Anaconda, however, just spits out some warnings to one of
the text consoles and goes on its merry way. FC5t1 doesn't count since it
doesn't do upgrades at all. :)

>  count since its clearly not representative of full anaconda
>  functionality. I don't see why this is a concern unless anaconda is
>  going to start handling upgrade situation differently. 
Well, it's a concern because currently, upgraded systems accumulate cruft.a


>  > It still could be; the obsoletes-packages RPM would just be a
> tool to
>  > help if you decide to go that way.
>  If you install obsoletes-packages... there is no decision point. If
>  obsoleteing is active in the transaction,  it obsoletes all the
>  packages that is set to obsolete.. like all other obsoletes in all
>  other packages. For this package to be a tool to help you "decide", it
>  would have to be treated differently than all other obsoletes in all
>  other packages which obsolete without user interaction. Are you know
>  suggesting that this mythical obsoletes-package be treated specially
>  by the tools? 
Anaconda certainly treats a lot of packages specially. And if an admin
wants to prevent this from happening, he or she could add it to the list of
packages yum ignores.

-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>
Boston University Linux      ------>              <http://linux.bu.edu/>

[Jef Spaleta]

On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  Anaconda certainly treats a lot of packages specially. And if an
> admin
>  wants to prevent this from happening, he or she could add it to the list of
>  packages yum ignores. 
Or conversely, the admin can clean out cruft manually. It would take a
rather large effort to know everything this catchall package is going
to do before it does it... to know whether or not it should be
excluded.  Sorry, your arguments have not  convinced me automating
cruft removal using a single payload-less catchall package via
obsolete  is appropriate for default behavior. I'd prefer if obsoletes
were used as sparingly as possible in this way and instead they be
used primarily for situations where packaging has changed but
equivalent functionality is still provided.

I'd rather see a focus on admin tools to notify that the packages are
no longer part of the repos they have enabled and let admins make
decisions on a case-by-case package-by-package basis without incurring
loss of functionality whenever possible.

-jef

[Matt Wilson]

On Tue, Dec 06, 2005 at 12:22:22PM -0500, Jeff Spaleta wrote:
>  On 12/6/05, Matthew Miller <mattdm(a)mattdm.org&gt; wrote:
>  > This could be worked around with an "obsolete-packages" rpm in the
>  > repository, which would mark as Obsolete all such known packages.
>  
>  Is that really appropriate? To unleash a package in a repository that
>  will remove functionality from a client system?  What happens to user
>  who were relying on that functionality? 
Let me outline how this works in a Conary world[1].  The distribution
is defined by a hierarchical set of groups.  When a package is
replaced by another package, we build a special "redirect" in the
repository that moves you from one package to the other.  The
classical example of this is a migration from kbd to consoletools to
kbd.  This is somewhat similar to Obsoletes.

When it comes time to put a package out to pasture (that is, remove it
from the distribution altogether, when there is nothing that assumes
the functionality of the package), we just remove it from the group.
When the user updates to the new version of the group, the removal of
the package from the group is applied to the system (that is, the
package is removed from the systeM).  If the user depends on the
package, they can "pin" it on the system and it won't be removed.

The interactive frontend can help guide users though the changes that
are going to be made by performing an update.  At that point, a user
could decide to pin the package instead of allowing it to be removed.

You may be able to apply this methodology to yum.  What do you think?

[1] Yes, I know everyone was dying to know!

Cheers,

Matt
-- 
Matt Wilson
rPath, Inc.
msw(a)rpath.com

[seth vidal]

>  The interactive frontend can help guide users though the changes
> that
>  are going to be made by performing an update.  At that point, a user
>  could decide to pin the package instead of allowing it to be removed. 
and for a non-interactive run of conary? What happens on the automated
runs?

-sv

[Matt Wilson]

On Tue, Dec 06, 2005 at 05:05:03PM -0500, seth vidal wrote:
>  
>  > The interactive frontend can help guide users though the changes that
>  > are going to be made by performing an update.  At that point, a user
>  > could decide to pin the package instead of allowing it to be removed.
>  
>  and for a non-interactive run of conary? What happens on the automated
>  runs? 
By default, the package will be removed if it was removed from a group
that you have installed and it wasn't added to any other group you
have installed -- _unless_ it creates a dependency problem. So, we're
not removing important deprecated shared libraries that are needed by
other things on the system.

Our philosophy for the automated case is to do the thing that most
people want by default.  In this case we believe that it's better to
be closer to the next version of the OS than it is to leave old
packages littered around forever.  We give the user two ways to rely
on deprecated packages (1. pin them and 2. rely on dependencies to
keep the system in a working state, even if it requires an old
package) and we have rollback functionality that can be used to go
back to the older state.  That makes cleaning up old packages seem
pretty safe to me.

If removing packages seems a little to drastic, you can set up the
groups in Conary one more way.  You can create group-dist-deprecated,
and add all of the obsoleted packages to that group with the default
flag set to False.  Users that have an affinity to old software can
install group-dist-deprecated.  When you remove something from the
distribution (which is group-dist in our case), you would add it to
group-dist-deprecated.  If the user had group-dist-deprecated
installed, the package won't be removed from the system.

Cheers,

Matt
-- 
Matt Wilson
rPath, Inc.
msw(a)rpath.com

[Rahul Sundaram]

Hi

> If removing packages seems a little to drastic, you can set up the
> groups in Conary one more way.  You can create group-dist-deprecated,
> and add all of the obsoleted packages to that group with the default
> flag set to False.  Users that have an affinity to old software can
> install group-dist-deprecated.  When you remove something from the
> distribution (which is group-dist in our case), you would add it to
> group-dist-deprecated.  If the user had group-dist-deprecated
> installed, the package won't be removed from the system.
>   
>  
There is already a yum group for providing compatibility libs for legacy 
software. This solution looks is consistent with the usage of yum group 
in favor of virtual packages in Fedora. It is also more flexible. 

regards
Rahul