debuginfo packages available in updates later than regular packages.

[Sandro Mani]

Hi,

I wondered what the reason is that debuginfo packages seem to enter the 
repos only at the successive push compared to the regular packages, 
which ultimately means that debuginfo packages are available in updates 
ca 1 day after the regular packages. From abrt-reported bugs where 
people generate the backtraces locally, it occasionally happens that 
they send incomplete backtraces due to mismatching debugsymbols, and it 
would certainly help increasing the quality of backtraces if such cases 
could be avoided (btw, can this also happen on the retrace server?).

A nice solution to ensure consistency could be to have each debuginfo 
package require the exact version of the base package installed. Since 
the debuginfo package however cannot know which base (sub)package it 
should depend on, I wonder whether it could work if the package and all 
subpackages should provide something like: Provides: 
debuginfo-requirement(%{name}) = %{version}-%{release}?

Thanks,
Sandro

[Michael Catanzaro]

On Sun, 2013-11-24 at 16:50 +0100, Sandro Mani wrote:
>  From abrt-reported bugs where 
>  people generate the backtraces locally, it occasionally happens that 
>  they send incomplete backtraces due to mismatching debugsymbols, and
>  it 
>  would certainly help increasing the quality of backtraces if such
>  cases 
>  could be avoided 
This does suck, but the bigger problem is that debuginfo packages are
not updated at all - not ever - unless you either

a) use yum instead of PackageKit
b) manually enable the updates-debug repository

So requiring the exact version of the base package will only work if
that gets fixed first.

[Sandro Mani]

On 24.11.2013 17:55, Michael Catanzaro wrote:
>  On Sun, 2013-11-24 at 16:50 +0100, Sandro Mani wrote:
> >  From abrt-reported bugs where
> > people generate the backtraces locally, it occasionally happens that
> > they send incomplete backtraces due to mismatching debugsymbols, and
> > it
> > would certainly help increasing the quality of backtraces if such
> > cases
> > could be avoided
>  This does suck, but the bigger problem is that debuginfo packages are
>  not updated at all - not ever - unless you either
>
>  a) use yum instead of PackageKit
>  b) manually enable the updates-debug repository
>
>  So requiring the exact version of the base package will only work if
>  that gets fixed first.
>  
Oh, I never noticed this! I take the reason the debuginfo packages do 
not live in the "normal" repos is that one wants to reduce the 
repodata/filelist size? Could the current situation be improved by an 
approach similar to:
- Move the debuginfo repo definitions to separate files
- Have a package fedora-release-debug (or similar) install the repo file 
in /etc/yum.repos.d. The repos would be enabled by default when installed.
- Have all debuginfo packages depend on fedora-release-debug
- (ugly) Have debuginfo-install install the repo file before proceeding 
as before.
?

Thanks,
Sandro

[Josh Stone]

On 11/24/2013 09:13 AM, Sandro Mani wrote:
>  Oh, I never noticed this! I take the reason the debuginfo packages do
>
>  not live in the "normal" repos is that one wants to reduce the 
>  repodata/filelist size? Could the current situation be improved by an 
>  approach similar to:
>  - Move the debuginfo repo definitions to separate files
>  - Have a package fedora-release-debug (or similar) install the repo file 
>  in /etc/yum.repos.d. The repos would be enabled by default when installed.
>  - Have all debuginfo packages depend on fedora-release-debug
>  - (ugly) Have debuginfo-install install the repo file before proceeding 
>  as before.
>  ? 
debuginfo-install does install yum-plugin-auto-update-debug-info, which
automatically enables $REPO-debuginfo for each $REPO you have enabled.

[Josh Stone]

On 11/24/2013 10:51 AM, Josh Stone wrote:
>  On 11/24/2013 09:13 AM, Sandro Mani wrote:
> > Oh, I never noticed this! I take the reason the debuginfo packages do 
> > not live in the "normal" repos is that one wants to reduce the 
> > repodata/filelist size? Could the current situation be improved by an 
> > approach similar to:
> > - Move the debuginfo repo definitions to separate files
> > - Have a package fedora-release-debug (or similar) install the repo file 
> > in /etc/yum.repos.d. The repos would be enabled by default when installed.
> > - Have all debuginfo packages depend on fedora-release-debug
> > - (ugly) Have debuginfo-install install the repo file before proceeding 
> > as before.
> > ?
>  
>  debuginfo-install does install yum-plugin-auto-update-debug-info, which
>  automatically enables $REPO-debuginfo for each $REPO you have enabled. 
... and now I see you're trying to solve this for !yum, e.g. PackageKit.
Sorry for the noise...

[Michael Schwendt]

On Sun, 24 Nov 2013 16:50:51 +0100, Sandro Mani wrote:

>  Hi,
>  
>  I wondered what the reason is that debuginfo packages seem to enter the 
>  repos only at the successive push compared to the regular packages, 
>  which ultimately means that debuginfo packages are available in updates 
>  ca 1 day after the regular packages. 
Where did you observe this? On a mirror or on the Fedora Project download
server?

[Sandro Mani]

On 24.11.2013 21:52, Michael Schwendt wrote:
>  On Sun, 24 Nov 2013 16:50:51 +0100, Sandro Mani wrote:
>
> > Hi,
> >
> > I wondered what the reason is that debuginfo packages seem to enter the
> > repos only at the successive push compared to the regular packages,
> > which ultimately means that debuginfo packages are available in updates
> > ca 1 day after the regular packages.
>  Where did you observe this? On a mirror or on the Fedora Project download
>  server? 
I am running rawhide and it always happens that updates come one day, 
and the corresponding debuginfo packages the next day. Actually I'm not 
sure if this is the case also in stable releases, but I though that was 
why the debuginfo symbols in various abrt bugs were mismatching. 
However, I didn't realize that the debuginfo packages were only updated 
via yum and not via PackageKit (as Michael mentioned before in this 
thread), so that is probably the more likely cause.

As far as the mirror is concerned: just using the mirror which yum picks 
for me, so I guess the answer is: pretty much any mirror.

Sandro

[Jan Kratochvil]

On Sun, 24 Nov 2013 16:50:51 +0100, Sandro Mani wrote:
>  A nice solution to ensure consistency could be to have each
>  debuginfo package require the exact version of the base package
>  installed. Since the debuginfo package however cannot know which
>  base (sub)package it should depend on, I wonder whether it could
>  work if the package and all subpackages should provide something
>  like: Provides: debuginfo-requirement(%{name}) =
>  %{version}-%{release}? 
There were multiple Bugs suggesting the same, they are linked together for
example from this one from 2005.  But it all got WONTFIXed:
	Debug info RPMs do not "require" exact maching binary rpm
	https://bugzilla.redhat.com/show_bug.cgi?id=151598

When I have all supported Fedoras in mock most of the days at least one of
them has debuginfos out of sync (=making nightly testing results invalid).


Jan

[Kevin Kofler]

Jan Kratochvil wrote:
>  There were multiple Bugs suggesting the same, they are linked
> together for
>  example from this one from 2005.  But it all got WONTFIXed:
>  Debug info RPMs do not "require" exact maching binary rpm
>  https://bugzilla.redhat.com/show_bug.cgi?id=151598 
The problem with those dependencies is that -debuginfo covers all the 
subpackages, so which one should it require? All of them? That's more often 
than not NOT what I want. (E.g., do we really want to force everyone to 
install gcc-* just because ABRT/DrKonqi/whatever wants to install the 
(usually unneeded) debugging information for libgcc? Getting gcc-debuginfo 
dragged in is bad enough!)

        Kevin Kofler

[Sandro Mani]

On 26.11.2013 03:55, Kevin Kofler wrote:
>  Jan Kratochvil wrote:
> > There were multiple Bugs suggesting the same, they are linked together for
> > example from this one from 2005.  But it all got WONTFIXed:
> > Debug info RPMs do not "require" exact maching binary rpm
> > https://bugzilla.redhat.com/show_bug.cgi?id=151598
>  The problem with those dependencies is that -debuginfo covers all the
>  subpackages, so which one should it require? All of them? That's more often
>  than not NOT what I want. (E.g., do we really want to force everyone to
>  install gcc-* just because ABRT/DrKonqi/whatever wants to install the
>  (usually unneeded) debugging information for libgcc? Getting gcc-debuginfo
>  dragged in is bad enough!) 
Here is a quick and dirty spec implementing the idea I
described: [1]. 
 From what I can see it behaves correctly with any combination of 
packages and subpackages installed. Am I missing something?

Sandro


[1] http://smani.fedorapeople.org/somepackage.spec

[Christopher Meng]

Isn't it should be implemented implicitly in RPM?

[Sandro Mani]

On 26.11.2013 11:50, Christopher Meng wrote:
>  Isn't it should be implemented implicitly in RPM? 
It is
just a proof of concept/idea. Will hardly be implemented like 
this, if something similar is going to be implemented.

Sandro

[Kevin Kofler]

Sandro Mani wrote:
>  [1] http://smani.fedorapeople.org/somepackage.spec 
Yeah, that hack looks like it could work, sorta. If you have multiple 
subpackages installed, having only one subpackage of the correct version and 
the others of wrong versions will still be satisfying the dependency in the 
-debuginfo package. (And subpackages do not always require each other, so 
you cannot always require on the EVR-locked inter-subpackage Requires.)

        Kevin Kofler

[Sandro Mani]

On 26.11.2013 19:55, Kevin Kofler wrote:
>  Sandro Mani wrote:
> > [1] http://smani.fedorapeople.org/somepackage.spec
>  Yeah, that hack looks like it could work, sorta. If you have multiple
>  subpackages installed, having only one subpackage of the correct version and
>  the others of wrong versions will still be satisfying the dependency in the
>  -debuginfo package. (And subpackages do not always require each other, so
>  you cannot always require on the EVR-locked inter-subpackage Requires.)
>
>           Kevin Kofler
>  
Right. Otherwise, I guess the Conflicts approach suggested by Nicolas 
could work and also handle such cases? But before all this, there is the 
current problem that debuginfo packages are only updated when using yum, 
as pointed out by Michael. If debuginfo packages are not shipped in the 
standard repos, some other solution is needed. I pointed out an approach 
which could work:
- Move the debuginfo repo definitions to separate files
- Have a package fedora-release-debug (or similar) install the repo file 
in /etc/yum.repos.d. The repos would be enabled by default when installed.
- Have all debuginfo packages depend on fedora-release-debug
- (ugly) Have debuginfo-install install the repo file before proceeding 
as before.

Any thoughts on that?

Sandro

[Jan Kratochvil]

On Tue, 26 Nov 2013 11:39:38 +0100, Sandro Mani wrote:
>  Here is a quick and dirty spec implementing the idea I described:
>  [1]. From what I can see it behaves correctly with any combination
>  of packages and subpackages installed. Am I missing something?
>  [1] http://smani.fedorapeople.org/somepackage.spec 
There were always intentions to allow installing multiple *-debuginfo.rpm
versions at once.  This would allow for example examining a core file
generated from older binaries (which could be still running when they crashed
while newer rpm was installed on disk).  Currently it is not possible but one
such open Bug with some proposals is:
	undesired cross-rpm elf build-id duplication due to strict content-hash handling in
debugedit.c
	https://bugzilla.redhat.com/show_bug.cgi?id=1002341

(In ideal case for debug purposes one needs also the matching binary, not just
its debuginfo.  But in practice the debuginfo should be sufficient.)

In reality ABRT solves the problem of multi debuginfo rpms by installing new
chroot each time with the needed package versions.  This is a bit expensive.

Your proposal definitely blocks this multiple *-debuginfo.rpm versions idea.
But the multi-version idea has never worked yet and it is not sure it will
ever be.  Your proposal is sure still better than what we have now.


Besides that your proposal has to be implemented into rpm tool code; currently
the debuginfo subpackage is implemented only as a rpm macro in /usr/lib/rpm.
But that is sure OK.


Jan

[Nicolas Mailhot]

Le Mar 26 novembre 2013 03:55, Kevin Kofler a écrit :
>  Jan Kratochvil wrote:
> > There were multiple Bugs suggesting the same, they are linked together
> > for
> > example from this one from 2005.  But it all got WONTFIXed:
> > Debug info RPMs do not "require" exact maching binary rpm
> > https://bugzilla.redhat.com/show_bug.cgi?id=151598
>
>  The problem with those dependencies is that -debuginfo covers all the
>  subpackages, so which one should it require? All of them? That's more
>  often
>  than not NOT what I want. 
Just conflict with all subpackages that do not match the exact debuginfo
version

Regards,

-- 
Nicolas Mailhot

[Sandro Mani]

On 26.11.2013 11:50, Nicolas Mailhot wrote:
>  Le Mar 26 novembre 2013 03:55, Kevin Kofler a écrit :
> > Jan Kratochvil wrote:
> >> There were multiple Bugs suggesting the same, they are linked together
> >> for
> >> example from this one from 2005.  But it all got WONTFIXed:
> >> Debug info RPMs do not "require" exact maching binary rpm
> >> https://bugzilla.redhat.com/show_bug.cgi?id=151598
> > The problem with those dependencies is that -debuginfo covers all the
> > subpackages, so which one should it require? All of them? That's more
> > often
> > than not NOT what I want.
>  Just conflict with all subpackages that do not match the exact debuginfo
>  version
>
>  Regards,
>  
Is there some rpm macro magic to list all the (sub)packages defined in a 
spec?

Sandro