7:29 a.m.
We received a bug report that generated RPM dependencies are too coarse
in rawhide (#1409557).
The bug report is correct at a technical level. But I assumed that it
was not a problem because partial upgrades are in rawhide are not
supported—it's always all-or-nothing.
Comments?
Thanks,
Florian
7:41 a.m.
On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
We received a bug report that generated RPM dependencies are too
coarse in
rawhide (#1409557).
The bug report is correct at a technical level. But I assumed that it was
not a problem because partial upgrades are in rawhide are not supported—it's
always all-or-nothing.
This is rawhide, if it breaks, people should keep the pieces. We shouldn't
change what we do with symbol versions just because of it.
Jakub
10:22 a.m.
On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
This is rawhide, if it breaks, people should keep the pieces.
We shouldn't
change what we do with symbol versions just because of it.
It has nothing to do with rawhide. It is a best practice how to use
map symbols/version script and to have stable API/ABI
https://www.akkadia.org/drepper/dsohowto.pdf
quote:
In short: a new
version is defined for a new release of a DSO whenever
new features are added. The interfaces which changed
in a compatible way get the new version associated. All
the other interfaces must not change and they keep the
version they had in the previous release.
I know it is not a high critical issue and therefore I suggested (in BZ) to
to automatically rebuild docker base image for such change in glibc in rawhide.
It would not fix the problem but it would reduce potential bugs. And in future
we might upgrade base modules for rawhide (if there will be any)
Another possible solution would be to patch the unreleased version with each build in
rawhide
e.g. GLIBC_2.25 - > GLIBC_2.25_unreleased_$(date). So GLIBC_2.25 would be available
only with official release.
I know that GLIBC_2.25_unreleased_$(date) would be different with each build but
that should not be a problem for officially unreleased snapshot of git master.
12:06 p.m.
On 01/02/2017 05:22 PM, Lukas Slebodnik wrote:
I know it is not a high critical issue and therefore I suggested (in
BZ) to
to automatically rebuild docker base image for such change in glibc in rawhide.
It would not fix the problem but it would reduce potential bugs. And in future
we might upgrade base modules for rawhide (if there will be any)
The bug is in the user-supplied container build scripts. Recommended
practice is to run “dnf update” (or “yum update”) as part of the build
process. Most distributions do not automatically respin containers for
every package update in the base image.
Thanks,
Florian
3:40 p.m.
On 01/02/2017 05:22 PM, Lukas Slebodnik wrote:
The bug is in the user-supplied container build scripts. Recommended
practice is to run “dnf update” (or “yum update”) as part of the build
process.
Could you provide some link where it is recommended?
Because most of pages say exactly opposite or does not mention that at all
https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practi...
"""
You should avoid RUN apt-get upgrade or dist-upgrade, as many of the “essential” packages
from the base images won’t upgrade inside an unprivileged container. If a package
contained in the base image is out-of-date, you should contact its maintainers.
"""
https://docs.openshift.com/container-platform/3.3/creating_images/guideli...
"""
Wherever possible, we recommend that you base your image on an appropriate upstream image
using the FROM statement. This ensures your image can easily pick up security fixes from
an upstream image when it is updated, rather than you having to update your dependencies
directly.
"""
Another problem with upgrading all packages in base image is that
it a little bit breaks caching of intermediate layers when building an image.
Most distributions do not automatically respin containers for
every package update in the base image.
Update of glibc should not be considered as a update ofa any other package.
glibc is very special. And updating glibc in rawhide base image would be just a
workaround
for improper release of git master into rawhide. The ideal would be to follow guidelines
for adding symbols to the latest version in glibc
LS
4:03 a.m.
It has nothing to do with rawhide. It is a best practice how to use
map symbols/version script and to have stable API/ABI
https://www.akkadia.org/drepper/dsohowto.pdf
quote:
I know it is not a high critical issue and therefore I suggested (in BZ) to
to automatically rebuild docker base image for such change in glibc in rawhide.
It would not fix the problem but it would reduce potential bugs. And in future
we might upgrade base modules for rawhide (if there will be any)
Another possible solution would be to patch the unreleased version with each build in
rawhide
e.g. GLIBC_2.25 - > GLIBC_2.25_unreleased_$(date). So GLIBC_2.25 would be available
only with official release.
I know that GLIBC_2.25_unreleased_$(date) would be different with each build but
that should not be a problem for officially unreleased snapshot of git master.
I though a little bit and replacing GLIBC_2.25 - > GLIBC_2.25_unreleased_$(date) would
be a little bit complicated because it would require to rebuild also all packages which
requires GLIBC_2.25 in rawhide.
ATM it's just 10 packages.
BTW I checked the rawhide releases for older version (2..22.90) and symbols were added
just once
between released glibc-2.22.90-13.fc24 and glibc-2.22.90-16.fc24.x86_64.rpm. No rebuild
would be needed because all new symbols were together.
The development process for is a little bit different:
3 symbols were added between glibc-2.24.90-13 and glibc-2.24.90-16
2 new symbols were added between glibc-2.24.90-22 and glibc-2.24.90-23
2 new symbols were added between glibc-2.24.90-23 and glibc-2.24.90-24
You would need to rebuild all packages which depends on GLIBC_2.25* just twice.
8:37 a.m.
On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
We received a bug report that generated RPM dependencies are too
coarse in rawhide (#1409557).
The bug report is correct at a technical level. But I assumed that
it was not a problem because partial upgrades are in rawhide are not
supported—it's always all-or-nothing.
I think officially, we don't "support" anything but all-or-nothing
upgrades in *all* branches. That is, if you cherry-pick an update from
updates (or even updates-testing) and it also needs some other package
to be upgraded even when that's not made clear from the deps, our
stance is "oh, upgrade everything and then it will work".
It might be nice to be more granular, but I think that describes the
praxis.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
9:22 a.m.
On Mon, Jan 02, 2017 at 09:37:01AM -0500, Matthew Miller wrote:
On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
> We received a bug report that generated RPM dependencies are too
> coarse in rawhide (#1409557).
>
> The bug report is correct at a technical level. But I assumed that
> it was not a problem because partial upgrades are in rawhide are not
> supported—it's always all-or-nothing.
I think officially, we don't "support" anything but all-or-nothing
upgrades in *all* branches. That is, if you cherry-pick an update from
updates (or even updates-testing) and it also needs some other package
to be upgraded even when that's not made clear from the deps, our
stance is "oh, upgrade everything and then it will work".
I would describe this particular situation as a bug. Dependencies
should be real, and encode version numbers if it's relevant.
--
Tomasz Torcz ,,If you try to upissue this patchset I shall be seeking
xmpp: zdzichubg(a)chrome.pl an IP-routable hand grenade.'' -- Andrew Morton
(LKML)
3:55 p.m.
On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
I think officially, we don't "support" anything but all-or-nothing
upgrades in *all* branches. That is, if you cherry-pick an update from
updates (or even updates-testing) and it also needs some other package
to be upgraded even when that's not made clear from the deps, our
stance is "oh, upgrade everything and then it will work".
Such practice
just hides packaging bugs.
Therefore such recommendation does not look good to me in general.
LS
10:43 p.m.
On Mon, 2017-01-02 at 09:37 -0500, Matthew Miller wrote:
On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
> We received a bug report that generated RPM dependencies are too
> coarse in rawhide (#1409557).
>
> The bug report is correct at a technical level. But I assumed that
> it was not a problem because partial upgrades are in rawhide are not
> supported—it's always all-or-nothing.
I think officially, we don't "support" anything but all-or-nothing
upgrades in *all* branches. That is, if you cherry-pick an update from
updates (or even updates-testing) and it also needs some other package
to be upgraded even when that's not made clear from the deps, our
stance is "oh, upgrade everything and then it will work".
It might be nice to be more granular, but I think that describes the
praxis.
No, I don't think that's correct. The policy requires updates to be
dep-complete.
https://fedoraproject.org/wiki/Updates_Policy#Updating_inter-dependent_pa...
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
10:45 p.m.
On Mon, 2017-01-02 at 20:43 -0800, Adam Williamson wrote:
On Mon, 2017-01-02 at 09:37 -0500, Matthew Miller wrote:
> On Mon, Jan 02, 2017 at 02:29:47PM +0100, Florian Weimer wrote:
> > We received a bug report that generated RPM dependencies are too
> > coarse in rawhide (#1409557).
> >
> > The bug report is correct at a technical level. But I assumed that
> > it was not a problem because partial upgrades are in rawhide are not
> > supported—it's always all-or-nothing.
>
> I think officially, we don't "support" anything but all-or-nothing
> upgrades in *all* branches. That is, if you cherry-pick an update from
> updates (or even updates-testing) and it also needs some other package
> to be upgraded even when that's not made clear from the deps, our
> stance is "oh, upgrade everything and then it will work".
>
> It might be nice to be more granular, but I think that describes the
> praxis.
No, I don't think that's correct. The policy requires updates to be
dep-complete.
https://fedoraproject.org/wiki/Updates_Policy#Updating_inter-dependent_pa...
...but to expand on that, that's for stable releases. So far as Rawhide
is concerned, historically my understanding has been the same as
Florian's, we haven't ever claimed that dependencies will be so
comprehensive that you can just cherry-pick packages and be sure
everything will work, you have always been expected to do a full system
update each time for Rawhide.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
2:08 a.m.
On Mon, 2017-01-02 at 20:43 -0800, Adam Williamson wrote:
...but to expand on that, that's for stable releases. So far as Rawhide
is concerned, historically my understanding has been the same as
Florian's, we haven't ever claimed that dependencies will be so
comprehensive that you can just cherry-pick packages and be sure
everything will work, you have always been expected to do a full system
update each time for Rawhide.
I use rawhide on many machines and even on my laptop.
I know it can break sometimes and i am fine with that. Because I have a btrfs snapshots of
older working rawhide and I can try to upgrade functional version to broken version. But
in this case i need to "bisect" which upgrade caused the bug. And every time I
am surprised which package cause the bug. Mostly I suspected a different package. However,
if the upgrade of glibc breaks things due to missing symbols then it is really huge
complication If you cannot upgrade packages with dnf and need to use rpm with URLs from
koji. That's the reason why upgrade of glibc should be treated more seriously in
rawhide. (IMHO)
LS
2:18 a.m.
On Tue, Jan 03, 2017 at 08:08:32AM -0000, Lukas Slebodnik wrote:
> On Mon, 2017-01-02 at 20:43 -0800, Adam Williamson wrote:
>
> ...but to expand on that, that's for stable releases. So far as Rawhide
> is concerned, historically my understanding has been the same as
> Florian's, we haven't ever claimed that dependencies will be so
> comprehensive that you can just cherry-pick packages and be sure
> everything will work, you have always been expected to do a full system
> update each time for Rawhide.
I use rawhide on many machines and even on my laptop. I know it can break
sometimes and i am fine with that. Because I have a btrfs snapshots of
older working rawhide and I can try to upgrade functional version to
broken version. But in this case i need to "bisect" which upgrade caused
the bug. And every time I am surprised which package cause the bug.
Mostly I suspected a different package. However, if the upgrade of glibc
breaks things due to missing symbols then it is really huge complication
If you cannot upgrade packages with dnf and need to use rpm with URLs from
koji. That's the reason why upgrade of glibc should be treated more
seriously in rawhide. (IMHO)
But there is nothing that can be done about it. The symbol versions come
from upstream, every release that adds new symbols adds new symbol version,
and we do want to test glibc before it is released, we can't just wait until
it is released and then push it into the distro, hoping other distros have tested
it. Adding a symbol version for each set of symbols added together would be
major change for upstream, one that would have severe runtime implications,
and also effectively require that as soon as you add something you've done
everything right, no further changes to the API are possible. While in the
current development model, until glibc is released, there is still time to
fix stuff up, remove symbols again (doesn't happen often) etc.
Jakub
4:18 a.m.
On Tue, Jan 03, 2017 at 08:08:32AM -0000, Lukas Slebodnik wrote:
But there is nothing that can be done about it. The symbol versions come
from upstream, every release that adds new symbols adds new symbol version,
and we do want to test glibc before it is released, we can't just wait until
it is released and then push it into the distro, hoping other distros have tested
it. Adding a symbol version for each set of symbols added together would be
major change for upstream, one that would have severe runtime implications,
and also effectively require that as soon as you add something you've done
everything right, no further changes to the API are possible. While in the
current development model, until glibc is released, there is still time to
fix stuff up, remove symbols again (doesn't happen often) etc.
That's the reason why i mention (in different mail) to rename unreleased
version to some snapshot.
GLIBC_2.25 - > GLIBC_2.25_unreleased_$num
It would solve the problem because packages would either require GLIBC_2.25_unreleased_1
or GLIBC_2.25_unreleased_2. In the first case, they would need to be rebuild against the
latest glibc. And people could not hit such bugs as 1409557 or 1409590.
But I can see that neither Florian nor you like such idea. Therefore it would be very good
to at least rebuild images(docker, cloud, maybe module in future) after adding adding new
symbols to glibc in rawhide.
I wrote in different mail that it does not happen very often (3 times in 26 minir releases
so far for fedora 26)
7:46 a.m.
On Mon, Jan 02, 2017 at 08:43:37PM -0800, Adam Williamson wrote:
> I think officially, we don't "support" anything
but all-or-nothing
> upgrades in *all* branches. That is, if you cherry-pick an update from
> updates (or even updates-testing) and it also needs some other package
> to be upgraded even when that's not made clear from the deps, our
> stance is "oh, upgrade everything and then it will work".
[...]
No, I don't think that's correct. The policy requires updates
to be
dep-complete.
https://fedoraproject.org/wiki/Updates_Policy#Updating_inter-dependent_pa...
I don't think that conflicts with what I'm saying — if in that
secenario (package A requires new versions of B and C), if the new
version of B has a change but didn't change soname, and someone updates
_just_ package A on their system, not the whole update, oops. We
*expect* them to take the whole update.
Or, new B is _already_ in updates, and A is built against it; if you
install a fresh system without updates and just apply A, again...
you're in an edge case situation, not the expected one.
I'm not sayin' that's not a bug in the packaging. When it happens, it
probably should be fixed. But I bet we've had this happen dozens of
times and not done anything about it.
(Honestly, if I'm wrong here, I'd be happy to hear it.)
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
11:22 a.m.
On Tue, 2017-01-03 at 08:46 -0500, Matthew Miller wrote:
On Mon, Jan 02, 2017 at 08:43:37PM -0800, Adam Williamson wrote:
> > I think officially, we don't "support" anything but
all-or-nothing
> > upgrades in *all* branches. That is, if you cherry-pick an update from
> > updates (or even updates-testing) and it also needs some other package
> > to be upgraded even when that's not made clear from the deps, our
> > stance is "oh, upgrade everything and then it will work".
[...]
> No, I don't think that's correct. The policy requires updates to be
> dep-complete.
>
> https://fedoraproject.org/wiki/Updates_Policy#Updating_inter-dependent_pa...
I don't think that conflicts with what I'm saying — if in that
secenario (package A requires new versions of B and C), if the new
version of B has a change but didn't change soname, and someone updates
_just_ package A on their system, not the whole update, oops. We
*expect* them to take the whole update.
Sure - but you talked about cherry-picking "an update", not "a
package". Cherry picking *packages* from updates-testing is not
required to work (or fail 'correctly' due to a dependency), no, but
cherry-picking *updates* more or less _is_ supposed to work.
If by 'update' you meant 'package' then we're not disagreeing.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
11:24 a.m.
On Tue, Jan 03, 2017 at 09:22:52AM -0800, Adam Williamson wrote:
Sure - but you talked about cherry-picking "an update", not
"a
package". Cherry picking *packages* from updates-testing is not
required to work (or fail 'correctly' due to a dependency), no, but
cherry-picking *updates* more or less _is_ supposed to work.
If by 'update' you meant 'package' then we're not disagreeing.
I did mean that. Sorry for introducing confusion through sloppy
language. :)
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
1:19 p.m.
I think that we need to wait for
https://bugzilla.redhat.com/show_bug.cgi?id=1320954
But I think it still would be good to to at least rebuild images.
LS
1:28 p.m.
On 01/05/2017 08:19 PM, Lukas Slebodnik wrote:
I think that we need to wait for
https://bugzilla.redhat.com/show_bug.cgi?id=1320954
Even if we had this capability, I'm not sure if we would use it in
rawhide. It could considerably increase the size of the dependency
information.
But I think it still would be good to to at least rebuild images.
We don't seem to have agreement whether image rebuilds are required, and
if image users should instead use an (uncached) yum update during
container builds.
Thanks,
Florian
4:36 p.m.
On 01/05/2017 08:19 PM, Lukas Slebodnik wrote:
Even if we had this capability, I'm not sure if we would use it in
rawhide. It could considerably increase the size of the dependency
information.
You would remove "temporary versions" with official release.
I know it's not ideal and would require to rebuild some packages.
But it would be much better then current problematic approach.
LS
1:44 a.m.
On 01/05/2017 11:36 PM, Lukas Slebodnik wrote:
> On 01/05/2017 08:19 PM, Lukas Slebodnik wrote:
>
> Even if we had this capability, I'm not sure if we would use it in
> rawhide. It could considerably increase the size of the dependency
> information.
>
You would remove "temporary versions" with official release.
I know it's not ideal and would require to rebuild some packages.
But it would be much better then current problematic approach.
This would potentially make a lot of software in third-party
repositories uninstallable. This would be much more difficult to work
around than a simple ”dnf update” command, so I don't think it's an
improvement.
Thanks,
Florian
2634
days inactive
2669
days old
20 comments
6 participants
participants (6)
-
Adam Williamson -
Florian Weimer -
Jakub Jelinek -
Lukas Slebodnik -
Matthew Miller -
Tomasz Torcz