Hello,
we are currently dealing with a tricky situation, that the NSS and Mozilla package maintainers have been discussing, and I'd like to publish our plan.
The most recent NSS update, version 3.28.1, is required to ship to the Firefox 51 update planned for January 24.
Unfortunately, NSS 3.28.1 is incompatible with Mozilla applications version 50 and older.
If Mozilla 50 or older is used together with NSS 3.28 or newer, and the application attempts to use HTTP v2, the connections to some servers may fail (including connections to Google servers).
The fix is simple, it's possible to apply a small patch to the older Mozilla applications, to make it compatible with NSS 3.28.1
The difficulty here is the timing, and it's a conflict between "don't break applications in Fedora" and "ship new Firefox security update as soon as possible".
If we start by shipping NSS 3.28.1 first, without yet having fixed the Mozilla applications, then we allow Firefox 51 to be shipped, but we risk that the other applications aren't fixed in time, and that users might see regressions, caused by the upgrade to NSS 3.28.1
Alternatively, if we wait until all affected Mozilla packages have been updated to fixed versions, it might delay the January 24 Firefox 51 update.
After discussing this, we have a preference to avoid the breakage in Fedora, and try to ship all required updates as soon as possible.
In order to avoid the breakage, we want to add "Conflicts:" statements to the NSS 3.28.1 package, that makes it conflict with all known Mozilla packages that don't contain the required fix yet.
The packages we have identified are: - firefox - thunderbird - seamonkey - xulrunner - icecat
I see that for all the above packages, build attempts that include the fix are already ongoing in koji, so there's hope that we might be able to resolve the situation in time.
However, if ANY of the above build cannot be completed soon, or, if ANY of the updates cannot move to the stable Fedora updates, it can block users from upgrading to the Firefox 51 update on Jan 24.
Is that acceptable?
Do you agree that we make NSS conflict with any known incompatible packages mentioned above, and thereby may inhibit a subset of Fedora users from upgrading to Firefox 51 immediately?
If we can get all the above builds done quickly, and all of them pushed to Fedora stable updates quickly, we're good.
Note that we have the remaining risk that we haven't identified all Mozilla packages that might be affected. The relevant code isn't in NSS, but in Mozilla's network code. That means, if the above list of packages isn't the complete set of affected Mozilla based applications, other packages might still experience the connectivity regression. But as soon as another package is identified, it can rebuild to pick up the mentioned fix.
Thanks Kai
PS: Tracking bug is here: https://bugzilla.redhat.com/show_bug.cgi?id=1381400 (Don't get confused with the separate, unrelated discussion on TLS 1.3) An example of the regression is here: https://bugzilla.redhat.com/show_bug.cgi?id=1414929
On Fri, Jan 20, 2017 at 9:52 AM, Kai Engert kaie@kuix.de wrote:
Hello,
we are currently dealing with a tricky situation, that the NSS and Mozilla package maintainers have been discussing, and I'd like to publish our plan.
The most recent NSS update, version 3.28.1, is required to ship to the Firefox 51 update planned for January 24.
Unfortunately, NSS 3.28.1 is incompatible with Mozilla applications version 50 and older.
If Mozilla 50 or older is used together with NSS 3.28 or newer, and the application attempts to use HTTP v2, the connections to some servers may fail (including connections to Google servers).
The fix is simple, it's possible to apply a small patch to the older Mozilla applications, to make it compatible with NSS 3.28.1
The difficulty here is the timing, and it's a conflict between "don't break applications in Fedora" and "ship new Firefox security update as soon as possible".
If we start by shipping NSS 3.28.1 first, without yet having fixed the Mozilla applications, then we allow Firefox 51 to be shipped, but we risk that the other applications aren't fixed in time, and that users might see regressions, caused by the upgrade to NSS 3.28.1
Alternatively, if we wait until all affected Mozilla packages have been updated to fixed versions, it might delay the January 24 Firefox 51 update.
After discussing this, we have a preference to avoid the breakage in Fedora, and try to ship all required updates as soon as possible.
In order to avoid the breakage, we want to add "Conflicts:" statements to the NSS 3.28.1 package, that makes it conflict with all known Mozilla packages that don't contain the required fix yet.
The packages we have identified are:
- firefox
- thunderbird
- seamonkey
- xulrunner
- icecat
I see that for all the above packages, build attempts that include the fix are already ongoing in koji, so there's hope that we might be able to resolve the situation in time.
However, if ANY of the above build cannot be completed soon, or, if ANY of the updates cannot move to the stable Fedora updates, it can block users from upgrading to the Firefox 51 update on Jan 24.
Is that acceptable?
Do you agree that we make NSS conflict with any known incompatible packages mentioned above, and thereby may inhibit a subset of Fedora users from upgrading to Firefox 51 immediately?
If we can get all the above builds done quickly, and all of them pushed to Fedora stable updates quickly, we're good.
Note that we have the remaining risk that we haven't identified all Mozilla packages that might be affected. The relevant code isn't in NSS, but in Mozilla's network code. That means, if the above list of packages isn't the complete set of affected Mozilla based applications, other packages might still experience the connectivity regression. But as soon as another package is identified, it can rebuild to pick up the mentioned fix.
Is bundling the newer NSS release inside of firefox itself an option? While it may not be the best long-term solution and we all know the downsides of bundling, it is at least pragmatic in the short-term. That would allow firefox to ship and time for the remaining packages to be updated. Once they're ready, the bundling in firefox could be dropped and an update with all the packages could be done.
josh
On 01/20/2017 04:13 PM, Josh Boyer wrote:
On Fri, Jan 20, 2017 at 9:52 AM, Kai Engert kaie@kuix.de wrote:
Hello,
we are currently dealing with a tricky situation, that the NSS and Mozilla package maintainers have been discussing, and I'd like to publish our plan.
The most recent NSS update, version 3.28.1, is required to ship to the Firefox 51 update planned for January 24.
Unfortunately, NSS 3.28.1 is incompatible with Mozilla applications version 50 and older.
If Mozilla 50 or older is used together with NSS 3.28 or newer, and the application attempts to use HTTP v2, the connections to some servers may fail (including connections to Google servers).
The fix is simple, it's possible to apply a small patch to the older Mozilla applications, to make it compatible with NSS 3.28.1
The difficulty here is the timing, and it's a conflict between "don't break applications in Fedora" and "ship new Firefox security update as soon as possible".
If we start by shipping NSS 3.28.1 first, without yet having fixed the Mozilla applications, then we allow Firefox 51 to be shipped, but we risk that the other applications aren't fixed in time, and that users might see regressions, caused by the upgrade to NSS 3.28.1
Alternatively, if we wait until all affected Mozilla packages have been updated to fixed versions, it might delay the January 24 Firefox 51 update.
After discussing this, we have a preference to avoid the breakage in Fedora, and try to ship all required updates as soon as possible.
In order to avoid the breakage, we want to add "Conflicts:" statements to the NSS 3.28.1 package, that makes it conflict with all known Mozilla packages that don't contain the required fix yet.
The packages we have identified are:
- firefox
- thunderbird
- seamonkey
- xulrunner
- icecat
I see that for all the above packages, build attempts that include the fix are already ongoing in koji, so there's hope that we might be able to resolve the situation in time.
However, if ANY of the above build cannot be completed soon, or, if ANY of the updates cannot move to the stable Fedora updates, it can block users from upgrading to the Firefox 51 update on Jan 24.
Is that acceptable?
Do you agree that we make NSS conflict with any known incompatible packages mentioned above, and thereby may inhibit a subset of Fedora users from upgrading to Firefox 51 immediately?
If we can get all the above builds done quickly, and all of them pushed to Fedora stable updates quickly, we're good.
Note that we have the remaining risk that we haven't identified all Mozilla packages that might be affected. The relevant code isn't in NSS, but in Mozilla's network code. That means, if the above list of packages isn't the complete set of affected Mozilla based applications, other packages might still experience the connectivity regression. But as soon as another package is identified, it can rebuild to pick up the mentioned fix.
Is bundling the newer NSS release inside of firefox itself an option? While it may not be the best long-term solution and we all know the downsides of bundling, it is at least pragmatic in the short-term. That would allow firefox to ship and time for the remaining packages to be updated. Once they're ready, the bundling in firefox could be dropped and an update with all the packages could be done.
All builds are ready except TB on arm. I'm sure we make that in time.
Martin
josh _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On Fri, 2017-01-20 at 10:22 -0600, Michael Catanzaro wrote:
On Fri, 2017-01-20 at 16:17 +0100, Martin Stransky wrote:
All builds are ready except TB on arm. I'm sure we make that in time.
Martin
Please just make sure they all get released in the same Bodhi update to avoid breakage.
Yes, that's our intention.
I see that the Icecat maintainer has already built updated packages for F24 and F25 that include the required patch.
In order to create the combined update, I need commit access for all involved packages. The remaining piece are the commit privileges for Icecat. I've requested them, but haven't received them yet.
Thanks Kai
On 01/20/2017 12:15 PM, Kai Engert wrote:
In order to create the combined update, I need commit access for all involved packages. The remaining piece are the commit privileges for Icecat. I've requested them, but haven't received them yet.
If we're under a time constraint I'm sure a provenpackager would be willing to create the update for you.
On Fri, 2017-01-20 at 13:12 -0600, Michael Cronenworth wrote:
On 01/20/2017 12:15 PM, Kai Engert wrote:
In order to create the combined update, I need commit access for all involved packages. The remaining piece are the commit privileges for Icecat. I've requested them, but haven't received them yet.
If we're under a time constraint I'm sure a provenpackager would be willing to create the update for you.
I've been granted the required permissions.
(Note that provenpackagers don't have access to firefox, I've been told, so their powers wouldn't have been sufficient.)
Thanks! Kai
On 01/20/2017 01:16 PM, Kai Engert wrote:
I've been granted the required permissions.
Good.
(Note that provenpackagers don't have access to firefox, I've been told, so their powers wouldn't have been sufficient.)
I'm not aware of that. I pulled up the current Firefox update that is unpushed and I am seeing buttons to push to testing or push to stable.
On Fri, 2017-01-20 at 10:22 -0600, Michael Catanzaro wrote:
Please just make sure they all get released in the same Bodhi update to avoid breakage.
The combined updates have been submitted to testing: F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bbb320ba18 F25: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
If you can, please help testing, thank in advance!
(Note that the NSS update for rawhide, which enforces the conflicts, hasn't been built yet, because we're waiting for a successful icecat rebuild on rawhide.)
Kai
On Sat, Jan 21, 2017 at 11:57 AM, Kai Engert kaie@kuix.de wrote:
On Fri, 2017-01-20 at 10:22 -0600, Michael Catanzaro wrote:
Please just make sure they all get released in the same Bodhi update to avoid breakage.
The combined updates have been submitted to testing: F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bbb320ba18 F25: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
If you can, please help testing, thank in advance!
Please edit the default karma and put it higher than the default 3 so we don't get 3 people that test the one thing pushing it stable
(Note that the NSS update for rawhide, which enforces the conflicts, hasn't been built yet, because we're waiting for a successful icecat rebuild on rawhide.)
Kai _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On 01/21/2017 01:17 PM, Peter Robinson wrote:
On Sat, Jan 21, 2017 at 11:57 AM, Kai Engert kaie@kuix.de wrote:
On Fri, 2017-01-20 at 10:22 -0600, Michael Catanzaro wrote:
Please just make sure they all get released in the same Bodhi update to avoid breakage.
The combined updates have been submitted to testing: F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bbb320ba18 F25: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
If you can, please help testing, thank in advance!
Please edit the default karma and put it higher than the default 3 so we don't get 3 people that test the one thing pushing it stable
Maybe disable autokarma? Then you can decide to push when you think it is well tested.
On Sat, 2017-01-21 at 13:20 +0100, Christian Dersch wrote:
The combined updates have been submitted to testing: F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bbb320ba18 F25: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
If you can, please help testing, thank in advance!
Please edit the default karma and put it higher than the default 3 so we don't get 3 people that test the one thing pushing it stable
Maybe disable autokarma? Then you can decide to push when you think it is well tested.
With autokarma disabled, is there a minimum test duration, before it can get pushed to stable?
Because we probably want to have it pushed to stable on Monday.
Kai
On 01/21/2017 03:25 PM, Kai Engert wrote:
With autokarma disabled, is there a minimum test duration, before it can get pushed to stable?
Because we probably want to have it pushed to stable on Monday.
Autokarma just means the package will not be pushed automatically, but karma mechanism is still active. So once your update reached the stable karma level you defined, you can hit the push to stable button. If level is not reached you have to wait 7 days (but thats the case with autokarma too), but as the update contains prominent applications I don't expect it takes so long ;)
On 01/21/2017 03:44 PM, Christian Dersch wrote:
On 01/21/2017 03:25 PM, Kai Engert wrote:
With autokarma disabled, is there a minimum test duration, before it can get pushed to stable?
Because we probably want to have it pushed to stable on Monday.
Autokarma just means the package will not be pushed automatically, but karma mechanism is still active. So once your update reached the stable karma level you defined, you can hit the push to stable button. If level is not reached you have to wait 7 days (but thats the case with autokarma too), but as the update contains prominent applications I don't expect it takes so long ;)
Just forgot: Until monday is a *short* time, we have weekend and the update did not reach testing yet. I suggest more time.
On Sat, 2017-01-21 at 15:47 +0100, Christian Dersch wrote:
On 01/21/2017 03:44 PM, Christian Dersch wrote:
Autokarma just means the package will not be pushed automatically, but karma mechanism is still active. So once your update reached the stable karma level you defined, you can hit the push to stable button. If level is not reached you have to wait 7 days (but thats the case with autokarma too), but as the update contains prominent applications I don't expect it takes so long ;)
Just forgot: Until monday is a *short* time, we have weekend and the update did not reach testing yet. I suggest more time.
I've disabled autokarma.
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
I suggest that we make a broad call for getting this update tested widely by Monday.
Kai
On 01/21/2017 04:15 PM, Kai Engert wrote:
I've disabled autokarma.
Thanks :)
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
Well, security fixes are one important point, possibly breaking things in a stable release another one (I tend to think everything is fixed now). Don't get me wrong, I don't want to slow down things, but NSS is a widely used library, even in essential system components. I don't like the idea of pushing it to stable without enough testing for some days (lets say 4-5 days at minimum), especially as there were other issues like non-working FreeIPA (which is fixed I guess?). First, the update has to be pushed to testing (not happened right now) and has to arrive @users, then they also should have some time to test. Some users like me also test ealier by downloading the stuff from Koji, but in general that is what we have updates-testing for. I'm a bit upset because I don't like the idea of forcing "that has to happen until Monday" when there might be side-effects like the one with FreeIPA. Firefox is not the only thing in the world. If Firefox update is extremely urgent it should get a bundled copy of nss until we have the update.
I suggest that we make a broad call for getting this update tested widely by Monday.
Good idea! It should contain some hints what and how to test.
Greetings, Christian
On 01/21/2017 04:33 PM, Christian Dersch wrote:
On 01/21/2017 04:15 PM, Kai Engert wrote:
I've disabled autokarma.
Thanks :)
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
Well, security fixes are one important point, possibly breaking things in a stable release another one (I tend to think everything is fixed now). Don't get me wrong, I don't want to slow down things, but NSS is a widely used library, even in essential system components. I don't like the idea of pushing it to stable without enough testing for some days (lets say 4-5 days at minimum), especially as there were other issues like non-working FreeIPA (which is fixed I guess?). First, the update has to be pushed to testing (not happened right now) and has to arrive @users, then they also should have some time to test. Some users like me also test ealier by downloading the stuff from Koji, but in general that is what we have updates-testing for. I'm a bit upset because I don't like the idea of forcing "that has to happen until Monday" when there might be side-effects like the one with FreeIPA. Firefox is not the only thing in the world. If Firefox update is extremely urgent it should get a bundled copy of nss until we have the update.
We should not break FreeIPA critically with that update (crashes on start or so) for sure. But don't expect rock solid experience from Fedora - there are RHEL/CentOS or other enterprise distros for that.
Fedora is focused mainly on new features which is the nss 3.28.1 update and minor bugs should be addressed later.
ma.
I suggest that we make a broad call for getting this update tested widely by Monday.
Good idea! It should contain some hints what and how to test.
Greetings, Christian _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On 01/23/2017 11:35 AM, Martin Stransky wrote:
We should not break FreeIPA critically with that update (crashes on start or so) for sure. But don't expect rock solid experience from Fedora - there are RHEL/CentOS or other enterprise distros for that.
Fedora is focused mainly on new features which is the nss 3.28.1 update and minor bugs should be addressed later.
I know that very well, thats my main reason for being a Fedora contributor ;) I called for a bit more time to test and investigate for bigger issues, not blocking it entirely. Just to ensure that we do not upset our users by breaking things.
Christian
Hi,
On 23-01-17 11:35, Martin Stransky wrote:
On 01/21/2017 04:33 PM, Christian Dersch wrote:
On 01/21/2017 04:15 PM, Kai Engert wrote:
I've disabled autokarma.
Thanks :)
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
Well, security fixes are one important point, possibly breaking things in a stable release another one (I tend to think everything is fixed now). Don't get me wrong, I don't want to slow down things, but NSS is a widely used library, even in essential system components. I don't like the idea of pushing it to stable without enough testing for some days (lets say 4-5 days at minimum), especially as there were other issues like non-working FreeIPA (which is fixed I guess?). First, the update has to be pushed to testing (not happened right now) and has to arrive @users, then they also should have some time to test. Some users like me also test ealier by downloading the stuff from Koji, but in general that is what we have updates-testing for. I'm a bit upset because I don't like the idea of forcing "that has to happen until Monday" when there might be side-effects like the one with FreeIPA. Firefox is not the only thing in the world. If Firefox update is extremely urgent it should get a bundled copy of nss until we have the update.
We should not break FreeIPA critically with that update (crashes on start or so) for sure. But don't expect rock solid experience from Fedora - there are RHEL/CentOS or other enterprise distros for that.
Ugh, no no no. Things like this are why every web-2.0 (or 3.0 or whatever we are at now) developer is using Ubuntu vms to develop on and not Fedora.
First most and foremost we must focus on our users and not break things, sure sometimes we do accidentally break something but saying: "don't expect rock solid experience from Fedora" is not the right attitude, we really need to stop people seeing Fedora as some experimental platform. And in order to do so we also should not see it as such ourselves.
Fedora is focused mainly on new features which is the nss 3.28.1 update and minor bugs should be addressed later.
Fedora is focussed on Freedom, Features, First and Friends, it is not very (user) Friendly to provide anything but a rock solid platform (once released / gold).
Regards,
Hans
ma.
I suggest that we make a broad call for getting this update tested widely by Monday.
Good idea! It should contain some hints what and how to test.
Greetings, Christian _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On ma, 23 tammi 2017, Martin Stransky wrote:
On 01/21/2017 04:33 PM, Christian Dersch wrote:
On 01/21/2017 04:15 PM, Kai Engert wrote:
I've disabled autokarma.
Thanks :)
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
Well, security fixes are one important point, possibly breaking things in a stable release another one (I tend to think everything is fixed now). Don't get me wrong, I don't want to slow down things, but NSS is a widely used library, even in essential system components. I don't like the idea of pushing it to stable without enough testing for some days (lets say 4-5 days at minimum), especially as there were other issues like non-working FreeIPA (which is fixed I guess?). First, the update has to be pushed to testing (not happened right now) and has to arrive @users, then they also should have some time to test. Some users like me also test ealier by downloading the stuff from Koji, but in general that is what we have updates-testing for. I'm a bit upset because I don't like the idea of forcing "that has to happen until Monday" when there might be side-effects like the one with FreeIPA. Firefox is not the only thing in the world. If Firefox update is extremely urgent it should get a bundled copy of nss until we have the update.
We should not break FreeIPA critically with that update (crashes on start or so) for sure. But don't expect rock solid experience from Fedora - there are RHEL/CentOS or other enterprise distros for that.
Breaking existing deployment should not be given a free ticket, I hope this is clear. Breaking things while developing is expected, of course.
Fedora is focused mainly on new features which is the nss 3.28.1 update and minor bugs should be addressed later.
Do it in Rawhide and nobody will complain.
Overall, I'm fine with how the nss upgrade was coordinated -- we've got proper fixes to unblock FreeIPA in time, for example. But this is not something that could be taken as granted, especially hurrying up before conferences' travel season for most of involved in this event -- devconf.cz and FOSDEM take enormous travel/dedication time and make fast turnaround for bug-fixing after the fact rather problematic.
On Sat, Jan 21, 2017 at 3:15 PM, Kai Engert kaie@kuix.de wrote:
On Sat, 2017-01-21 at 15:47 +0100, Christian Dersch wrote:
On 01/21/2017 03:44 PM, Christian Dersch wrote:
Autokarma just means the package will not be pushed automatically, but karma mechanism is still active. So once your update reached the stable karma level you defined, you can hit the push to stable button. If level is not reached you have to wait 7 days (but thats the case with autokarma too), but as the update contains prominent applications I don't expect it takes so long ;)
Just forgot: Until monday is a *short* time, we have weekend and the update did not reach testing yet. I suggest more time.
I've disabled autokarma.
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
Well it'll be in testing for those that are super desperate to get it so it's not like it's unavailable.
Now you see why I and some others asked you to wait for more testing: https://bugzilla.redhat.com/show_bug.cgi?id=1415137 So pushing on monday was really the wrong solution. Such things *have* to be handled more carefully!
On 01/21/2017 04:15 PM, Kai Engert wrote:
On Sat, 2017-01-21 at 15:47 +0100, Christian Dersch wrote:
On 01/21/2017 03:44 PM, Christian Dersch wrote:
Autokarma just means the package will not be pushed automatically, but karma mechanism is still active. So once your update reached the stable karma level you defined, you can hit the push to stable button. If level is not reached you have to wait 7 days (but thats the case with autokarma too), but as the update contains prominent applications I don't expect it takes so long ;)
Just forgot: Until monday is a *short* time, we have weekend and the update did not reach testing yet. I suggest more time.
I've disabled autokarma.
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Unless we want to delay Firefox 51, this update must go out earlier, or at the same as Firefox 51.
I suggest that we make a broad call for getting this update tested widely by Monday.
Kai _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On 01/21/2017 04:15 PM, Kai Engert wrote:
On Sat, 2017-01-21 at 15:47 +0100, Christian Dersch wrote:
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Going unsupported upstream is not the end of the world. If we keep 50.x for a few days, worst case scenario is to have to address a fresh critical security vulnerability and then you have two possible option: 1) rush to 51.x (with good reason, this time) 2) backport any fix to 50.x even if upstream does not (this was once normal way to maintain packages)
Regards.
On Tue, Jan 24, 2017 at 7:44 PM, Roberto Ragusa mail@robertoragusa.it wrote:
On 01/21/2017 04:15 PM, Kai Engert wrote:
On Sat, 2017-01-21 at 15:47 +0100, Christian Dersch wrote:
The current Firefox 50.x will go unsupported on Tuesday, replaced by Firefox 51. Usually new important security fixes are contained in the new Firefox update.
Going unsupported upstream is not the end of the world. If we keep 50.x for a few days, worst case scenario is to have to address a fresh critical security vulnerability and then you have two possible option:
- rush to 51.x (with good reason, this time)
- backport any fix to 50.x even if upstream does not (this was once normal way
to maintain packages)
3) Temporary bundle nss (like has been suggested in this thread before)
Christian Dersch wrote:
Autokarma just means the package will not be pushed automatically, but karma mechanism is still active. So once your update reached the stable karma level you defined, you can hit the push to stable button.
This may be true for Firefox because it is a critpath package and Bodhi special-cases critpath packages (last I was in this situation with a critpath package, it surprisingly worked), but in general, this has been broken in Bodhi for months: https://github.com/fedora-infra/bodhi/issues/1033
Kevin Kofler
On Sat, Jan 21, 2017 at 2:25 PM, Kai Engert kaie@kuix.de wrote:
On Sat, 2017-01-21 at 13:20 +0100, Christian Dersch wrote:
The combined updates have been submitted to testing: F24: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bbb320ba18 F25: https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
If you can, please help testing, thank in advance!
Please edit the default karma and put it higher than the default 3 so we don't get 3 people that test the one thing pushing it stable
Maybe disable autokarma? Then you can decide to push when you think it is well tested.
With autokarma disabled, is there a minimum test duration, before it can get pushed to stable?
Because we probably want to have it pushed to stable on Monday.
That doesn't give people a lot of time to test dependencies of nss which is not ideal given the depth/breath of possible impact that could have given that it's not even pushed to testing yet and it's the weekend.
On pe, 20 tammi 2017, Kai Engert wrote:
Hello,
we are currently dealing with a tricky situation, that the NSS and Mozilla package maintainers have been discussing, and I'd like to publish our plan.
The most recent NSS update, version 3.28.1, is required to ship to the Firefox 51 update planned for January 24.
Unfortunately, NSS 3.28.1 is incompatible with Mozilla applications version 50 and older.
If Mozilla 50 or older is used together with NSS 3.28 or newer, and the application attempts to use HTTP v2, the connections to some servers may fail (including connections to Google servers).
The fix is simple, it's possible to apply a small patch to the older Mozilla applications, to make it compatible with NSS 3.28.1
The difficulty here is the timing, and it's a conflict between "don't break applications in Fedora" and "ship new Firefox security update as soon as possible".
If we start by shipping NSS 3.28.1 first, without yet having fixed the Mozilla applications, then we allow Firefox 51 to be shipped, but we risk that the other applications aren't fixed in time, and that users might see regressions, caused by the upgrade to NSS 3.28.1
Alternatively, if we wait until all affected Mozilla packages have been updated to fixed versions, it might delay the January 24 Firefox 51 update.
After discussing this, we have a preference to avoid the breakage in Fedora, and try to ship all required updates as soon as possible.
In order to avoid the breakage, we want to add "Conflicts:" statements to the NSS 3.28.1 package, that makes it conflict with all known Mozilla packages that don't contain the required fix yet.
The packages we have identified are:
- firefox
- thunderbird
- seamonkey
- xulrunner
- icecat
I see that for all the above packages, build attempts that include the fix are already ongoing in koji, so there's hope that we might be able to resolve the situation in time.
FreeIPA is broken when trying to install with nss 3.28.1. We reliably reproduce this issue with https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
It seems that new nss also breaks 389-ds LDAP server's selection of available ciphers. As result, ldapsearch does not work against the 389-ds LDAP server configured as part of FreeIPA deployment.
However, if ANY of the above build cannot be completed soon, or, if ANY of the updates cannot move to the stable Fedora updates, it can block users from upgrading to the Firefox 51 update on Jan 24.
Is that acceptable?
I think failing server applications is unacceptable.
Do you agree that we make NSS conflict with any known incompatible packages mentioned above, and thereby may inhibit a subset of Fedora users from upgrading to Firefox 51 immediately?
If we can get all the above builds done quickly, and all of them pushed to Fedora stable updates quickly, we're good.
Note that we have the remaining risk that we haven't identified all Mozilla packages that might be affected. The relevant code isn't in NSS, but in Mozilla's network code. That means, if the above list of packages isn't the complete set of affected Mozilla based applications, other packages might still experience the connectivity regression. But as soon as another package is identified, it can rebuild to pick up the mentioned fix.
Thanks Kai
PS: Tracking bug is here: https://bugzilla.redhat.com/show_bug.cgi?id=1381400 (Don't get confused with the separate, unrelated discussion on TLS 1.3) An example of the regression is here: https://bugzilla.redhat.com/show_bug.cgi?id=1414929 _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On Fri, 2017-01-20 at 18:40 +0200, Alexander Bokovoy wrote:
FreeIPA is broken when trying to install with nss 3.28.1. We reliably reproduce this issue with https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
It seems that new nss also breaks 389-ds LDAP server's selection of available ciphers. As result, ldapsearch does not work against the 389-ds LDAP server configured as part of FreeIPA deployment.
However, if ANY of the above build cannot be completed soon, or, if ANY of the updates cannot move to the stable Fedora updates, it can block users from upgrading to the Firefox 51 update on Jan 24.
Is that acceptable?
I think failing server applications is unacceptable.
Alexander,
the test of NSS 3.28.1 in Fedora has uncovered multiple issues, and the issue with FreeIPA is a different issue than the one I had explained in this thread.
We'll prevent the FreeIPA issue, by disabling the experimental TLS 1.3 support at compile time in the Fedora NSS build, until the FreeIPA team is able to adjust their code to be compatible with TLS 1.3 support being enabled in NSS.
Thanks Kai
On pe, 20 tammi 2017, Kai Engert wrote:
On Fri, 2017-01-20 at 18:40 +0200, Alexander Bokovoy wrote:
FreeIPA is broken when trying to install with nss 3.28.1. We reliably reproduce this issue with https://bodhi.fedoraproject.org/updates/FEDORA-2017-e42b513012
It seems that new nss also breaks 389-ds LDAP server's selection of available ciphers. As result, ldapsearch does not work against the 389-ds LDAP server configured as part of FreeIPA deployment.
However, if ANY of the above build cannot be completed soon, or, if ANY of the updates cannot move to the stable Fedora updates, it can block users from upgrading to the Firefox 51 update on Jan 24.
Is that acceptable?
I think failing server applications is unacceptable.
Alexander,
the test of NSS 3.28.1 in Fedora has uncovered multiple issues, and the issue with FreeIPA is a different issue than the one I had explained in this thread.
We'll prevent the FreeIPA issue, by disabling the experimental TLS 1.3 support at compile time in the Fedora NSS build, until the FreeIPA team is able to adjust their code to be compatible with TLS 1.3 support being enabled in NSS.
Thanks, Kai.
-
Alexander Bokovoy -
Christian Dersch -
drago01 -
Hans de Goede -
Josh Boyer -
Kai Engert -
Kevin Kofler -
Martin Stransky -
Michael Catanzaro -
Michael Cronenworth -
Peter Robinson -
Roberto Ragusa