Is OpenSSL 3.0 still planned for Fedora 35?

[Miro Hrončok]

Hello Sahana, Ben, other Fedorans,

We have an accepted change proposal to include OpenSSL 3.0 in Fedora 35:

   https://fedoraproject.org/wiki/Changes/OpenSSL3.0

The contingency deadline is "Before release" which is not very specific.

At this point, we are not yet on OpenSSL 3.0 in Fedora 35.
What is the plan regarding the upgrade?

The change tracking bugzilla has a question posted in June: "OpenSSL3 beta is 
out, do we have some updated repository for testing in Fedora? What's the plan 
now?" But there is no answer.

The mass rebuild is over. Branching is in 1 week.
"Change Checkpoint: Completion deadline (testable)" is in 1 week.
The beta freeze is in 3 weeks.

Do we still plan to upgrade to OpenSSL 3.0 in Fedora 35? Should we defer this 
change to Fedora 36 instead?

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Neal Gompa]

On Mon, Aug 2, 2021 at 9:01 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  Hello Sahana, Ben, other Fedorans,
>
>  We have an accepted change proposal to include OpenSSL 3.0 in Fedora 35:
>
>     https://fedoraproject.org/wiki/Changes/OpenSSL3.0
>
>  The contingency deadline is "Before release" which is not very specific.
>
>  At this point, we are not yet on OpenSSL 3.0 in Fedora 35.
>  What is the plan regarding the upgrade?
>
>  The change tracking bugzilla has a question posted in June: "OpenSSL3 beta is
>  out, do we have some updated repository for testing in Fedora? What's the plan
>  now?" But there is no answer.
>
>  The mass rebuild is over. Branching is in 1 week.
>  "Change Checkpoint: Completion deadline (testable)" is in 1 week.
>  The beta freeze is in 3 weeks.
>
>  Do we still plan to upgrade to OpenSSL 3.0 in Fedora 35? Should we defer this
>  change to Fedora 36 instead?
>  
For reasons that are unclear to me, it seems that all the work for
this feature happened in CentOS Stream 9 development[1] instead of
Rawhide.

From what it can tell, the feature is fairly well-developed there, so
I would hope it can be easily brought over to Rawhide for F35.

[1]: https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commits/c9s/



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Florian Weimer]

* Neal Gompa:

>  For reasons that are unclear to me, it seems that all the work for
>  this feature happened in CentOS Stream 9 development[1] instead of
>  Rawhide. 
OpenSSL developers were under the impression that the Alpha snapshots
could not be imported into rawhide for policy reasons.  And in the end,
there was an ABI break between 3.0 Alpha and 3.0 Beta, which is why c9s
currently has a custom dual OpenSSL ABI (compatible with both Alpha and
Beta).

>  From what it can tell, the feature is fairly well-developed there,
> so
>  I would hope it can be easily brought over to Rawhide for F35.
>
>  [1]: https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commits/c9s/ 
This view misses the distribution-wide work to enable this, which still
hasn't concluded.

“Change Checkpoint: 100% Code Complete” is 2021-08-24.  I probably won't
have time to help out with this in the coming weeks.  Not sure about the
OpenSSL maintainers themselves.

We also need a different approach for Fedora with a compat -devel
package because there is just no way we can port everything within one
release cycle.

Thanks,
Florian

[Neal Gompa]

On Mon, Aug 2, 2021 at 10:31 AM Florian Weimer <fweimer(a)redhat.com&gt; wrote:
>  * Neal Gompa:
>
>  > For reasons that are unclear to me, it seems that all the work for
>  > this feature happened in CentOS Stream 9 development[1] instead of
>  > Rawhide.
>
>  OpenSSL developers were under the impression that the Alpha snapshots
>  could not be imported into rawhide for policy reasons.  And in the end,
>  there was an ABI break between 3.0 Alpha and 3.0 Beta, which is why c9s
>  currently has a custom dual OpenSSL ABI (compatible with both Alpha and
>  Beta).
>  
As proven time and again with glibc, this is obviously not true. Every
time an ABI break occurs, you need to plan a targeted mass rebuild,
but otherwise it's fine.

In any case, the OpenSSL ABI stabilized upstream with the 3.0 beta
release, didn't it?

OpenSSL 3.0 beta 2 was just tagged last week:
https://github.com/openssl/openssl/releases/tag/openssl-3.0.0-beta2

>  > From what it can tell, the feature is fairly well-developed
> there, so
>  > I would hope it can be easily brought over to Rawhide for F35.
>  >
>  > [1]: https://gitlab.com/redhat/centos-stream/rpms/openssl/-/commits/c9s/
>
>  This view misses the distribution-wide work to enable this, which still
>  hasn't concluded.
>
>  “Change Checkpoint: 100% Code Complete” is 2021-08-24.  I probably won't
>  have time to help out with this in the coming weeks.  Not sure about the
>  OpenSSL maintainers themselves.
>
>  We also need a different approach for Fedora with a compat -devel
>  package because there is just no way we can port everything within one
>  release cycle.
>  
We already have the compat package:
https://src.fedoraproject.org/rpms/openssl1.1

It is not required to port everything already.


--
真実はいつも一つ！/ Always, there's only one truth!

[Simo Sorce]

I think at this stage it may be safer to defer to F36, and land OpenSSL
3.0 in rawhide right after F35 forks out.

Simo.

On Mon, 2021-08-02 at 15:00 +0200, Miro Hrončok wrote:
>  Hello Sahana, Ben, other Fedorans,
>  
>  We have an accepted change proposal to include OpenSSL 3.0 in Fedora 35:
>  
>     https://fedoraproject.org/wiki/Changes/OpenSSL3.0
>  
>  The contingency deadline is "Before release" which is not very specific.
>  
>  At this point, we are not yet on OpenSSL 3.0 in Fedora 35.
>  What is the plan regarding the upgrade?
>  
>  The change tracking bugzilla has a question posted in June: "OpenSSL3 beta is 
>  out, do we have some updated repository for testing in Fedora? What's the plan 
>  now?" But there is no answer.
>  
>  The mass rebuild is over. Branching is in 1 week.
>  "Change Checkpoint: Completion deadline (testable)" is in 1 week.
>  The beta freeze is in 3 weeks.
>  
>  Do we still plan to upgrade to OpenSSL 3.0 in Fedora 35? Should we defer this 
>  change to Fedora 36 instead?
>  
>  -- 
>  Miro Hrončok
>  -- 
>  Phone: [phone]
>  IRC: mhroncok
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- 
Simo Sorce
RHEL Crypto Team
Red Hat, Inc

[Stephen Gallagher]

On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  I think at this stage it may be safer to defer to F36, and land OpenSSL
>  3.0 in rawhide right after F35 forks out.
>  
I'm generally in agreement here; I think it's too much risk too late
in the cycle. Could you re-propose the Change for F36?

[Neal Gompa]

On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  >
>  > I think at this stage it may be safer to defer to F36, and land OpenSSL
>  > 3.0 in rawhide right after F35 forks out.
>  >
>
>  I'm generally in agreement here; I think it's too much risk too late
>  in the cycle. Could you re-propose the Change for F36? 
I'm not sure I agree, but the Change owners can request the proposal
to be deferred to F36, which I *personally* would accept if
they intended to import OpenSSL 3.0 into Rawhide *right* after
branching. No more delaying it since it's clearly being done in RHEL
(which is already super-backwards to begin with). This Change has
already been deferred once (it was originally planned for F34). I
don't want it deferred again without a plan to work on it *in Fedora*.

Otherwise, just abandon the Change entirely.




--
真実はいつも一つ！/ Always, there's only one truth!

[Simo Sorce]

On Mon, 2021-08-02 at 17:43 -0400, Neal Gompa wrote:
>  On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher
> <sgallagh(a)redhat.com&gt; wrote:
>  > 
>  > On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > > 
>  > > I think at this stage it may be safer to defer to F36, and land OpenSSL
>  > > 3.0 in rawhide right after F35 forks out.
>  > > 
>  > 
>  > I'm generally in agreement here; I think it's too much risk too late
>  > in the cycle. Could you re-propose the Change for F36?
>  
>  I'm not sure I agree, but the Change owners can request the proposal
>  to be deferred to F36, which I *personally* would accept if
>  they intended to import OpenSSL 3.0 into Rawhide *right* after
>  branching. No more delaying it since it's clearly being done in RHEL
>  (which is already super-backwards to begin with). This Change has
>  already been deferred once (it was originally planned for F34). I
>  don't want it deferred again without a plan to work on it *in Fedora*.
>  
>  Otherwise, just abandon the Change entirely. 
Neal,
you are addressing this as if the OpenSSL maintainers are being
capricious.

We deferred the introduction of OpenSSL 3.0 in Fedora because we did
not want a mess in a distribution that is actually used, out of concern
for our users.

We can "dump" OpenSSL 3.0 in Fedora at any time, but we consciously
choose not to as to avoid pain for users. We cannot drop the Change
because we have to introduce OpenSSL 3.0 at some point, we just want to
introduce it when it's right for Fedora.

Simo.

>  
>  
>  
>  --
>  真実はいつも一つ！/ Always, there's only one truth!
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- 
Simo Sorce
RHEL Crypto Team
Red Hat, Inc

[Neal Gompa]

On Tue, Aug 3, 2021 at 5:59 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  On Mon, 2021-08-02 at 17:43 -0400, Neal Gompa wrote:
>  > On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  > >
>  > > On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > > >
>  > > > I think at this stage it may be safer to defer to F36, and land OpenSSL
>  > > > 3.0 in rawhide right after F35 forks out.
>  > > >
>  > >
>  > > I'm generally in agreement here; I think it's too much risk too late
>  > > in the cycle. Could you re-propose the Change for F36?
>  >
>  > I'm not sure I agree, but the Change owners can request the proposal
>  > to be deferred to F36, which I *personally* would accept if
>  > they intended to import OpenSSL 3.0 into Rawhide *right* after
>  > branching. No more delaying it since it's clearly being done in RHEL
>  > (which is already super-backwards to begin with). This Change has
>  > already been deferred once (it was originally planned for F34). I
>  > don't want it deferred again without a plan to work on it *in Fedora*.
>  >
>  > Otherwise, just abandon the Change entirely.
>
>  Neal,
>  you are addressing this as if the OpenSSL maintainers are being
>  capricious.
>
>  We deferred the introduction of OpenSSL 3.0 in Fedora because we did
>  not want a mess in a distribution that is actually used, out of concern
>  for our users.
>
>  We can "dump" OpenSSL 3.0 in Fedora at any time, but we consciously
>  choose not to as to avoid pain for users. We cannot drop the Change
>  because we have to introduce OpenSSL 3.0 at some point, we just want to
>  introduce it when it's right for Fedora.
>  
My irritation comes from the lack of communication from the Change
owner. This Change has already been deferred once (for good reason,
mind you). I'm annoyed that this is being deferred again because this
time the Change owner hasn't said *anything* at all. Everyone else
seems to be speaking (even Florian, which confuses me). I wouldn't
mind the Change being deferred again for solid technical reasons, but
I don't know how to trust that this Change is ever going to get done
because zero work happened and zero communication happened.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Simo Sorce]

On Tue, 2021-08-03 at 06:50 -0400, Neal Gompa wrote:
>  On Tue, Aug 3, 2021 at 5:59 AM Simo Sorce <simo(a)redhat.com&gt;
> wrote:
>  > 
>  > On Mon, 2021-08-02 at 17:43 -0400, Neal Gompa wrote:
>  > > On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher <sgallagh(a)redhat.com&gt;
> wrote:
>  > > > 
>  > > > On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > > > > 
>  > > > > I think at this stage it may be safer to defer to F36, and land
> OpenSSL
>  > > > > 3.0 in rawhide right after F35 forks out.
>  > > > > 
>  > > > 
>  > > > I'm generally in agreement here; I think it's too much risk too
> late
>  > > > in the cycle. Could you re-propose the Change for F36?
>  > > 
>  > > I'm not sure I agree, but the Change owners can request the proposal
>  > > to be deferred to F36, which I *personally* would accept if
>  > > they intended to import OpenSSL 3.0 into Rawhide *right* after
>  > > branching. No more delaying it since it's clearly being done in RHEL
>  > > (which is already super-backwards to begin with). This Change has
>  > > already been deferred once (it was originally planned for F34). I
>  > > don't want it deferred again without a plan to work on it *in Fedora*.
>  > > 
>  > > Otherwise, just abandon the Change entirely.
>  > 
>  > Neal,
>  > you are addressing this as if the OpenSSL maintainers are being
>  > capricious.
>  > 
>  > We deferred the introduction of OpenSSL 3.0 in Fedora because we did
>  > not want a mess in a distribution that is actually used, out of concern
>  > for our users.
>  > 
>  > We can "dump" OpenSSL 3.0 in Fedora at any time, but we consciously
>  > choose not to as to avoid pain for users. We cannot drop the Change
>  > because we have to introduce OpenSSL 3.0 at some point, we just want to
>  > introduce it when it's right for Fedora.
>  > 
>  
>  My irritation comes from the lack of communication from the Change
>  owner. This Change has already been deferred once (for good reason,
>  mind you). I'm annoyed that this is being deferred again because this
>  time the Change owner hasn't said *anything* at all. Everyone else
>  seems to be speaking (even Florian, which confuses me). I wouldn't
>  mind the Change being deferred again for solid technical reasons, but
>  I don't know how to trust that this Change is ever going to get done
>  because zero work happened and zero communication happened. 
The fact work isn't visible, doesn't mean nothing happened.

That said, upstream broke the ABI between alpha and beta1 so we are
very happy that we "have done nothing" in Fedora and delayed the
change.

Simo.

-- 
Simo Sorce
RHEL Crypto Team
Red Hat, Inc

[Neal Gompa]

On Tue, Aug 3, 2021 at 7:10 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  On Tue, 2021-08-03 at 06:50 -0400, Neal Gompa wrote:
>  > On Tue, Aug 3, 2021 at 5:59 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > >
>  > > On Mon, 2021-08-02 at 17:43 -0400, Neal Gompa wrote:
>  > > > On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher
> <sgallagh(a)redhat.com&gt; wrote:
>  > > > >
>  > > > > On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce <simo(a)redhat.com&gt;
> wrote:
>  > > > > >
>  > > > > > I think at this stage it may be safer to defer to F36, and land
> OpenSSL
>  > > > > > 3.0 in rawhide right after F35 forks out.
>  > > > > >
>  > > > >
>  > > > > I'm generally in agreement here; I think it's too much risk
> too late
>  > > > > in the cycle. Could you re-propose the Change for F36?
>  > > >
>  > > > I'm not sure I agree, but the Change owners can request the proposal
>  > > > to be deferred to F36, which I *personally* would accept if
>  > > > they intended to import OpenSSL 3.0 into Rawhide *right* after
>  > > > branching. No more delaying it since it's clearly being done in RHEL
>  > > > (which is already super-backwards to begin with). This Change has
>  > > > already been deferred once (it was originally planned for F34). I
>  > > > don't want it deferred again without a plan to work on it *in
> Fedora*.
>  > > >
>  > > > Otherwise, just abandon the Change entirely.
>  > >
>  > > Neal,
>  > > you are addressing this as if the OpenSSL maintainers are being
>  > > capricious.
>  > >
>  > > We deferred the introduction of OpenSSL 3.0 in Fedora because we did
>  > > not want a mess in a distribution that is actually used, out of concern
>  > > for our users.
>  > >
>  > > We can "dump" OpenSSL 3.0 in Fedora at any time, but we consciously
>  > > choose not to as to avoid pain for users. We cannot drop the Change
>  > > because we have to introduce OpenSSL 3.0 at some point, we just want to
>  > > introduce it when it's right for Fedora.
>  > >
>  >
>  > My irritation comes from the lack of communication from the Change
>  > owner. This Change has already been deferred once (for good reason,
>  > mind you). I'm annoyed that this is being deferred again because this
>  > time the Change owner hasn't said *anything* at all. Everyone else
>  > seems to be speaking (even Florian, which confuses me). I wouldn't
>  > mind the Change being deferred again for solid technical reasons, but
>  > I don't know how to trust that this Change is ever going to get done
>  > because zero work happened and zero communication happened.
>
>  The fact work isn't visible, doesn't mean nothing happened.
>  
To most people, it appears that nothing has happened, yes. It would
have been nice to know that stuff happened.

>  That said, upstream broke the ABI between alpha and beta1 so we are
>  very happy that we "have done nothing" in Fedora and delayed the
>  change.
>  
Sure, but the Change proposal[1] explicitly says that the work would
start *after* the beta release in June. The beta release came out June
17[2], and nothing happened afterward in Fedora, presumably because
Sahana was working on rebasing to it in CentOS Stream 9, which
completed a month later[3]. Note that now Beta 2 came out a week
ago[4], which seems to carry *some* ABI stability (which is a surprise
to me, honestly...).

From my naive point of view, once that rebase work was complete, I
would have expected the same effort to land in Fedora, since we
already had the openssl1.1 compatibility package created a year
ago[5]. Then we could have integrated it as part of the mass build
last week instead of needing a targeted rebuild for it in a side-tag.

For what it's worth, there seem to be 633 source packages that produce
940 binary packages that link to OpenSSL, so it's not *that* crazy to
do a targeted rebuild:

>  ngompa@localhost ~> sudo dnf -q repoquery --qf
> "%{SOURCERPM}" --whatdepends openssl-libs --latest=1 --exclude=\*.i686 | wc -l
>  633
>  ngompa@localhost ~> sudo dnf -q repoquery --whatdepends openssl-libs --latest=1
> --exclude=\*.i686 | wc -l
>  940 
Basically, my problem is that I don't think Sahana was prepared on how
to handle doing this work properly and they just need to be aware that
communication is extremely important when doing stuff like this.
Sahara took this over from Tomáš Mráz, who left Red Hat to work for
the OpenSSL Foundation on OpenSSL full-time. In that transfer, I don't
think anyone educated Sahana on how to handle Fedora Changes.

However, regardless of all the previous mistakes, I still don't think
that OpenSSL 3.0 necessarily needs to be skipped for Fedora Linux 35
based on all that. What *would* concern me is OpenSSL 3.0's own
release schedule, or rather the lack of one. It is unclear when
OpenSSL 3.0 final is supposed to be released. Digging into the
upstream project information, it seems like there's not much left[6].
But I'm unsure if they do relatively fixed milestones or very fluid
milestones. That's also combined with no estimates of when OpenSSL 3.0
final is supposed to be released, at least none that I can find.

[1]: https://fedoraproject.org/wiki/Changes/OpenSSL3.0
[2]: https://www.openssl.org/blog/blog/2021/06/17/OpenSSL3.0ReleaseCandidate/
[3]: https://gitlab.com/redhat/centos-stream/rpms/openssl/-/merge_requests/15
[4]: https://github.com/openssl/openssl/releases/tag/openssl-3.0.0-beta2
[5]: https://src.fedoraproject.org/rpms/openssl1.1
[6]: https://github.com/openssl/openssl/milestone/15



--
真実はいつも一つ！/ Always, there's only one truth!

[Robbie Harwood]

Neal Gompa <ngompa13(a)gmail.com&gt; writes:

>  Basically, my problem is that I don't think Sahana was prepared
> on how
>  to handle doing this work properly and they just need to be aware that
>  communication is extremely important when doing stuff like this.
>  Sahara took this over from Tomáš Mráz, who left Red Hat to work for
>  the OpenSSL Foundation on OpenSSL full-time. In that transfer, I don't
>  think anyone educated Sahana on how to handle Fedora Changes. 
Hey, this ^ isn't okay.  We don't talk to each other like this.

Thanks,
--Robbie

[Simo Sorce]

On Tue, 2021-08-03 at 07:52 -0400, Neal Gompa wrote:
>  On Tue, Aug 3, 2021 at 7:10 AM Simo Sorce <simo(a)redhat.com&gt;
> wrote:
>  > 
>  > On Tue, 2021-08-03 at 06:50 -0400, Neal Gompa wrote:
>  > > On Tue, Aug 3, 2021 at 5:59 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > > > 
>  > > > On Mon, 2021-08-02 at 17:43 -0400, Neal Gompa wrote:
>  > > > > On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher
> <sgallagh(a)redhat.com&gt; wrote:
>  > > > > > 
>  > > > > > On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce
> <simo(a)redhat.com&gt; wrote:
>  > > > > > > 
>  > > > > > > I think at this stage it may be safer to defer to F36, and
> land OpenSSL
>  > > > > > > 3.0 in rawhide right after F35 forks out.
>  > > > > > > 
>  > > > > > 
>  > > > > > I'm generally in agreement here; I think it's too much
> risk too late
>  > > > > > in the cycle. Could you re-propose the Change for F36?
>  > > > > 
>  > > > > I'm not sure I agree, but the Change owners can request the
> proposal
>  > > > > to be deferred to F36, which I *personally* would accept if
>  > > > > they intended to import OpenSSL 3.0 into Rawhide *right* after
>  > > > > branching. No more delaying it since it's clearly being done in
> RHEL
>  > > > > (which is already super-backwards to begin with). This Change has
>  > > > > already been deferred once (it was originally planned for F34). I
>  > > > > don't want it deferred again without a plan to work on it *in
> Fedora*.
>  > > > > 
>  > > > > Otherwise, just abandon the Change entirely.
>  > > > 
>  > > > Neal,
>  > > > you are addressing this as if the OpenSSL maintainers are being
>  > > > capricious.
>  > > > 
>  > > > We deferred the introduction of OpenSSL 3.0 in Fedora because we did
>  > > > not want a mess in a distribution that is actually used, out of concern
>  > > > for our users.
>  > > > 
>  > > > We can "dump" OpenSSL 3.0 in Fedora at any time, but we
> consciously
>  > > > choose not to as to avoid pain for users. We cannot drop the Change
>  > > > because we have to introduce OpenSSL 3.0 at some point, we just want to
>  > > > introduce it when it's right for Fedora.
>  > > > 
>  > > 
>  > > My irritation comes from the lack of communication from the Change
>  > > owner. This Change has already been deferred once (for good reason,
>  > > mind you). I'm annoyed that this is being deferred again because this
>  > > time the Change owner hasn't said *anything* at all. Everyone else
>  > > seems to be speaking (even Florian, which confuses me). I wouldn't
>  > > mind the Change being deferred again for solid technical reasons, but
>  > > I don't know how to trust that this Change is ever going to get done
>  > > because zero work happened and zero communication happened.
>  > 
>  > The fact work isn't visible, doesn't mean nothing happened.
>  > 
>  
>  To most people, it appears that nothing has happened, yes. It would
>  have been nice to know that stuff happened. 

Lemme push back on you right here.

Did you express interest in this change before?
Did you write to Sahana when you realized you had not seen the
communication you wanted?

>  > That said, upstream broke the ABI between alpha and beta1 so we
> are
>  > very happy that we "have done nothing" in Fedora and delayed the
>  > change.
>  > 
>  
>  Sure, but the Change proposal[1] explicitly says that the work would
>  start *after* the beta release in June. The beta release came out June
>  17[2], and nothing happened afterward in Fedora, presumably because
>  Sahana was working on rebasing to it in CentOS Stream 9, which
>  completed a month later[3]. Note that now Beta 2 came out a week
>  ago[4], which seems to carry *some* ABI stability (which is a surprise
>  to me, honestly...).
>  
>  From my naive point of view, once that rebase work was complete, I
>  would have expected the same effort to land in Fedora, since we
>  already had the openssl1.1 compatibility package created a year
>  ago[5]. Then we could have integrated it as part of the mass build
>  last week instead of needing a targeted rebuild for it in a side-tag. 
It would have been nice, but also a disaster to rush changes into
Fedora. There was also no strong need.

We are aware of exactly ZERO packages clamoring to get OpenSSL 3.0 in
Fedora right now.


>  For what it's worth, there seem to be 633 source packages that
> produce
>  940 binary packages that link to OpenSSL, so it's not *that* crazy to
>  do a targeted rebuild:
>  
>  > ngompa@localhost ~> sudo dnf -q repoquery --qf "%{SOURCERPM}"
> --whatdepends openssl-libs --latest=1 --exclude=\*.i686 | wc -l
>  > 633
>  > ngompa@localhost ~> sudo dnf -q repoquery --whatdepends openssl-libs --latest=1
> --exclude=\*.i686 | wc -l
>  > 940 
It is crazy, with a lot less packages we had a lot of work in
Centos9/RHEL9, all that work is going to benefit Fedora for once,
hopefully, but it is still work that needs to be done.

We did not want to throw work on maintainers without knowing there are
good solutions, especially at the last minute.

>  Basically, my problem is that I don't think Sahana was prepared
> on how
>  to handle doing this work properly and they just need to be aware that
>  communication is extremely important when doing stuff like this.
>  Sahara took this over from Tomáš Mráz, who left Red Hat to work for
>  the OpenSSL Foundation on OpenSSL full-time. In that transfer, I don't
>  think anyone educated Sahana on how to handle Fedora Changes. 

My problem is that you are throwing baseless accusations, and for no
good reason.
*nothing* happened, it means we did not destabilize Fedora, nor caused
unnecessary work to anyone. sure we did not go around telling everyone
that nothing was happening, sorry but "Big deal!".

Communication is important if we are going to break stuff, but luckily
we decided to do a lot of legwork bfore throwing stuff in,
unfortunately all this work and the delays upstream meant we ultimately
decided it was more prudent to delay the change in Fedora. I call this
handling the change very appropriately, and I think you should
apologize to Sahana.

>  However, regardless of all the previous mistakes, 
What mistakes? 


>   I still don't think
>  that OpenSSL 3.0 necessarily needs to be skipped for Fedora Linux 35
>  based on all that. 
So you now think you can make better choices than the maintainers and
people that closely worked on this?

>   What *would* concern me is OpenSSL 3.0's own
>  release schedule, or rather the lack of one. 
Oh boy, like we do not have this problem with like 98% of the Open
Source projects we use in Fedora? Why is OpenSSL special now?

>   It is unclear when
>  OpenSSL 3.0 final is supposed to be released. Digging into the
>  upstream project information, it seems like there's not much left[6].
>  But I'm unsure if they do relatively fixed milestones or very fluid
>  milestones. That's also combined with no estimates of when OpenSSL 3.0
>  final is supposed to be released, at least none that I can find. 

This is exactly why we have been very cautios and not cavalier with
this change. We planned, we worked on it, and ultimately quitely
decided to postpone instead of inflicting a lot of pain on our
developers and users. I call this considerate, and not a mistake.

Yeah we did not constantly tell you "not yet", "not yet", "not
yet".

Sorry if we've been busy, but accusing people of nefarious behavior for
nothing, is not exactly friendly.

Thanks for your understanding ... or lack thereof.

Simo.

>  [1]: https://fedoraproject.org/wiki/Changes/OpenSSL3.0
>  [2]: https://www.openssl.org/blog/blog/2021/06/17/OpenSSL3.0ReleaseCandidate/
>  [3]: https://gitlab.com/redhat/centos-stream/rpms/openssl/-/merge_requests/15
>  [4]: https://github.com/openssl/openssl/releases/tag/openssl-3.0.0-beta2
>  [5]: https://src.fedoraproject.org/rpms/openssl1.1
>  [6]: https://github.com/openssl/openssl/milestone/15
>  
>  
>  
>  --
>  真実はいつも一つ！/ Always, there's only one truth!
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- 
Simo Sorce
RHEL Crypto Team
Red Hat, Inc

[Ben Cotton]

Okay, let's take a step back for a moment.

Sahana, do you still intend to land OpenSSL 3.0 in F35 or should we
defer it to F36? If you do plan on including it in F35, what help from
the rest of the community (if any) do you need?

Anything that's not a direct answer to the above questions should sit
in the drafts folder for a bit while we all take some deep breaths.


Thanks,
BC

-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis

[Neal Gompa]

On Tue, Aug 3, 2021 at 10:42 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  On Tue, 2021-08-03 at 07:52 -0400, Neal Gompa wrote:
>  > On Tue, Aug 3, 2021 at 7:10 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > >
>  > > On Tue, 2021-08-03 at 06:50 -0400, Neal Gompa wrote:
>  > > > On Tue, Aug 3, 2021 at 5:59 AM Simo Sorce <simo(a)redhat.com&gt; wrote:
>  > > > >
>  > > > > On Mon, 2021-08-02 at 17:43 -0400, Neal Gompa wrote:
>  > > > > > On Mon, Aug 2, 2021 at 5:39 PM Stephen Gallagher
> <sgallagh(a)redhat.com&gt; wrote:
>  > > > > > >
>  > > > > > > On Mon, Aug 2, 2021 at 11:11 AM Simo Sorce
> <simo(a)redhat.com&gt; wrote:
>  > > > > > > >
>  > > > > > > > I think at this stage it may be safer to defer to F36,
> and land OpenSSL
>  > > > > > > > 3.0 in rawhide right after F35 forks out.
>  > > > > > > >
>  > > > > > >
>  > > > > > > I'm generally in agreement here; I think it's too
> much risk too late
>  > > > > > > in the cycle. Could you re-propose the Change for F36?
>  > > > > >
>  > > > > > I'm not sure I agree, but the Change owners can request the
> proposal
>  > > > > > to be deferred to F36, which I *personally* would accept if
>  > > > > > they intended to import OpenSSL 3.0 into Rawhide *right* after
>  > > > > > branching. No more delaying it since it's clearly being done
> in RHEL
>  > > > > > (which is already super-backwards to begin with). This Change
> has
>  > > > > > already been deferred once (it was originally planned for F34).
> I
>  > > > > > don't want it deferred again without a plan to work on it
> *in Fedora*.
>  > > > > >
>  > > > > > Otherwise, just abandon the Change entirely.
>  > > > >
>  > > > > Neal,
>  > > > > you are addressing this as if the OpenSSL maintainers are being
>  > > > > capricious.
>  > > > >
>  > > > > We deferred the introduction of OpenSSL 3.0 in Fedora because we did
>  > > > > not want a mess in a distribution that is actually used, out of
> concern
>  > > > > for our users.
>  > > > >
>  > > > > We can "dump" OpenSSL 3.0 in Fedora at any time, but we
> consciously
>  > > > > choose not to as to avoid pain for users. We cannot drop the Change
>  > > > > because we have to introduce OpenSSL 3.0 at some point, we just want
> to
>  > > > > introduce it when it's right for Fedora.
>  > > > >
>  > > >
>  > > > My irritation comes from the lack of communication from the Change
>  > > > owner. This Change has already been deferred once (for good reason,
>  > > > mind you). I'm annoyed that this is being deferred again because this
>  > > > time the Change owner hasn't said *anything* at all. Everyone else
>  > > > seems to be speaking (even Florian, which confuses me). I wouldn't
>  > > > mind the Change being deferred again for solid technical reasons, but
>  > > > I don't know how to trust that this Change is ever going to get done
>  > > > because zero work happened and zero communication happened.
>  > >
>  > > The fact work isn't visible, doesn't mean nothing happened.
>  > >
>  >
>  > To most people, it appears that nothing has happened, yes. It would
>  > have been nice to know that stuff happened.
>
>
>  Lemme push back on you right here.
>
>  Did you express interest in this change before?
>  Did you write to Sahana when you realized you had not seen the
>  communication you wanted?
>  
Hm, that's fair. I should have written to Sahana before this point. I
had privately asked about this before months ago, but I should have
followed up again last month when I saw the MRs landing in CentOS
Stream 9. That is definitely my bad.

>  > > That said, upstream broke the ABI between alpha and beta1
> so we are
>  > > very happy that we "have done nothing" in Fedora and delayed the
>  > > change.
>  > >
>  >
>  > Sure, but the Change proposal[1] explicitly says that the work would
>  > start *after* the beta release in June. The beta release came out June
>  > 17[2], and nothing happened afterward in Fedora, presumably because
>  > Sahana was working on rebasing to it in CentOS Stream 9, which
>  > completed a month later[3]. Note that now Beta 2 came out a week
>  > ago[4], which seems to carry *some* ABI stability (which is a surprise
>  > to me, honestly...).
>  >
>  > From my naive point of view, once that rebase work was complete, I
>  > would have expected the same effort to land in Fedora, since we
>  > already had the openssl1.1 compatibility package created a year
>  > ago[5]. Then we could have integrated it as part of the mass build
>  > last week instead of needing a targeted rebuild for it in a side-tag.
>
>  It would have been nice, but also a disaster to rush changes into
>  Fedora. There was also no strong need.
>
>  We are aware of exactly ZERO packages clamoring to get OpenSSL 3.0 in
>  Fedora right now.
>  
Honestly? The reason I want OpenSSL 3.0 is the license change. I don't
really care about anything else.

>  > For what it's worth, there seem to be 633 source packages that produce
>  > 940 binary packages that link to OpenSSL, so it's not *that* crazy to
>  > do a targeted rebuild:
>  >
>  > > ngompa@localhost ~> sudo dnf -q repoquery --qf "%{SOURCERPM}"
> --whatdepends openssl-libs --latest=1 --exclude=\*.i686 | wc -l
>  > > 633
>  > > ngompa@localhost ~> sudo dnf -q repoquery --whatdepends openssl-libs
> --latest=1 --exclude=\*.i686 | wc -l
>  > > 940
>
>  It is crazy, with a lot less packages we had a lot of work in
>  Centos9/RHEL9, all that work is going to benefit Fedora for once,
>  hopefully, but it is still work that needs to be done.
>
>  We did not want to throw work on maintainers without knowing there are
>  good solutions, especially at the last minute.
>  
That's fair.

>  > Basically, my problem is that I don't think Sahana was
> prepared on how
>  > to handle doing this work properly and they just need to be aware that
>  > communication is extremely important when doing stuff like this.
>  > Sahara took this over from Tomáš Mráz, who left Red Hat to work for
>  > the OpenSSL Foundation on OpenSSL full-time. In that transfer, I don't
>  > think anyone educated Sahana on how to handle Fedora Changes.
>
>
>  My problem is that you are throwing baseless accusations, and for no
>  good reason.
>  *nothing* happened, it means we did not destabilize Fedora, nor caused
>  unnecessary work to anyone. sure we did not go around telling everyone
>  that nothing was happening, sorry but "Big deal!".
>
>  Communication is important if we are going to break stuff, but luckily
>  we decided to do a lot of legwork bfore throwing stuff in,
>  unfortunately all this work and the delays upstream meant we ultimately
>  decided it was more prudent to delay the change in Fedora. I call this
>  handling the change very appropriately, and I think you should
>  apologize to Sahana.
>
>  > However, regardless of all the previous mistakes,
>
>  What mistakes?
>  
I'm not actually sure what I meant here. I thought I deleted that when
drafting it. Oops.

>  >  I still don't think
>  > that OpenSSL 3.0 necessarily needs to be skipped for Fedora Linux 35
>  > based on all that.
>
>  So you now think you can make better choices than the maintainers and
>  people that closely worked on this?
>  
Of course not.

>  >  What *would* concern me is OpenSSL 3.0's own
>  > release schedule, or rather the lack of one.
>
>  Oh boy, like we do not have this problem with like 98% of the Open
>  Source projects we use in Fedora? Why is OpenSSL special now?
>  
It's special because we as a distribution made it special by making it
a requirement for RPM to function. OpenSSL infamously didn't offer a
stable ABI across releases, so I'm cognizant of that issue potentially
plaguing us if we released with a beta release and there was an ABI
break between beta and stable. I couldn't find any documentation of
what stage they finalized the ABI at for releases.

>  >  It is unclear when
>  > OpenSSL 3.0 final is supposed to be released. Digging into the
>  > upstream project information, it seems like there's not much left[6].
>  > But I'm unsure if they do relatively fixed milestones or very fluid
>  > milestones. That's also combined with no estimates of when OpenSSL 3.0
>  > final is supposed to be released, at least none that I can find.
>
>
>  This is exactly why we have been very cautios and not cavalier with
>  this change. We planned, we worked on it, and ultimately quitely
>  decided to postpone instead of inflicting a lot of pain on our
>  developers and users. I call this considerate, and not a mistake.
>
>  Yeah we did not constantly tell you "not yet", "not yet", "not
> yet".
>
>  Sorry if we've been busy, but accusing people of nefarious behavior for
>  nothing, is not exactly friendly.
>
>  Thanks for your understanding ... or lack thereof.
>  
I am not accusing anyone of nefarious behavior. What I am saying is
that absent any communication, the whole thing looks very weird.

It seemed like from my naive view, it seemed like all that stuff was
already getting done fairly aggressively in CentOS Stream and nothing
was happening in Fedora for an approved Change that was already
delayed once.

The inversion for development is a smell that I am very sensitive to,
as the last time that happened, things went very badly. I also don't
want Changes going in where people aren't prepared to communicate as a
Change is being implemented. I've personally been burned by this on
multiple separate occasions and I'm extremely sensitive to that.

Sahana, if I've offended or insulted you, I'm *extremely* sorry. I am
just trying to make sense of a weird situation and I'm probably not
taking it very well.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Sahana Prasad]

On Tue, Aug 3, 2021 at 4:50 PM Ben Cotton <bcotton(a)redhat.com&gt; wrote:

>  Okay, let's take a step back for a moment.
>
>  Sahana, do you still intend to land OpenSSL 3.0 in F35 or should we
>  defer it to F36? 

Hi Ben, others

I would like to defer it to F36. OpenSSL 3.0 Beta2 (or Beta 3, if
available) will land
into rawhide right after branching. This will give fellow maintainers time
to properly plan
porting activities and ensure smooth transition.


>  If you do plan on including it in F35, what help from
>  the rest of the community (if any) do you need? 

 Like Simo mentioned, we haven't had any request/interest for OpenSSL 3.0
to be introduced
into F35 immediately. However, if it would be very beneficial, then I'd say
we could try the following -
(Only if the package maintainers are willing to add patches / merge their
WIP OpenSSL 3.0 branches before 8/24)

1.  Do a targeted rebuild of packages that depend on OpenSSL on the
side-tag f35-build-side-44202.
     I have built OpenSSL 3.0 Beta2 [1] and openssl1.1 compat [2] packages
in this side-tag.
     The compat package is rebased to 1.1.1k. It includes the devel
subpackage.
     Note that it intentionally conflicts with openssl-devel.
     Neal, or others willing to volunteer, could help me out here to get
the list of packages that fail.
     (If it was done in the past, there could be some automated scripts we
could run overnight to get results quickly.)
     We had ~60 packages that FTBFS when OpenSSL 3.0 landed in c9s.
     (Most of them are already ported to 3.0 now, so we would gain from
them)

2.   We can report FTBTS bugs to these failing packages.

3.   If the failure rate of packages is within permissible limits and can
all be fixed before 8/24, then good.

I wanted to do these steps myself, but not for F35, and not immediately.
I'm sure trying step1 will
not be a waste of time and we would only gain from it even if we defer to
F36.

Package maintainers who want to test building their package with OpenSSL
3.0 Beta2 version
can do so using this copr repo [3] and add builds. Kindly let me know if
you need any help in testing/building
your package with OpenSSL 3.0. Code is available here [4]

Neal, as far as communication is concerned, I might not have communicated
on this list directly,
but I have definitely discussed it on multiple occasions in different
channels on how best to bring
OpenSSL 3.0 into rawhide. Other than regressions and instability being the
technical reasons,
there was no other reason to stall communication.
I am learning everyday, and I will definitely work on improving my
communication in the future.
I apologize for any inconvenience caused.


[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=73215212
[2] https://koji.fedoraproject.org/koji/taskinfo?taskID=73219610
[3] https://copr.fedorainfracloud.org/coprs/saprasad/openssl-3.0/builds/
[4] https://src.fedoraproject.org/fork/saprasad/rpms/openssl/tree/rawhide

Looking forward to working together.

Thank you,
Regards,
Sahana Prasad



>  Anything that's not a direct answer to the above questions should sit
>  in the drafts folder for a bit while we all take some deep breaths.
>
>
>  Thanks,
>  BC
>
>  --
>  Ben Cotton
>  He / Him / His
>  Fedora Program Manager
>  Red Hat
>  TZ=America/Indiana/Indianapolis
>
>

[Ben Cotton]

On Tue, Aug 3, 2021 at 6:17 PM Sahana Prasad <sahana(a)redhat.com&gt; wrote:
>  I would like to defer it to F36. OpenSSL 3.0 Beta2 (or Beta 3, if available) will land
>  into rawhide right after branching. This will give fellow maintainers time to properly
> plan
>  porting activities and ensure smooth transition.
>  
Okay, I'll take care of the paperwork to make that happen. :-)


Thanks,
BC

-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis

[Miro Hrončok]

On 04. 08. 21 0:16, Sahana Prasad wrote:
>  Hi Ben, others
>  
>  I would like to defer it to F36. 
Thanks for the confirmation.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok