switching libcurl back to OpenSSL and providing the libcurl-minimal subpackage

[Kamil Dudka]

In order to make even smaller Fedora base images, it was proposed to switch
libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
deprecated and libcurl is the only package that pulls NSS as its dependency
into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
could create Fedora base image that contains fewer crypto libraries inside.

Additional proposal that would help to reduce the size of base image is the
libcurl-minimal subpackage, which can be installed installed as a lightweight
replacement of the libcurl package, with smaller size and fewer dependencies.
The libcurl-minimal subpackage was already proposed on this mailing list one
year ago but at that time not many people knew that it would be useful today:

   
https://lists.fedoraproject.org/archives/list/[e-mail].o...

I am CCing the actual requestors so they can fight for these proposals.

A proof of concept is available in the private-kdudka-libcurl-minimal branch:

   
http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-li...

I have also prepared Copr repositories for f24+:

    https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/

Any feedback is welcome!

Kamil

[Colin Walters]

On Wed, Apr 5, 2017, at 09:52 AM, Kamil Dudka wrote:
>  In order to make even smaller Fedora base images, it was proposed to
> switch
>  libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
>  motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
>  deprecated and libcurl is the only package that pulls NSS as its dependency
>  into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
>  could create Fedora base image that contains fewer crypto libraries inside. 
Makes sense to me - from the Atomic Host perspective, we are switching
ostree to use libcurl, since libdnf already does (and librepo hard depends
on OpenSSL, even though libcurl used NSS).

>  Additional proposal that would help to reduce the size of base image
> is the
>  libcurl-minimal subpackage, which can be installed installed as a lightweight
>  replacement of the libcurl package, with smaller size and fewer dependencies. 
I'm in agreement with this except:

# configure minimal build
...
        --without-nghttp2

I'd really prefer to keep HTTP2 available by default - it can be dramatically
better.

[Stephen Gallagher]

On 04/05/2017 09:59 AM, Colin Walters wrote:
>  
>  
>  On Wed, Apr 5, 2017, at 09:52 AM, Kamil Dudka wrote:
> > In order to make even smaller Fedora base images, it was proposed to switch
> > libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
> > motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
> > deprecated and libcurl is the only package that pulls NSS as its dependency
> > into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
> > could create Fedora base image that contains fewer crypto libraries inside.
>  
>  Makes sense to me - from the Atomic Host perspective, we are switching
>  ostree to use libcurl, since libdnf already does (and librepo hard depends
>  on OpenSSL, even though libcurl used NSS).
>  
> > Additional proposal that would help to reduce the size of base image is the
> > libcurl-minimal subpackage, which can be installed installed as a lightweight
> > replacement of the libcurl package, with smaller size and fewer dependencies.
>  
>  I'm in agreement with this except:
>  
>  # configure minimal build
>  ...
>          --without-nghttp2
>  
>  I'd really prefer to keep HTTP2 available by default - it can be dramatically
>  better. 

I'll second this: it looks like libnghttp2 does not pull in any dependencies
that wouldn't already be part of any minimal install (just glibc and ld) and its
filesystem space is only about 150k uncompressed.

It's probably reasonable to keep this in our minimal set for the HTTP2
functionality.

[Jan Kurik]

Might not be directly related, but just for a reference - one of the
F26 Changes (currently deferred to F27) is doing the same for
OpenLDAP: https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL

Regards,
Jan

On Wed, Apr 5, 2017 at 4:33 PM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  On 04/05/2017 09:59 AM, Colin Walters wrote:
> >
> >
> > On Wed, Apr 5, 2017, at 09:52 AM, Kamil Dudka wrote:
> >> In order to make even smaller Fedora base images, it was proposed to switch
> >> libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
> >> motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
> >> deprecated and libcurl is the only package that pulls NSS as its dependency
> >> into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
> >> could create Fedora base image that contains fewer crypto libraries inside.
> >
> > Makes sense to me - from the Atomic Host perspective, we are switching
> > ostree to use libcurl, since libdnf already does (and librepo hard depends
> > on OpenSSL, even though libcurl used NSS).
> >
> >> Additional proposal that would help to reduce the size of base image is the
> >> libcurl-minimal subpackage, which can be installed installed as a lightweight
> >> replacement of the libcurl package, with smaller size and fewer dependencies.
> >
> > I'm in agreement with this except:
> >
> > # configure minimal build
> > ...
> >         --without-nghttp2
> >
> > I'd really prefer to keep HTTP2 available by default - it can be dramatically
> > better.
>
>
>  I'll second this: it looks like libnghttp2 does not pull in any dependencies
>  that wouldn't already be part of any minimal install (just glibc and ld) and its
>  filesystem space is only about 150k uncompressed.
>
>  It's probably reasonable to keep this in our minimal set for the HTTP2
>  functionality.
>
>
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  


-- 
Jan Kuřík
Platform & Fedora Program Manager
Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic

[Kamil Dudka]

On Wednesday, April 05, 2017 17:09:34 Jan Kurik wrote:
>  Might not be directly related, but just for a reference - one of the
>  F26 Changes (currently deferred to F27) is doing the same for
>  OpenLDAP: https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL 
I have prepared a draft of the change proposal.  Could you please have a look?

https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL

Thanks in advance!

Kamil

>  Regards,
>  Jan
>  
>  On Wed, Apr 5, 2017 at 4:33 PM, Stephen Gallagher <sgallagh(a)redhat.com&gt; 
wrote:
> > On 04/05/2017 09:59 AM, Colin Walters wrote:
> >> On Wed, Apr 5, 2017, at 09:52 AM, Kamil Dudka wrote:
> >>> In order to make even smaller Fedora base images, it was proposed to
> >>> switch
> >>> libcurl back to OpenSSL.  The Fedora Crypto Consolidation project,
which
> >>> motivated the switch of libcurl from OpenSSL to NSS ten years ago, is
> >>> now
> >>> deprecated and libcurl is the only package that pulls NSS as its
> >>> dependency
> >>> into the Fedora base image.  Hence, by switching libcurl back to
> >>> OpenSSL, we could create Fedora base image that contains fewer crypto
> >>> libraries inside.>> 
> >> Makes sense to me - from the Atomic Host perspective, we are switching
> >> ostree to use libcurl, since libdnf already does (and librepo hard
> >> depends
> >> on OpenSSL, even though libcurl used NSS).
> >> 
> >>> Additional proposal that would help to reduce the size of base image is
> >>> the
> >>> libcurl-minimal subpackage, which can be installed installed as a
> >>> lightweight replacement of the libcurl package, with smaller size and
> >>> fewer dependencies.>> 
> >> I'm in agreement with this except:
> >> 
> >> # configure minimal build
> >> ...
> >> 
> >>         --without-nghttp2
> >> 
> >> I'd really prefer to keep HTTP2 available by default - it can be
> >> dramatically better.
> > 
> > I'll second this: it looks like libnghttp2 does not pull in any
> > dependencies that wouldn't already be part of any minimal install (just
> > glibc and ld) and its filesystem space is only about 150k uncompressed.
> > 
> > It's probably reasonable to keep this in our minimal set for the HTTP2
> > functionality.
> > 
> > 
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org

[Jan Kurik]

On Thu, Apr 6, 2017 at 2:49 PM, Kamil Dudka <kdudka(a)redhat.com&gt; wrote:
>  On Wednesday, April 05, 2017 17:09:34 Jan Kurik wrote:
> > Might not be directly related, but just for a reference - one of the
> > F26 Changes (currently deferred to F27) is doing the same for
> > OpenLDAP: https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL
>
>  I have prepared a draft of the change proposal.  Could you please have a look?
>
>  https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL
>
>  Thanks in advance! 
The "Scope" section should contain description of tasks/actions which
need to be done by the Change owner, Other developers, etc. May I ask
you please to update this section as expected ?
Also, in case there is any action expected from RelEng, please open a
ticket in https://pagure.io/releng/issues and put the ticket number
into the Change page.

Otherwise it looks good.

Thanks,
Jan

[Kamil Dudka]

On Thursday, April 06, 2017 15:00:31 Jan Kurik wrote:
>  On Thu, Apr 6, 2017 at 2:49 PM, Kamil Dudka <kdudka(a)redhat.com&gt;
> wrote:
>  > On Wednesday, April 05, 2017 17:09:34 Jan Kurik wrote:
>  >> Might not be directly related, but just for a reference - one of the
>  >> F26 Changes (currently deferred to F27) is doing the same for
>  >> OpenLDAP: https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL
>  > 
>  > I have prepared a draft of the change proposal.  Could you please have a
>  > look?
>  > 
>  > https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL
>  > 
>  > Thanks in advance!
>  
>  The "Scope" section should contain description of tasks/actions which
>  need to be done by the Change owner, Other developers, etc. May I ask
>  you please to update this section as expected ?
>  Also, in case there is any action expected from RelEng, please open a
>  ticket in https://pagure.io/releng/issues and put the ticket number
>  into the Change page. 
Clarified via
https://fedoraproject.org/w/index.php?title=Changes%2FlibcurlBackToOpenSS...

>  Otherwise it looks good. 
Thanks for review!

Kamil

> Thanks,
> Jan

[Stephen Gallagher]

On 04/06/2017 09:12 AM, Kamil Dudka wrote:
>  On Thursday, April 06, 2017 15:00:31 Jan Kurik wrote:
> > On Thu, Apr 6, 2017 at 2:49 PM, Kamil Dudka <kdudka(a)redhat.com&gt; wrote:
> >> On Wednesday, April 05, 2017 17:09:34 Jan Kurik wrote:
> >>> Might not be directly related, but just for a reference - one of the
> >>> F26 Changes (currently deferred to F27) is doing the same for
> >>> OpenLDAP: https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL
> >>
> >> I have prepared a draft of the change proposal.  Could you please have a
> >> look?
> >>
> >> https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL
> >>
> >> Thanks in advance!
> >
> > The "Scope" section should contain description of tasks/actions which
> > need to be done by the Change owner, Other developers, etc. May I ask
> > you please to update this section as expected ?
> > Also, in case there is any action expected from RelEng, please open a
> > ticket in https://pagure.io/releng/issues and put the ticket number
> > into the Change page.
>  
>  Clarified via
> https://fedoraproject.org/w/index.php?title=Changes%2FlibcurlBackToOpenSS...
>  
> > Otherwise it looks good.
>  
>  Thanks for review!
>   
There have been other responses to this thread that indicate that this change
may impact other critical packages in the distribution (fixable, but impacted).
I'd probably classify this as a System-Wide Change.

[Jan Kurik]

On Thu, Apr 6, 2017 at 3:47 PM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  On 04/06/2017 09:12 AM, Kamil Dudka wrote:
> > On Thursday, April 06, 2017 15:00:31 Jan Kurik wrote:
> >> On Thu, Apr 6, 2017 at 2:49 PM, Kamil Dudka <kdudka(a)redhat.com&gt; wrote:
> >>> On Wednesday, April 05, 2017 17:09:34 Jan Kurik wrote:
> >>>> Might not be directly related, but just for a reference - one of the
> >>>> F26 Changes (currently deferred to F27) is doing the same for
> >>>> OpenLDAP: https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL
> >>>
> >>> I have prepared a draft of the change proposal.  Could you please have a
> >>> look?
> >>>
> >>> https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL
> >>>
> >>> Thanks in advance!
> >>
> >> The "Scope" section should contain description of tasks/actions which
> >> need to be done by the Change owner, Other developers, etc. May I ask
> >> you please to update this section as expected ?
> >> Also, in case there is any action expected from RelEng, please open a
> >> ticket in https://pagure.io/releng/issues and put the ticket number
> >> into the Change page.
> >
> > Clarified via
> https://fedoraproject.org/w/index.php?title=Changes%2FlibcurlBackToOpenSS...
Thanks Kamil. Looks good IMO.

> >> Otherwise it looks good.
> >
> > Thanks for review!
> >
>
>  There have been other responses to this thread that indicate that this change
>  may impact other critical packages in the distribution (fixable, but impacted).
>  I'd probably classify this as a System-Wide Change. 
Kamil, please consider the impact on other packages as Stephen
suggesting. Once you feel it is ready to go through the process,
please switch the category to "ChangeReadyForWrangler" and I will take
the next steps in the process.

Regards,
Jan

>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  


-- 
Jan Kuřík
Platform & Fedora Program Manager
Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic

[Kamil Dudka]

On Thursday, April 06, 2017 16:05:16 Jan Kurik wrote:
>  On Thu, Apr 6, 2017 at 3:47 PM, Stephen Gallagher
> <sgallagh(a)redhat.com&gt; wrote:
>  > On 04/06/2017 09:12 AM, Kamil Dudka wrote:
>  >> On Thursday, April 06, 2017 15:00:31 Jan Kurik wrote:
>  >>> On Thu, Apr 6, 2017 at 2:49 PM, Kamil Dudka <kdudka(a)redhat.com&gt;
> wrote:
>  >>>> On Wednesday, April 05, 2017 17:09:34 Jan Kurik wrote:
>  >>>>> Might not be directly related, but just for a reference - one of
> the
>  >>>>> F26 Changes (currently deferred to F27) is doing the same for
>  >>>>> OpenLDAP:
> https://fedoraproject.org/wiki/Changes/OpenLDAPwithOpenSSL
>  >>>> 
>  >>>> I have prepared a draft of the change proposal.  Could you please have
>  >>>> a
>  >>>> look?
>  >>>> 
>  >>>> https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL
>  >>>> 
>  >>>> Thanks in advance!
>  >>> 
>  >>> The "Scope" section should contain description of tasks/actions
> which
>  >>> need to be done by the Change owner, Other developers, etc. May I ask
>  >>> you please to update this section as expected ?
>  >>> Also, in case there is any action expected from RelEng, please open a
>  >>> ticket in https://pagure.io/releng/issues and put the ticket number
>  >>> into the Change page.
>  >> 
>  >> Clarified via
>  >> https://fedoraproject.org/w/index.php?title=Changes%2FlibcurlBackToOpenS
>  >> SL&diff=490362&oldid=490361
>  Thanks Kamil. Looks good IMO.
>  
>  >>> Otherwise it looks good.
>  >> 
>  >> Thanks for review!
>  > 
>  > There have been other responses to this thread that indicate that this
>  > change may impact other critical packages in the distribution (fixable,
>  > but impacted). I'd probably classify this as a System-Wide Change. 
Sure.  This was meant to be a system-wide change.  I only overlooked the
classification at the end of the template.  It is fixed now.

>  Kamil, please consider the impact on other packages as Stephen
>  suggesting. Once you feel it is ready to go through the process,
>  please switch the category to "ChangeReadyForWrangler" and I will take
>  the next steps in the process. 
Done.  Thanks!

Kamil

> Regards,
> Jan

[Kamil Dudka]

On Wednesday, April 05, 2017 10:33:11 Stephen Gallagher wrote:
>  On 04/05/2017 09:59 AM, Colin Walters wrote:
>  > On Wed, Apr 5, 2017, at 09:52 AM, Kamil Dudka wrote:
>  >> In order to make even smaller Fedora base images, it was proposed to
>  >> switch
>  >> libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
>  >> motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
>  >> deprecated and libcurl is the only package that pulls NSS as its
>  >> dependency
>  >> into the Fedora base image.  Hence, by switching libcurl back to OpenSSL,
>  >> we could create Fedora base image that contains fewer crypto libraries
>  >> inside.> 
>  > Makes sense to me - from the Atomic Host perspective, we are switching
>  > ostree to use libcurl, since libdnf already does (and librepo hard depends
>  > on OpenSSL, even though libcurl used NSS).
>  > 
>  >> Additional proposal that would help to reduce the size of base image is
>  >> the
>  >> libcurl-minimal subpackage, which can be installed installed as a
>  >> lightweight replacement of the libcurl package, with smaller size and
>  >> fewer dependencies.> 
>  > I'm in agreement with this except:
>  > 
>  > # configure minimal build
>  > ...
>  > 
>  >         --without-nghttp2
>  > 
>  > I'd really prefer to keep HTTP2 available by default - it can be
>  > dramatically better.
>  
>  I'll second this: it looks like libnghttp2 does not pull in any dependencies
>  that wouldn't already be part of any minimal install (just glibc and ld)
>  and its filesystem space is only about 150k uncompressed.
>  
>  It's probably reasonable to keep this in our minimal set for the HTTP2
>  functionality. 
Thanks for the suggestion!  I have implemented it in my private branch:

http://pkgs.fedoraproject.org/cgit/rpms/curl.git/commit/?id=e8208d3e

... and scheduled a new Copr build with the change included.

Anyway, do not overestimate the power of HTTP/2.  It will not transparently
bring you better transfers for free.  You can speak HTTP/2 even while using
the curl tool but it is mainly useful for testing.  If you want to take the
advantage of the HTTP/2 features, you need to use the multi API of libcurl
and your software built on top of libcurl needs to be aware of the HTTP/2
protocol.  Moreover, the throughput of HTTP/2 can be much lower compared to
HTTP/1 if you are communicating over IP network with some packet loss.

I would suggest to watch the following talks by Daniel Stenberg to obtain
realistic expectations about the HTTP/2 support in libcurl and the HTTP/2
protocol in general:

https://thomas.glanzmann.de/curl-meet-2017/2017-03-19_03_Daniel_Stenberg_...
https://thomas.glanzmann.de/curl-meet-2017/2017-03-19_08_Daniel_Stenberg_...

Kamil

[Colin Walters]

On Wed, Apr 5, 2017, at 11:28 AM, Kamil Dudka wrote:

>  Anyway, do not overestimate the power of HTTP/2.  It will not
> transparently
>  bring you better transfers for free.  You can speak HTTP/2 even while using
>  the curl tool but it is mainly useful for testing.  If you want to take the
>  advantage of the HTTP/2 features, you need to use the multi API of libcurl
>  and your software built on top of libcurl needs to be aware of the HTTP/2
>  protocol.  
libostree does that -
https://github.com/ostreedev/ostree/blob/c937305c0e7f5609273e25753912c294...

In the exploded archive case, I get quite a lot of speedup.

librepo is not quite as good at this yet, but it matters less for RPMs.

[Kamil Dudka]

On Wednesday, April 05, 2017 11:38:35 Colin Walters wrote:
>  On Wed, Apr 5, 2017, at 11:28 AM, Kamil Dudka wrote:
>  > Anyway, do not overestimate the power of HTTP/2.  It will not
>  > transparently
>  > bring you better transfers for free.  You can speak HTTP/2 even while
>  > using
>  > the curl tool but it is mainly useful for testing.  If you want to take
>  > the
>  > advantage of the HTTP/2 features, you need to use the multi API of libcurl
>  > and your software built on top of libcurl needs to be aware of the HTTP/2
>  > protocol.
>  
>  libostree does that -
>  https://github.com/ostreedev/ostree/blob/c937305c0e7f5609273e25753912c294b0
>  40a6ac/src/libostree/ostree-fetcher-curl.c
>  
>  In the exploded archive case, I get quite a lot of speedup.
>  
>  librepo is not quite as good at this yet, but it matters less for RPMs. 
I cannot see CURLOPT_PIPEWAIT or CURLMOPT_PIPELINING options in your code.  
They are needed if you want to take advantage of the core HTTP/2 features.  
Otherwise you can slightly benefit from the compressed HTTP headers but you 
are not using the HTTP/2 pipelining/multiplexing.  I would suggest to have a 
look at the http2-download.c example:

https://github.com/curl/curl/blob/curl-7_53_0/docs/examples/http2-download.c

Kamil

[Dusty Mabe]

On 04/05/2017 12:17 PM, Kamil Dudka wrote:
>  On Wednesday, April 05, 2017 11:38:35 Colin Walters wrote:
> >
> > libostree does that -
> > https://github.com/ostreedev/ostree/blob/c937305c0e7f5609273e25753912c294b0
> > 40a6ac/src/libostree/ostree-fetcher-curl.c
> >
> > In the exploded archive case, I get quite a lot of speedup.
> >
> > librepo is not quite as good at this yet, but it matters less for RPMs.
>  
>  I cannot see CURLOPT_PIPEWAIT or CURLMOPT_PIPELINING options in your code.  
>  They are needed if you want to take advantage of the core HTTP/2 features.  
>  Otherwise you can slightly benefit from the compressed HTTP headers but you 
>  are not using the HTTP/2 pipelining/multiplexing.  I would suggest to have a 
>  look at the http2-download.c example:
>  
>  https://github.com/curl/curl/blob/curl-7_53_0/docs/examples/http2-download.c 
FYI: https://github.com/ostreedev/ostree/pull/780

Thanks for your suggestion!

Dusty

[Kamil Dudka]

On Wednesday, April 05, 2017 18:28:53 Dusty Mabe wrote:
>  On 04/05/2017 12:17 PM, Kamil Dudka wrote:
>  > On Wednesday, April 05, 2017 11:38:35 Colin Walters wrote:
>  >> libostree does that -
>  >> https://github.com/ostreedev/ostree/blob/c937305c0e7f5609273e25753912c294
>  >> b0
>  >> 40a6ac/src/libostree/ostree-fetcher-curl.c
>  >> 
>  >> In the exploded archive case, I get quite a lot of speedup.
>  >> 
>  >> librepo is not quite as good at this yet, but it matters less for RPMs.
>  > 
>  > I cannot see CURLOPT_PIPEWAIT or CURLMOPT_PIPELINING options in your code.
>  > They are needed if you want to take advantage of the core HTTP/2 features.
>  > Otherwise you can slightly benefit from the compressed HTTP headers but
>  > you
>  > are not using the HTTP/2 pipelining/multiplexing.  I would suggest to have
>  > a look at the http2-download.c example:
>  > 
>  > https://github.com/curl/curl/blob/curl-7_53_0/docs/examples/http2-download
>  > .c
>  FYI: https://github.com/ostreedev/ostree/pull/780 
Looks good!

Kamil

> Thanks for your suggestion!
> 
> Dusty

[Rob Crittenden]

Kamil Dudka wrote:
>  In order to make even smaller Fedora base images, it was proposed to
> switch
>  libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
>  motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
>  deprecated and libcurl is the only package that pulls NSS as its dependency
>  into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
>  could create Fedora base image that contains fewer crypto libraries inside.
>  
>  Additional proposal that would help to reduce the size of base image is the
>  libcurl-minimal subpackage, which can be installed installed as a lightweight
>  replacement of the libcurl package, with smaller size and fewer dependencies.
>  The libcurl-minimal subpackage was already proposed on this mailing list one
>  year ago but at that time not many people knew that it would be useful today:
>  
>     
> https://lists.fedoraproject.org/archives/list/[e-mail].o...
>  
>  I am CCing the actual requestors so they can fight for these proposals.
>  
>  A proof of concept is available in the private-kdudka-libcurl-minimal branch:
>  
>     
> http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-li...
>  
>  I have also prepared Copr repositories for f24+:
>  
>      https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/
>  
>  Any feedback is welcome! 
This has the potential to wreak havoc with freeIPA and certmonger. How
much so won't be known until someone tries the updated builds.

rob

[Alexander Bokovoy]

On ke, 05 huhti 2017, Kamil Dudka wrote:
> In order to make even smaller Fedora base images, it was proposed to
> switch
> libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
> motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
> deprecated and libcurl is the only package that pulls NSS as its dependency
> into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
> could create Fedora base image that contains fewer crypto libraries inside.
>
> Additional proposal that would help to reduce the size of base image is the
> libcurl-minimal subpackage, which can be installed installed as a lightweight
> replacement of the libcurl package, with smaller size and fewer dependencies.
> The libcurl-minimal subpackage was already proposed on this mailing list one
> year ago but at that time not many people knew that it would be useful today:
>
>    
> https://lists.fedoraproject.org/archives/list/[e-mail].o...
>
> I am CCing the actual requestors so they can fight for these proposals.
>
> A proof of concept is available in the private-kdudka-libcurl-minimal branch:
>
>    
> http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-li...
>
> I have also prepared Copr repositories for f24+:
>
>     https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/
>
> Any feedback is welcome! 
It is unclear at which timeframe you want to implement
this. I'm
concerned with FreeIPA because we use libcurl internally and we are not
yet migrated to OpenSSL fully (definitely not yet in F26). We also need
to look at what happens with certmonger.

-- 
/ Alexander Bokovoy

[Stephen Gallagher]

On 04/05/2017 11:42 AM, Alexander Bokovoy wrote:
>  On ke, 05 huhti 2017, Kamil Dudka wrote:
> > In order to make even smaller Fedora base images, it was proposed to switch
> > libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
> > motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
> > deprecated and libcurl is the only package that pulls NSS as its dependency
> > into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
> > could create Fedora base image that contains fewer crypto libraries inside.
> >
> > Additional proposal that would help to reduce the size of base image is the
> > libcurl-minimal subpackage, which can be installed installed as a lightweight
> > replacement of the libcurl package, with smaller size and fewer dependencies.
> > The libcurl-minimal subpackage was already proposed on this mailing list one
> > year ago but at that time not many people knew that it would be useful today:
> >
> >   
> >
> https://lists.fedoraproject.org/archives/list/[e-mail].o...
> >
> >
> > I am CCing the actual requestors so they can fight for these proposals.
> >
> > A proof of concept is available in the private-kdudka-libcurl-minimal branch:
> >
> >   
> >
> http://pkgs.fedoraproject.org/cgit/rpms/curl.git/log/?h=private-kdudka-li...
> >
> >
> > I have also prepared Copr repositories for f24+:
> >
> >    https://copr.fedorainfracloud.org/coprs/kdudka/curl-minimal/
> >
> > Any feedback is welcome!
>  It is unclear at which timeframe you want to implement this. I'm
>  concerned with FreeIPA because we use libcurl internally and we are not
>  yet migrated to OpenSSL fully (definitely not yet in F26). We also need
>  to look at what happens with certmonger.
>   
The initial request here was to make this migration for the Fedora 27 timeframe.
Definitely *not* in Fedora 26.

[Dominik 'Rathann' Mierzejewski]

Hello, Kamil.

On Wednesday, 05 April 2017 at 15:52, Kamil Dudka wrote:
>  In order to make even smaller Fedora base images, it was proposed to
> switch
>  libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
>  motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
>  deprecated and libcurl is the only package that pulls NSS as its dependency
>  into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
>  could create Fedora base image that contains fewer crypto libraries inside. 
Please open a system wide change request for F27 for this, as this affects
a couple hundred of packages, including some on the critical path.

https://fedoraproject.org/wiki/Changes/Policy#Complex_system_wide_changes

This is a good move to reduce the dependencies in my opinion, but as
seen elsewhere in this thread, it needs to be thoroughly tested.

Regards,
Dominik
-- 
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
        -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

[Miroslav Lichvar]

On Wed, Apr 05, 2017 at 03:52:22PM +0200, Kamil Dudka wrote:
>  In order to make even smaller Fedora base images, it was proposed to
> switch
>  libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
>  motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
>  deprecated and libcurl is the only package that pulls NSS as its dependency
>  into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
>  could create Fedora base image that contains fewer crypto libraries inside. 
I'm just wondering, does this change anything from the security point
of view? Has history shown one library to be better than the other,
for instance in the number of important issues found in the TLS
implementation?

Also, wasn't there an issue with the OpenSSL's licensing and GPL?
If it still is, could it affect any of the packages that are now using
libcurl?

-- 
Miroslav Lichvar

[Matthew Miller]

On Thu, Apr 06, 2017 at 05:50:16PM +0200, Miroslav Lichvar wrote:
>  > In order to make even smaller Fedora base images, it was
> proposed to switch
>  > libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
>  > motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
>  > deprecated and libcurl is the only package that pulls NSS as its dependency
>  > into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
>  > could create Fedora base image that contains fewer crypto libraries inside.
>  I'm just wondering, does this change anything from the security point
>  of view? Has history shown one library to be better than the other,
>  for instance in the number of important issues found in the TLS
>  implementation? 
I don't think that's necessarily a great predictor of future results.
However, going from two different things to just one will _definitely_
result in fewer future CVES which impact the base.


>  Also, wasn't there an issue with the OpenSSL's licensing and
> GPL?
>  If it still is, could it affect any of the packages that are now using
>  libcurl? 
There is this: https://www.openssl.org/blog/blog/2017/03/22/license/

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Stephen Gallagher]

On 04/06/2017 12:15 PM, Matthew Miller wrote:
>  On Thu, Apr 06, 2017 at 05:50:16PM +0200, Miroslav Lichvar wrote:
> >> In order to make even smaller Fedora base images, it was proposed to switch
> >> libcurl back to OpenSSL.  The Fedora Crypto Consolidation project, which
> >> motivated the switch of libcurl from OpenSSL to NSS ten years ago, is now
> >> deprecated and libcurl is the only package that pulls NSS as its dependency
> >> into the Fedora base image.  Hence, by switching libcurl back to OpenSSL, we
> >> could create Fedora base image that contains fewer crypto libraries inside.
> > I'm just wondering, does this change anything from the security point
> > of view? Has history shown one library to be better than the other,
> > for instance in the number of important issues found in the TLS
> > implementation?
>  
>  I don't think that's necessarily a great predictor of future results.
>  However, going from two different things to just one will _definitely_
>  result in fewer future CVES which impact the base.
>  
>  
> > Also, wasn't there an issue with the OpenSSL's licensing and GPL?
> > If it still is, could it affect any of the packages that are now using
> > libcurl?
>  
>  There is this: https://www.openssl.org/blog/blog/2017/03/22/license/
>   
Also this, which is more immediately relevant:
https://fedoraproject.org/wiki/Licensing:FAQ#What.27s_the_deal_with_the_O...

tl;dr: "However, we consider that the OpenSSL library is a system library, as
defined by the GPL, on Fedora and therefore we are allowed to ship GPL software
that links to the OpenSSL library."

[David Woodhouse]

On Thu, 2017-04-06 at 12:57 -0400, Stephen Gallagher wrote:
>  > > Also, wasn't there an issue with the OpenSSL's
> licensing and GPL?
>  > > If it still is, could it affect any of the packages that are now using
>  > > libcurl?
>  > There is this: https://www.openssl.org/blog/blog/2017/03/22/license/ 
Which doesn't really help as they're moving to a different but still
GPL-incompatible licence. Grr :)

>  Also this, which is more immediately relevant:
>
> https://fedoraproject.org/wiki/Licensing:FAQ#What.27s_the_deal_with_the_O...
>  
>  tl;dr: "However, we consider that the OpenSSL library is a system library, as
>  defined by the GPL, on Fedora and therefore we are allowed to ship GPL software
>  that links to the OpenSSL library." 
That would seem to resolve it, sure :)

[Tomas Mraz]

On Thu, 2017-04-13 at 10:42 +0100, David Woodhouse wrote:
>  On Thu, 2017-04-06 at 12:57 -0400, Stephen Gallagher wrote:
>  > > > Also, wasn't there an issue with the OpenSSL's licensing and
>  > > > GPL?
>  > > > If it still is, could it affect any of the packages that are
>  > > > now using
>  > > > libcurl?
>  > > 
>  > > There is this: https://www.openssl.org/blog/blog/2017/03/22/licen
>  > > se/
>  
>  Which doesn't really help as they're moving to a different but still
>  GPL-incompatible licence. Grr :) 
GPLv2-only incompatible licence. It is compatible with GPLv3 or GPLv2+.
 So the situation is better and given the objectives for the licence
change they had I am afraid there was no better choice.
-- 
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
                                              Turkish proverb
(You'll never know whether the road is wrong though.)

[Kai Engert]

I would like to make you aware that the certificate validation of openssl isn't
as complete as in NSS.

For example, NSS is able to handle the blacklisted/distrusted CAs, which have
been published by Mozilla, and are being made available as part of the ca-
certificates package, while I believe openssl isn't.

In addition, a few CA distrust mechanisms have been implemented at the NSS code
level, and no equivalent mechanisms are currently being implemented at the
openssl level [1].

As a consequence of the switch to openssl, software that currently uses libcurl
would lose these additional trust checks when doing certificate validation for
SSL/TLS connections.

Kai

[1] https://wiki.mozilla.org/CA:Root_Store_Trust_Mods

[Adam Williamson]

On Thu, 2017-04-06 at 18:22 +0200, Kai Engert wrote:
>  I would like to make you aware that the certificate validation of
> openssl isn't
>  as complete as in NSS.
>  
>  For example, NSS is able to handle the blacklisted/distrusted CAs, which have
>  been published by Mozilla, and are being made available as part of the ca-
>  certificates package, while I believe openssl isn't.
>  
>  In addition, a few CA distrust mechanisms have been implemented at the NSS code
>  level, and no equivalent mechanisms are currently being implemented at the
>  openssl level [1]. 
I don't believe this is accurate. There is an extended certificate
format which OpenSSL will accept which allows you to indicate specific
trust or *dis*trust of a given certificate for specific purposes. You
could, I think, use this format to produce a certificate file which
basically says "I distrust this CA certificate for all purposes".

I wrote a bit about this at
https://www.happyassassin.net/2015/01/16/openssl-trust-and-purpose/ .

Corrections welcome, of course...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Kai Engert]

On Thu, 2017-04-06 at 09:29 -0700, Adam Williamson wrote:
>  On Thu, 2017-04-06 at 18:22 +0200, Kai Engert wrote:
>  > I would like to make you aware that the certificate validation of openssl
>  > isn't
>  > as complete as in NSS.
>  > 
>  > For example, NSS is able to handle the blacklisted/distrusted CAs, which
>  > have
>  > been published by Mozilla, and are being made available as part of the ca-
>  > certificates package, while I believe openssl isn't.
>  > 
>  > In addition, a few CA distrust mechanisms have been implemented at the NSS
>  > code
>  > level, and no equivalent mechanisms are currently being implemented at the
>  > openssl level [1].
>  
>  I don't believe this is accurate. There is an extended certificate
>  format which OpenSSL will accept which allows you to indicate specific
>  trust or *dis*trust of a given certificate for specific purposes. You
>  could, I think, use this format to produce a certificate file which
>  basically says "I distrust this CA certificate for all purposes".
>  
>  I wrote a bit about this at
>  https://www.happyassassin.net/2015/01/16/openssl-trust-and-purpose/ .
>  
>  Corrections welcome, of course... 
The ca-certificates package indeed produces two versions of the PEM format
files, one as a simple list of CAs, and another version that uses the BEGIN
TRUSTED CERTIFICATE file format, which includes the distrust flags.

A couple of year ago, I had filed a bug to request that the openssl library
default is switched to make use of this advanced format:
  https://bugzilla.redhat.com/show_bug.cgi?id=873373

However, that bug is still in NEW state, so I guess it depends on the individual
applications, if they use the list that includes distrust information.

Which one is libcurl using?

Even if you switch that to the distrust list, you still don't get the partial
distrust, which may be implemented at the NSS code level (such as date based
distrust for StartCom/WoSign roots, and the domain constraints for some CA).

Kai

[Kamil Dudka]

On Thursday, April 06, 2017 18:39:26 Kai Engert wrote:
>  On Thu, 2017-04-06 at 09:29 -0700, Adam Williamson wrote:
>  > On Thu, 2017-04-06 at 18:22 +0200, Kai Engert wrote:
>  > > I would like to make you aware that the certificate validation of
>  > > openssl
>  > > isn't
>  > > as complete as in NSS.
>  > > 
>  > > For example, NSS is able to handle the blacklisted/distrusted CAs, which
>  > > have
>  > > been published by Mozilla, and are being made available as part of the
>  > > ca-
>  > > certificates package, while I believe openssl isn't.
>  > > 
>  > > In addition, a few CA distrust mechanisms have been implemented at the
>  > > NSS
>  > > code
>  > > level, and no equivalent mechanisms are currently being implemented at
>  > > the
>  > > openssl level [1].
>  > 
>  > I don't believe this is accurate. There is an extended certificate
>  > format which OpenSSL will accept which allows you to indicate specific
>  > trust or *dis*trust of a given certificate for specific purposes. You
>  > could, I think, use this format to produce a certificate file which
>  > basically says "I distrust this CA certificate for all purposes".
>  > 
>  > I wrote a bit about this at
>  > https://www.happyassassin.net/2015/01/16/openssl-trust-and-purpose/ .
>  > 
>  > Corrections welcome, of course...
>  
>  The ca-certificates package indeed produces two versions of the PEM format
>  files, one as a simple list of CAs, and another version that uses the BEGIN
>  TRUSTED CERTIFICATE file format, which includes the distrust flags.
>  
>  A couple of year ago, I had filed a bug to request that the openssl library
>  default is switched to make use of this advanced format:
>    https://bugzilla.redhat.com/show_bug.cgi?id=873373
>  
>  However, that bug is still in NEW state, so I guess it depends on the
>  individual applications, if they use the list that includes distrust
>  information.
>  
>  Which one is libcurl using? 
Although we build libcurl against NSS now, it loads the same CA bundle as
if we built it against OpenSSL:

    /etc/pki/tls/certs/ca-bundle.crt

So I doubt it could actually take advantage of those extra flags.  If you
have a reproducer at hand, you can give it a try.

>  Even if you switch that to the distrust list, you still don't get
> the
>  partial distrust, which may be implemented at the NSS code level (such as
>  date based distrust for StartCom/WoSign roots, and the domain constraints
>  for some CA). 
You say "may be implemented at the NSS code level".  Do I understand it 
correctly that NSS currently does not implement the date based distrust
and the domain constraints?

Kamil

> Kai

[Kai Engert]

On Fri, 2017-04-07 at 10:38 +0200, Kamil Dudka wrote:
>  
>  Although we build libcurl against NSS now, it loads the same CA bundle as
>  if we built it against OpenSSL:
>  
>      /etc/pki/tls/certs/ca-bundle.crt
>  
>  So I doubt it could actually take advantage of those extra flags. 
This file doesn't contain the distrust flags.

The correct file would be /etc/pki/tls/certs/ca-bundle.trust.crt


>  If you
>  have a reproducer at hand, you can give it a try. 
I currently don't know of a public test site that uses a blacklisted
certificate.

As long as libcurl/openssl doesn't load the right file, we don't need to test.

When you're able to switch openssl to use the correct one, I can help to create
a test for that.


>  > Even if you switch that to the distrust list, you still
> don't get the
>  > partial distrust, which may be implemented at the NSS code level (such as
>  > date based distrust for StartCom/WoSign roots, and the domain constraints
>  > for some CA).
>  
>  You say "may be implemented at the NSS code level". 
The intention was to say, that additional distrust rules might get implemented
at the NSS code level in the future.


>  Do I understand it 
>  correctly that NSS currently does not implement the date based distrust
>  and the domain constraints? 
NSS does implement them, see the places where the wiki page mentions NSS:
  https://wiki.mozilla.org/CA:Root_Store_Trust_Mods

Kai

[Kamil Dudka]

On Friday, April 07, 2017 11:01:35 Kai Engert wrote:
>  On Fri, 2017-04-07 at 10:38 +0200, Kamil Dudka wrote:
>  > Although we build libcurl against NSS now, it loads the same CA bundle as
>  > if we built it against OpenSSL:
>  > 
>  >     /etc/pki/tls/certs/ca-bundle.crt
>  > 
>  > So I doubt it could actually take advantage of those extra flags.
>  
>  This file doesn't contain the distrust flags.
>  
>  The correct file would be /etc/pki/tls/certs/ca-bundle.trust.crt 
Yes, but it does not make sense to load such a file by nss-pem because it does 
not support those flags anyway.  The correct fix for NSS-linked libcurl would 
probably be to just disable loading the CA roots from file by default.

>  > If you
>  > have a reproducer at hand, you can give it a try.
>  
>  I currently don't know of a public test site that uses a blacklisted
>  certificate.
>  
>  As long as libcurl/openssl doesn't load the right file, we don't need to
>  test.
>  
>  When you're able to switch openssl to use the correct one, I can help to
>  create a test for that.
>  
>  > > Even if you switch that to the distrust list, you still don't get the
>  > > partial distrust, which may be implemented at the NSS code level (such
>  > > as
>  > > date based distrust for StartCom/WoSign roots, and the domain
>  > > constraints
>  > > for some CA).
>  > 
>  > You say "may be implemented at the NSS code level".
>  
>  The intention was to say, that additional distrust rules might get
>  implemented at the NSS code level in the future.
>  
>  > Do I understand it 
>  > correctly that NSS currently does not implement the date based distrust
>  > and the domain constraints?
>  
>  NSS does implement them, see the places where the wiki page mentions NSS:
>    https://wiki.mozilla.org/CA:Root_Store_Trust_Mods 
Thanks for the link!  I did not know it was implemented while talking about 
this topic with curl upstream recently (and Daniel Stenberg, who works for 
Mozilla, did not know it either).  It is good that it is implemented in NSS 
now.

Kamil

> Kai

[Kai Engert]

On Fri, 2017-04-07 at 11:54 +0200, Kamil Dudka wrote:
>  On Friday, April 07, 2017 11:01:35 Kai Engert wrote:
>  > On Fri, 2017-04-07 at 10:38 +0200, Kamil Dudka wrote:
>  > > Although we build libcurl against NSS now, it loads the same CA bundle as
>  > > if we built it against OpenSSL:
>  > > 
>  > >     /etc/pki/tls/certs/ca-bundle.crt
>  > > 
>  > > So I doubt it could actually take advantage of those extra flags.
>  > 
>  > This file doesn't contain the distrust flags.
>  > 
>  > The correct file would be /etc/pki/tls/certs/ca-bundle.trust.crt
>  
>  Yes, but it does not make sense to load such a file by nss-pem because it
>  does 
>  not support those flags anyway.  The correct fix for NSS-linked libcurl would 
>  probably be to just disable loading the CA roots from file by default. 
Why do you mentioned a need to fix curl-nss?

The regular approach for NSS applications is to load the NSS libnssckbi.so (now
the drop-in replacement p11-kit-trust.so), which provides all trust and distrust
information in a format that NSS can handle.

How does curl-nss load the CA trust list?

If curl-nss doesn't load libnssckbi.so/p11-kit-trust.so but rather loads a
simple PEM file, then today's curl-nss doesn't use distrust information.

Kai

[Kamil Dudka]

On Friday, April 07, 2017 13:34:42 Kai Engert wrote:
>  On Fri, 2017-04-07 at 11:54 +0200, Kamil Dudka wrote:
>  > On Friday, April 07, 2017 11:01:35 Kai Engert wrote:
>  > > On Fri, 2017-04-07 at 10:38 +0200, Kamil Dudka wrote:
>  > > > Although we build libcurl against NSS now, it loads the same CA bundle
>  > > > as
>  > > > if we built it against OpenSSL:
>  > > > 
>  > > >     /etc/pki/tls/certs/ca-bundle.crt
>  > > > 
>  > > > So I doubt it could actually take advantage of those extra flags.
>  > > 
>  > > This file doesn't contain the distrust flags.
>  > > 
>  > > The correct file would be /etc/pki/tls/certs/ca-bundle.trust.crt
>  > 
>  > Yes, but it does not make sense to load such a file by nss-pem because it
>  > does 
>  > not support those flags anyway.  The correct fix for NSS-linked libcurl
>  > would  probably be to just disable loading the CA roots from file by
>  > default.
>  Why do you mentioned a need to fix curl-nss? 
Because the NSS-linked libcurl in Fedora currently works in a way that it does 
not take advantage of the extended validation features implemented in NSS, as 
I understand it.

>  The regular approach for NSS applications is to load the NSS
> libnssckbi.so
>  (now the drop-in replacement p11-kit-trust.so), which provides all trust
>  and distrust information in a format that NSS can handle.
>  
>  How does curl-nss load the CA trust list? 
libcurl loads /etc/pki/tls/certs/ca-bundle.trust.crt using the nss-pem module.

>  If curl-nss doesn't load libnssckbi.so/p11-kit-trust.so but
> rather loads a
>  simple PEM file, then today's curl-nss doesn't use distrust information. 
Exactly.  It sounds like there is still some room for improvement.

Kamil

> Kai

[Kamil Dudka]

On Friday, April 07, 2017 13:45:48 Kamil Dudka wrote:
>  On Friday, April 07, 2017 13:34:42 Kai Engert wrote:
>  > On Fri, 2017-04-07 at 11:54 +0200, Kamil Dudka wrote:
>  > > On Friday, April 07, 2017 11:01:35 Kai Engert wrote:
>  > > > On Fri, 2017-04-07 at 10:38 +0200, Kamil Dudka wrote:
>  > > > > Although we build libcurl against NSS now, it loads the same CA
>  > > > > bundle
>  > > > > as
>  > > > > 
>  > > > > if we built it against OpenSSL:
>  > > > >     /etc/pki/tls/certs/ca-bundle.crt
>  > > > > 
>  > > > > So I doubt it could actually take advantage of those extra flags.
>  > > > 
>  > > > This file doesn't contain the distrust flags.
>  > > > 
>  > > > The correct file would be /etc/pki/tls/certs/ca-bundle.trust.crt
>  > > 
>  > > Yes, but it does not make sense to load such a file by nss-pem because
>  > > it
>  > > does
>  > > not support those flags anyway.  The correct fix for NSS-linked libcurl
>  > > would  probably be to just disable loading the CA roots from file by
>  > > default.
>  > 
>  > Why do you mentioned a need to fix curl-nss?
>  
>  Because the NSS-linked libcurl in Fedora currently works in a way that it
>  does not take advantage of the extended validation features implemented in
>  NSS, as I understand it.
>  
>  > The regular approach for NSS applications is to load the NSS libnssckbi.so
>  > (now the drop-in replacement p11-kit-trust.so), which provides all trust
>  > and distrust information in a format that NSS can handle.
>  > 
>  > How does curl-nss load the CA trust list?
>  
>  libcurl loads /etc/pki/tls/certs/ca-bundle.trust.crt using the nss-pem
>  module. 
Sorry.  Copy-paste error.  I meant /etc/pki/tls/certs/ca-bundle.crt of course.

Kamil

> > If curl-nss doesn't load libnssckbi.so/p11-kit-trust.so but rather loads a
> > simple PEM file, then today's curl-nss doesn't use distrust
information.
> 
> Exactly.  It sounds like there is still some room for improvement.
> 
> Kamil
> 
> > Kai

[Kai Engert]

You convinced me, that it would be good to have test cases to demonstrate how
nss/openssl/gnutls are behaving related to the distrust rules.

I setup the following page, wich provides multiple test cases, and intructions
how to test:
https://kuix.de/misc/test-distrust/

Kai

[Kamil Dudka]

On Friday, April 07, 2017 18:46:33 Kai Engert wrote:
>  You convinced me, that it would be good to have test cases to
> demonstrate
>  how nss/openssl/gnutls are behaving related to the distrust rules.
>  
>  I setup the following page, wich provides multiple test cases, and
>  intructions how to test:
>  https://kuix.de/misc/test-distrust/ 
Thanks!  I can confirm it works as expected if I load p11-kit-trust.so instead 
of using nss-pem to load the CA bundle from file.  However, it might be not so 
easy to switch curl to use it because the trust is global.  If we make libcurl 
load/unload the whole module per connection, it will hardly work as expected 
in case we run multiple handshakes in parallel.

Anyway, I guess we should move this discussion to some curl- or nss-related 
channel...

Kamil

> Kai

[Kai Engert]

On Mon, 2017-04-10 at 15:31 +0200, Kamil Dudka wrote:
>  Anyway, I guess we should move this discussion to some curl- or
> nss-related 
>  channel... 
The question remains, if it makes sense to switch back to openssl, if the
consequence is a loss in completeness of certificate trust checking.

In my opinion, a little bit of space saving shouldn't be a sufficient argument
for removing existing security functionality.

In the future, we should work on improving the certificate validation in a way
that can benefit all of our crypto libraries. This will certainly require
additional code, too.

There were some thoughts to potentially reuse the functionality that Firefox has
implemented at the application level, because currently there don't seem other
implementations in sight. That code is based on top of NSS code. If that gets
done, and if you want SSL/TLS connectivity inside the base image to be as secure
as in the rest of Fedora, you might have to eventually add NSS back to it.

Kai

[Florian Weimer]

On 04/10/2017 03:52 PM, Kai Engert wrote:
>  On Mon, 2017-04-10 at 15:31 +0200, Kamil Dudka wrote:
> > Anyway, I guess we should move this discussion to some curl- or nss-related
> > channel...
>
>  The question remains, if it makes sense to switch back to openssl, if the
>  consequence is a loss in completeness of certificate trust checking. 
I think OpenSSL is committed to fix their library initialization issues. 
  This would mean that the application is free to use OpenSSL in any way 
it sees fit, while using libcurl at the same time.

My understanding is that this is not the case with NSS.  libcurl's use 
of NSS and what the application does with NSS would be tightly coupled 
if libcurl uses NSS.  Sometimes, this is desirable, but often, it is not.

Thanks,
Florian

[David Woodhouse]

On Mon, 2017-04-10 at 15:52 +0200, Kai Engert wrote:
>  On Mon, 2017-04-10 at 15:31 +0200, Kamil Dudka wrote:
>  > Anyway, I guess we should move this discussion to some curl- or nss-related 
>  > channel...
>  
>  The question remains, if it makes sense to switch back to openssl, if the
>  consequence is a loss in completeness of certificate trust checking.
>  
>  In my opinion, a little bit of space saving shouldn't be a sufficient argument
>  for removing existing security functionality. 

FWIW I don't care much about "a little bit of space saving".

I've been advocating that we build curl against something other than
NSS for a long time, given that it violates our packaging guidelines
because NSS doesn't properly integrate with the p11-kit configured
tokens and doesn't support RFC7512 — and nss-pem fails to support lots
of key files.

I was thinking of GnuTLS though, which AUIU *would* have supported the
non-trivial trust metadata because it uses p11-kit-trust.so/libnssckbi
just like NSS does.

I'm not sure what reasoning there was for switching to OpenSSL instead
of GnuTLS...?

[Kamil Dudka]

On Thursday, April 13, 2017 10:45:13 David Woodhouse wrote:
>  On Mon, 2017-04-10 at 15:52 +0200, Kai Engert wrote:
>  > On Mon, 2017-04-10 at 15:31 +0200, Kamil Dudka wrote:
>  > > Anyway, I guess we should move this discussion to some curl- or
>  > > nss-related  channel...
>  > 
>  > The question remains, if it makes sense to switch back to openssl, if the
>  > consequence is a loss in completeness of certificate trust checking.
>  > 
>  > In my opinion, a little bit of space saving shouldn't be a sufficient
>  > argument for removing existing security functionality.
>  
>  FWIW I don't care much about "a little bit of space saving".
>  
>  I've been advocating that we build curl against something other than
>  NSS for a long time, given that it violates our packaging guidelines
>  because NSS doesn't properly integrate with the p11-kit configured
>  tokens and doesn't support RFC7512 — and nss-pem fails to support lots
>  of key files.
>  
>  I was thinking of GnuTLS though, which AUIU *would* have supported the
>  non-trivial trust metadata because it uses p11-kit-trust.so/libnssckbi
>  just like NSS does.
>  
>  I'm not sure what reasoning there was for switching to OpenSSL instead
>  of GnuTLS...? 
It was not my decision to be honest.  Nevertheless, one objective reason
could be that libcurl already loads OpenSSL libraries transitively as a 
dependency of libssh2.  So after switching libcurl to OpenSSL, only one
crypto library will be sufficient for curl at run time.

Kamil

[Chris Adams]

Once upon a time, David Woodhouse <dwmw2(a)infradead.org&gt; said:
>  I'm not sure what reasoning there was for switching to OpenSSL
> instead
>  of GnuTLS...? 
I think the general idea is to move things to what upstream considers
the "preferred" library.  If you had all the relevant -devel packages
installed and ran configure with no options, which one would it pick?
That is most likely the best-supported choice by the upstream
developers.

-- 
Chris Adams <linux(a)cmadams.net&gt;

[Matthew Miller]

On Mon, Apr 10, 2017 at 03:52:32PM +0200, Kai Engert wrote:
>  In my opinion, a little bit of space saving shouldn't be a
> sufficient
>  argument for removing existing security functionality. 
Space saving is nice, but that's not the real issue. It's a given that
all security libraries will have critical and urgent vulnerabilities
sometime in the future. By reducing the number of libraries in use in
the base system, we reduce the surface area. This is particularly
important where we have cascading artifacts built on a base — if the
bottom changes, there's a lot of churn.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Kamil Dudka]

On Friday, April 07, 2017 18:46:33 Kai Engert wrote:
>  You convinced me, that it would be good to have test cases to
> demonstrate
>  how nss/openssl/gnutls are behaving related to the distrust rules.
>  
>  I setup the following page, wich provides multiple test cases, and
>  intructions how to test:
>  https://kuix.de/misc/test-distrust/ 
I have opened a pull-request upstream for this:

https://github.com/curl/curl/pull/1414

Could you please have a look at preferably review directly on Github?

Thanks in advance!

Kamil

> Kai
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org