possibly of interest... making sure our certbot rpms are a first-class experience

[Matthew Miller]

See https://pagure.io/fesco/issue/2570 -- in short, Let's Encrypt is
recommending installing certbot via snap. While it's good that snap works on
Fedora systems, it's not the best experience.

The CertBot website https://certbot.eff.org/docs/install.html#operating-system-packages
has this kind of nasty warning:

   "While the Certbot team tries to keep the Certbot packages offered by
    various operating systems working in the most basic sense, due to
    distribution policies and/or the limited resources of distribution
    maintainers, Certbot OS packages often have problems that other
    distribution mechanisms do not. The packages are often old resulting in
    a lack of bug fixes and features and a worse TLS configuration than is
    generated by newer versions of Certbot. They also may not configure
    certificate renewal for you or have all of Certbot’s plugins available.
    For reasons like these, we recommend most users follow the instructions
    at https://certbot.eff.org/instructions and OS packages are only
    documented here as an alternative."

I'd like to see three things here:

1. Ensure that the scary things that they suggest are never true on Fedora
   systems. I think this *should* be the case because of Fedora is generally
   on the leading edge in crypto policies and keeps packages up to date
   rather than stagnating.

2. Convince EFF/Certbot to not FUD our packages and instead recommend them.
   Maybe harder, because there's not the appeal of "one size fits all"

3. Promote slick, out-of-the-box certbot as a Fedora Server feature. And do
   whatever it takes to make it really easy. Because that's actually really
   valuable.


Is this of interest to anyone?


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Stephen John Smoogen]

On Mon, 1 Feb 2021 at 11:46, Matthew Miller <mattdm(a)fedoraproject.org&gt;
wrote:

>  See https://pagure.io/fesco/issue/2570 -- in short, Let's Encrypt is
>  recommending installing certbot via snap. While it's good that snap works
>  on
>  Fedora systems, it's not the best experience.
>
>  The CertBot website
>  https://certbot.eff.org/docs/install.html#operating-system-packages
>  has this kind of nasty warning:
>
>     "While the Certbot team tries to keep the Certbot packages offered by
>      various operating systems working in the most basic sense, due to
>      distribution policies and/or the limited resources of distribution
>      maintainers, Certbot OS packages often have problems that other
>      distribution mechanisms do not. The packages are often old resulting in
>      a lack of bug fixes and features and a worse TLS configuration than is
>      generated by newer versions of Certbot. They also may not configure
>      certificate renewal for you or have all of Certbot’s plugins available.
>      For reasons like these, we recommend most users follow the instructions
>      at https://certbot.eff.org/instructions and OS packages are only
>      documented here as an alternative."
>
>  I'd like to see three things here:
>
>  1. Ensure that the scary things that they suggest are never true on Fedora
>     systems. I think this *should* be the case because of Fedora is
>  generally
>     on the leading edge in crypto policies and keeps packages up to date
>     rather than stagnating.
>
>  2. Convince EFF/Certbot to not FUD our packages and instead recommend them.
>     Maybe harder, because there's not the appeal of "one size fits all"
>
>  3. Promote slick, out-of-the-box certbot as a Fedora Server feature. And do
>     whatever it takes to make it really easy. Because that's actually really
>     valuable.
>
>
>  Is this of interest to anyone?
>
>  
I think that the fact they are recommending snap is because
1) they (and most systems I run into) are mostly Ubuntu based
2) It is not about being leading edge on this. It is about being the exact
things the Certbot team have tested and know to work. That is harder to
manage because we may be ahead/behind/around the bend for anything from
python/perl versions to not having some particular XYZ package.

The first thing would be to listen to what problems they have been having
with Fedora and see if it is something we can solve. It may be that this is
not something that RPM can solve one way or another.




>  --
>  Matthew Miller
>  <mattdm(a)fedoraproject.org&gt;
>  Fedora Project Leader
>  _______________________________________________
>  server mailing list -- server(a)lists.fedoraproject.org
>  To unsubscribe send an email to server-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Stephen J Smoogen.

[Matthew Miller]

On Mon, Feb 01, 2021 at 12:18:27PM -0500, Stephen John Smoogen wrote:
>  The first thing would be to listen to what problems they have been
> having
>  with Fedora and see if it is something we can solve. It may be that this is
>  not something that RPM can solve one way or another. 
Yeah, that's a good point. I note that under the section about installing
with a docker container, they talk about it being impossible to modify the
host. I wonder if we can find some solutions there.


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Stephen Gallagher]

On Mon, Feb 1, 2021 at 11:47 AM Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:
>  See https://pagure.io/fesco/issue/2570 -- in short, Let's Encrypt is
>  recommending installing certbot via snap. While it's good that snap works on
>  Fedora systems, it's not the best experience. 
...

>  3. Promote slick, out-of-the-box certbot as a Fedora Server feature.
> And do
>     whatever it takes to make it really easy. Because that's actually really
>     valuable.
>
>
>  Is this of interest to anyone?
>  
I've had it in the back of my mind to build a Cockpit plugin to manage
the system's and users' certificate stores for quite some time;
building such a solution around Let's Encrypt seems like a great fit.

Some useful prior art we could get inspiration from:
https://www.synology.com/tr-tr/knowledgebase/DSM/tutorial/Network/How_to_...

[Neal Gompa]

On Mon, Feb 1, 2021 at 12:31 PM Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  On Mon, Feb 1, 2021 at 11:47 AM Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:
>  >
>  >
>  > See https://pagure.io/fesco/issue/2570 -- in short, Let's Encrypt is
>  > recommending installing certbot via snap. While it's good that snap works on
>  > Fedora systems, it's not the best experience.
>
>  ...
>
>  > 3. Promote slick, out-of-the-box certbot as a Fedora Server feature. And do
>  >    whatever it takes to make it really easy. Because that's actually really
>  >    valuable.
>  >
>  >
>  > Is this of interest to anyone?
>  >
>
>  I've had it in the back of my mind to build a Cockpit plugin to manage
>  the system's and users' certificate stores for quite some time;
>  building such a solution around Let's Encrypt seems like a great fit.
>
>  Some useful prior art we could get inspiration from:
>
> https://www.synology.com/tr-tr/knowledgebase/DSM/tutorial/Network/How_to_...
This may be increasingly interesting even from an Enterprise(TM)
standpoint now that FreeIPA is also an ACME provider that you could
use Certbot with.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Peter Boy]

>  Am 01.02.2021 um 17:46 schrieb Matthew Miller
> <mattdm(a)fedoraproject.org&gt;:
>  
>  recommending installing certbot via snap. While it's good that snap works on
>  Fedora systems, it's not the best experience 
It’s really a bad recommendation, not only regarding Fedora but also other non-debian
distributions as SuSE Archlinux, etc.

The main problem in not only
>  most particularly that Fedora sysadmins do not have any experience
> with snap 
(https://pagure.io/fesco/issue/2570)

Snap fiddles with central system configurations, such as the order of libraries in the
system path, which can and does lead to all kinds of conflicts introduced by third-party
snap packages.

The stability of Fedora Server is based precisely on the fact that all packages go through
the same Fedora QA process. We must not compromise on this.

And by the way, what they specifically have in mind in relation to the Fedora letsencrypt
package. I can't recall any problem of the kind that the letsencrypt project cites
there.

[Alexander Bokovoy]

On ti, 02 helmi 2021, Peter Boy wrote:
> > Am 01.02.2021 um 17:46 schrieb Matthew Miller <mattdm(a)fedoraproject.org&gt;:
> >
> > recommending installing certbot via snap. While it's good that snap works on
> > Fedora systems, it's not the best experience
>
> It’s really a bad recommendation, not only regarding Fedora but also
> other non-debian distributions as SuSE Archlinux, etc.
>
> The main problem in not only
> > most particularly that Fedora sysadmins do not have any experience with snap
> (https://pagure.io/fesco/issue/2570)
>
> Snap fiddles with central system configurations, such as the order of
> libraries in the system path, which can and does lead to all kinds of
> conflicts introduced by third-party snap packages.
>
> The stability of Fedora Server is based precisely on the fact that all
> packages go through the same Fedora QA process. We must not compromise
> on this.
>
> And by the way, what they specifically have in mind in relation to the
> Fedora letsencrypt package. I can't recall any problem of the kind that
> the letsencrypt project cites there. 
We (FreeIPA) have support for ACME in FreeIPA 4.9 series which are part
of Fedora Rawhide since last year and yesterday were submitted as
updates to Fedora 32/33. This is a server-side ACME support but we are
testing it with Fedora-provided certbot and mod_md, the two ACME clients
we have in Fedora. These tests are part of our CI testing process
upstream.

Fraser Tweedale blogged along the way of adding ACME support on how it
works and how integration is done here:
https://frasertweedale.github.io/blog-redhat/tags/acme.html

Here is how we test ACME. Note that this is working against our own ACME
implementation, not Let's Encrypt, but it is a comprehensive test suite
for ACME clients in Fedora and RHEL/CentOS Stream:
https://github.com/freeipa/freeipa/blob/master/ipatests/test_integration/...

We had ups and downs with these tests but the client side mostly works
just fine. As a side note, we are planning to reuse CI pipelines which
we run for RHEL builds and in FreeIPA upstream for testing Fedora
updates. With Fedora 34 we would be able to test ACME clients routinely.

We are also working on a better flow for custom ACME CA discoveries, see
an Internet Draft proposal by Fraser:
https://frasertweedale.github.io/blog-redhat/posts/2020-11-13-acme-servic...

Finally, our prefered tool to manage certificate renewals on Fedora/RHEL
clients is certmonger. It has a pluggable infrastructure and can
integrate with almost anything. For example, there is (incomplete)
plugin that integrates certbot and certmonger to issue Let's Encrypt
certificates for FreeIPA-enrolled clients using dns-01 challenge:
https://github.com/antevens/cerlet


-- 
/ Alexander Bokovoy