F41 Change Proposal: Disable openSSL Engine Support (system-wide)

[Aoife Moloney]

Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.

== Summary ==
We disable support of engines in OpenSSL

== Owner ==
* Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
* Email: dbelyavs(a)redhat.com

== Detailed Description ==
We are going to build OpenSSL without engine support. Engines are not
FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
The engine functionality we are aware of (PKCS#11, TPM) is either
covered by providers or will be covered soon.

== Feedback ==


== Benefit to Fedora ==
We get rid of deprecated functionality and enforce using up-to-date
API. Engine support is deprecated in OpenSSL upstream, and after
provider migration caused some deficiencies with engine support. No
new features will be added to the engine. So we reduce the maintenance
burden and potentially attack surface.

It follows the approach planned for CentOS 10.

== Scope ==
* Proposal owners: maintainers of packages enumerated here:
https://clang.fedorapeople.org/c10s-engine-users/ plus probably owners
of some Fedora-only packages

For most of the packages the maintainers will just have to rebuild
their packages after the OpenSSL change lands in compose. For  several
packages some patches should be implemented to prevent compilation
errors.

* Other developers: -

* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
This change probably requires mass-rebuild.

* Policies and guidelines: We need reject/modify packages providing
OpenSSL engines

* Trademark approval: N/A (not needed for this Change)

* Alignment with Community Initiatives:

== Upgrade/compatibility impact ==
OpenSSL engines will no longer be supported. Engines will not be
supported in openssl configuration files (presumably silently
ignored). Users will have to reconfigure systems to providers if they
use engines.


== How To Test ==
OpenSSL libcrypto.so doesn't export any ENGINE_* symbols (~120 lines).
Application is normally built.



== User Experience ==
Users will have to reconfigure systems to providers if they use
engines. No other changes are expected.


== Dependencies ==
In theory, all OpenSSL-dependent packages. In practice, only those
that explicitly use ENGINE api.



== Contingency Plan ==
Reenable engine support but remove engine header file to allow old
applications work preventing appearing new ones.

* Contingency mechanism: (What to do?  Who will do it?) rebuild
OpenSSL and dependent packages
* Contingency deadline: beta freeze?
* Blocks release? Yes


== Documentation ==
TBD


== Release Notes ==
TBD



-- 
Aoife Moloney

Fedora Operations Architect

Fedora Project

Matrix: @amoloney:fedora.im

IRC: amoloney
--
_______________________________________________
devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org
To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]...
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Zbigniew Jędrzejewski-Szmek]

On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
>  Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  
>  This is a proposed Change for Fedora Linux.
>  This document represents a proposed Change. As part of the Changes
>  process, proposals are publicly announced in order to receive
>  community feedback. This proposal will only be implemented if approved
>  by the Fedora Engineering Steering Committee.
>  
>  == Summary ==
>  We disable support of engines in OpenSSL
>  
>  == Owner ==
>  * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  * Email: dbelyavs(a)redhat.com
>  
>  == Detailed Description ==
>  We are going to build OpenSSL without engine support. Engines are not
>  FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  The engine functionality we are aware of (PKCS#11, TPM) is either
>  covered by providers or will be covered soon.
>  
>  == Feedback ==
>  
>  
>  == Benefit to Fedora ==
>  We get rid of deprecated functionality and enforce using up-to-date
>  API. Engine support is deprecated in OpenSSL upstream, and after
>  provider migration caused some deficiencies with engine support. No
>  new features will be added to the engine. So we reduce the maintenance
>  burden and potentially attack surface. 
Hi,

In systemd, we recently added support for engines in various tools:
- systemd-{repart,measure} have --private-key-source=file|engine|provider
  (this is C code).
- ukify has --signing-engine.
  This is Python code that calls sbsign or pesign to do parts of the
  heavy lifting, and those binaries do not support providers. (At least
  the docs are silent on this, please correct it they do.)

So it seems we'd lose support for signing with keys stored on yubikeys
and tpms and other fancy approaches if the proposed change goes through.

--

Also, what is the impact on:
- kernel module signing in the build system
- signing of shim, grub2, fwupd, and the kernel in the build system
- mokutil

Thanks,
Zbyszek

[Dmitry Belyavskiy]

Hi!

On Wed, Mar 20, 2024 at 9:50 AM Zbigniew Jędrzejewski-Szmek <
zbyszek(a)in.waw.pl&gt; wrote:

>  On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
>  > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  >
>  > This is a proposed Change for Fedora Linux.
>  > This document represents a proposed Change. As part of the Changes
>  > process, proposals are publicly announced in order to receive
>  > community feedback. This proposal will only be implemented if approved
>  > by the Fedora Engineering Steering Committee.
>  >
>  > == Summary ==
>  > We disable support of engines in OpenSSL
>  >
>  > == Owner ==
>  > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  > * Email: dbelyavs(a)redhat.com
>  >
>  > == Detailed Description ==
>  > We are going to build OpenSSL without engine support. Engines are not
>  > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  > The engine functionality we are aware of (PKCS#11, TPM) is either
>  > covered by providers or will be covered soon.
>  >
>  > == Feedback ==
>  >
>  >
>  > == Benefit to Fedora ==
>  > We get rid of deprecated functionality and enforce using up-to-date
>  > API. Engine support is deprecated in OpenSSL upstream, and after
>  > provider migration caused some deficiencies with engine support. No
>  > new features will be added to the engine. So we reduce the maintenance
>  > burden and potentially attack surface.
>
>  Hi,
>
>  In systemd, we recently added support for engines in various tools:
>  - systemd-{repart,measure} have --private-key-source=file|engine|provider
>    (this is C code).
>  
As `provider` is a possible source, you will have to replace `engine` with
a particular provider.
tpm2 provider is on the way to rawhide, and pkcs11 provider has already
landed, so TPMs and Yubikeys



>  - ukify has --signing-engine.
>    This is Python code that calls sbsign or pesign to do parts of the
>    heavy lifting, and those binaries do not support providers. (At least
>    the docs are silent on this, please correct it they do.)
>  
Have no idea but it means we have to change this code

>  So it seems we'd lose support for signing with keys stored on yubikeys
>  and tpms and other fancy approaches if the proposed change goes through.
>  
We don't lose this support but we still have to adjust configurations.


>  --
>
>  Also, what is the impact on:
>  - kernel module signing in the build system
>  - signing of shim, grub2, fwupd, and the kernel in the build system
>  - mokutil
>  
Does any kernel module rely on OpenSSL?


>  Thanks,
>  Zbyszek
>  --
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>  

-- 
Dmitry Belyavskiy

[Peter Robinson]

On Wed, 20 Mar 2024 at 09:05, Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt; wrote:
>  Hi!
>
>  On Wed, Mar 20, 2024 at 9:50 AM Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl&gt;
> wrote:
> >
> > On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
> > > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
> > >
> > > This is a proposed Change for Fedora Linux.
> > > This document represents a proposed Change. As part of the Changes
> > > process, proposals are publicly announced in order to receive
> > > community feedback. This proposal will only be implemented if approved
> > > by the Fedora Engineering Steering Committee.
> > >
> > > == Summary ==
> > > We disable support of engines in OpenSSL
> > >
> > > == Owner ==
> > > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
> > > * Email: dbelyavs(a)redhat.com
> > >
> > > == Detailed Description ==
> > > We are going to build OpenSSL without engine support. Engines are not
> > > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
> > > The engine functionality we are aware of (PKCS#11, TPM) is either
> > > covered by providers or will be covered soon.
> > >
> > > == Feedback ==
> > >
> > >
> > > == Benefit to Fedora ==
> > > We get rid of deprecated functionality and enforce using up-to-date
> > > API. Engine support is deprecated in OpenSSL upstream, and after
> > > provider migration caused some deficiencies with engine support. No
> > > new features will be added to the engine. So we reduce the maintenance
> > > burden and potentially attack surface.
> >
> > Hi,
> >
> > In systemd, we recently added support for engines in various tools:
> > - systemd-{repart,measure} have --private-key-source=file|engine|provider
> >   (this is C code).
>
>
>  As `provider` is a possible source, you will have to replace `engine` with a particular
> provider.
>  tpm2 provider is on the way to rawhide, and pkcs11 provider has already landed, so TPMs
> and Yubikeys
>
>
> >
> > - ukify has --signing-engine.
> >   This is Python code that calls sbsign or pesign to do parts of the
> >   heavy lifting, and those binaries do not support providers. (At least
> >   the docs are silent on this, please correct it they do.)
>
>
>  Have no idea but it means we have to change this code
> >
> >
> > So it seems we'd lose support for signing with keys stored on yubikeys
> > and tpms and other fancy approaches if the proposed change goes through.
>
>
>  We don't lose this support but we still have to adjust configurations.
>
> >
> > --
> >
> > Also, what is the impact on:
> > - kernel module signing in the build system
> > - signing of shim, grub2, fwupd, and the kernel in the build system
> > - mokutil
>
>
>  Does any kernel module rely on OpenSSL? 
No but they use openssl for signing kernel modules, you can see
details in the spec [1], search openssl.

[1] https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel.spec

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 20, 2024 at 10:04:13AM +0100, Dmitry Belyavskiy wrote:
>  > Hi,
>  >
>  > In systemd, we recently added support for engines in various tools:
>  > - systemd-{repart,measure} have --private-key-source=file|engine|provider
>  >   (this is C code).
>  >
>  
>  As `provider` is a possible source, you will have to replace `engine` with
>  a particular provider.
>  tpm2 provider is on the way to rawhide, and pkcs11 provider has already
>  landed, so TPMs and Yubikeys 
$ nm build/src/shared/libsystemd-shared-256.so.0 |rg 'ENGINE.*OPENSSL'           
                 U ENGINE_by_id(a)OPENSSL_3.0.0
                 U ENGINE_free(a)OPENSSL_3.0.0
                 U ENGINE_init(a)OPENSSL_3.0.0
                 U ENGINE_load_private_key(a)OPENSSL_3.0.0

That's how it looks with the upcoming systemd 256.

Zbyszek

[Alexander Bokovoy]

On Срд, 20 сак 2024, Zbigniew Jędrzejewski-Szmek wrote:
> On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
> > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
> >
> > This is a proposed Change for Fedora Linux.
> > This document represents a proposed Change. As part of the Changes
> > process, proposals are publicly announced in order to receive
> > community feedback. This proposal will only be implemented if approved
> > by the Fedora Engineering Steering Committee.
> >
> > == Summary ==
> > We disable support of engines in OpenSSL
> >
> > == Owner ==
> > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
> > * Email: dbelyavs(a)redhat.com
> >
> > == Detailed Description ==
> > We are going to build OpenSSL without engine support. Engines are not
> > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
> > The engine functionality we are aware of (PKCS#11, TPM) is either
> > covered by providers or will be covered soon.
> >
> > == Feedback ==
> >
> >
> > == Benefit to Fedora ==
> > We get rid of deprecated functionality and enforce using up-to-date
> > API. Engine support is deprecated in OpenSSL upstream, and after
> > provider migration caused some deficiencies with engine support. No
> > new features will be added to the engine. So we reduce the maintenance
> > burden and potentially attack surface.
>
> Hi,
>
> In systemd, we recently added support for engines in various tools:
> - systemd-{repart,measure} have --private-key-source=file|engine|provider
>   (this is C code).
> - ukify has --signing-engine.
>   This is Python code that calls sbsign or pesign to do parts of the
>   heavy lifting, and those binaries do not support providers. (At least
>   the docs are silent on this, please correct it they do.)
>
> So it seems we'd lose support for signing with keys stored on yubikeys
> and tpms and other fancy approaches if the proposed change goes through.
>
> --
>
> Also, what is the impact on:
> - kernel module signing in the build system 
scrips/sign-file.c would need to migrate from use of ENGINE_* API to
providers. This is trivial as the only use is to find pkcs11 engine and
then load a private key through it:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/s...
static EVP_PKEY *read_private_key(const char *private_key_name)
{
	EVP_PKEY *private_key;

	if (!strncmp(private_key_name, "pkcs11:", 7)) {
		ENGINE *e;

		ENGINE_load_builtin_engines();
		drain_openssl_errors();
		e = ENGINE_by_id("pkcs11");
		ERR(!e, "Load PKCS#11 ENGINE");
		if (ENGINE_init(e))
			drain_openssl_errors();
		else
			ERR(1, "ENGINE_init");
		if (key_pass)
			ERR(!ENGINE_ctrl_cmd_string(e, "PIN", key_pass, 0),
			    "Set PKCS#11 PIN");
		private_key = ENGINE_load_private_key(e, private_key_name,
						      NULL, NULL);
		ERR(!private_key, "%s", private_key_name);
	} else {
		BIO *b;

		b = BIO_new_file(private_key_name, "rb");
		ERR(!b, "%s", private_key_name);
		private_key = PEM_read_bio_PrivateKey(b, NULL, pem_pw_cb,
						      NULL);
		ERR(!private_key, "%s", private_key_name);
		BIO_free(b);
	}

	return private_key;
}

Dmitry, I think it is something your team needs to handle (submit
support for provider vs engine to Linux kernel upstream).

> - signing of shim, grub2, fwupd, and the kernel in the build system
> - mokutil 
mokutil does not use ENGINE_* APIs at all.



-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Luca Boccassi]

There are 2 major issues with this:

1) A lot of site-specific build systems implement signing via private/local/proprietary
engines, which means those build systems will no longer be able to run on Fedora (and if
this spreads to CentOS/RHEL, those too)
2) Even open source providers are still mostly broken, missing core functionality, and
largely in a "developers preview" state and years of work away from being
anywhere close to stability and reliability of engines. When adding engines support to
various systemd tools recently, I tried to use the tpm2 and pkcs11 providers, and just
gave up, as there was simply no way to make them work, they are simply not fit for purpose
at this stage.

[Daniel P. Berrangé]

On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
>  Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  
>  This is a proposed Change for Fedora Linux.
>  This document represents a proposed Change. As part of the Changes
>  process, proposals are publicly announced in order to receive
>  community feedback. This proposal will only be implemented if approved
>  by the Fedora Engineering Steering Committee.
>  
>  == Summary ==
>  We disable support of engines in OpenSSL
>  
>  == Owner ==
>  * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  * Email: dbelyavs(a)redhat.com
>  
>  == Detailed Description ==
>  We are going to build OpenSSL without engine support. Engines are not
>  FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  The engine functionality we are aware of (PKCS#11, TPM) is either
>  covered by providers or will be covered soon. 
"will be covered soon"

... so lets wait until that work is actually complete before
removing this from openssl, otherwise there's a window of
brokenness in Fedora where the old feature is removed and
the new feature is not ready.

>  == Benefit to Fedora ==
>  We get rid of deprecated functionality and enforce using up-to-date
>  API. Engine support is deprecated in OpenSSL upstream, and after
>  provider migration caused some deficiencies with engine support. No
>  new features will be added to the engine. So we reduce the maintenance
>  burden and potentially attack surface. 
What is upstream's intention with the 'engine' feature deprecation ?

Are they going actively remove this functionality after some
period of deprecation ? If so what's upstream timeframe, and
should Fedora just wait for that, rather than jumping the
gun ?


>  == Upgrade/compatibility impact ==
>  OpenSSL engines will no longer be supported. Engines will not be
>  supported in openssl configuration files (presumably silently
>  ignored). Users will have to reconfigure systems to providers if they
>  use engines.
>  
>  
>  == How To Test ==
>  OpenSSL libcrypto.so doesn't export any ENGINE_* symbols (~120 lines).
>  Application is normally built. 
Removing symbols is an ABI break, so would imply the need for
an SONAME version bump. This is not normally something that
downstreams should ever touch though - it is an upstream
decision when to bump their SONAME version.

Should we not preserve the ENGINE_* symbols, but turn
their impl into either a no-op, or reporting a runtime
error, as appropriate for each API.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Dmitry Belyavskiy]

Dear Daniel,

On Wed, Mar 20, 2024 at 1:44 PM Daniel P. Berrangé <berrange(a)redhat.com&gt;
wrote:

>  On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
>  > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  >
>  > This is a proposed Change for Fedora Linux.
>  > This document represents a proposed Change. As part of the Changes
>  > process, proposals are publicly announced in order to receive
>  > community feedback. This proposal will only be implemented if approved
>  > by the Fedora Engineering Steering Committee.
>  >
>  > == Summary ==
>  > We disable support of engines in OpenSSL
>  >
>  > == Owner ==
>  > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  > * Email: dbelyavs(a)redhat.com
>  >
>  > == Detailed Description ==
>  > We are going to build OpenSSL without engine support. Engines are not
>  > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  > The engine functionality we are aware of (PKCS#11, TPM) is either
>  > covered by providers or will be covered soon.
>
>  "will be covered soon"
>
>  ... so lets wait until that work is actually complete before
>  removing this from openssl, otherwise there's a window of
>  brokenness in Fedora where the old feature is removed and
>  the new feature is not ready.
>  
I am not going to land this change until the tpm2 provider is landed in
Fedora.
But the affected packages must start prepare to this change as early as
possible.


>  > == Benefit to Fedora ==
>  > We get rid of deprecated functionality and enforce using up-to-date
>  > API. Engine support is deprecated in OpenSSL upstream, and after
>  > provider migration caused some deficiencies with engine support. No
>  > new features will be added to the engine. So we reduce the maintenance
>  > burden and potentially attack surface.
>
>  What is upstream's intention with the 'engine' feature deprecation ?
>
>  Are they going actively remove this functionality after some
>  period of deprecation ? If so what's upstream timeframe, and
>  should Fedora just wait for that, rather than jumping the
>  gun ?
>  
As I understand, upstream is going to remove engines but it wouldn't happen
before OpenSSL 4.0
I don't think Fedora should wait for that. We definitely want to land
no-engine in RHEL10 so Fedora should be ready for that.


>  > == Upgrade/compatibility impact ==
>  > OpenSSL engines will no longer be supported. Engines will not be
>  > supported in openssl configuration files (presumably silently
>  > ignored). Users will have to reconfigure systems to providers if they
>  > use engines.
>  >
>  >
>  > == How To Test ==
>  > OpenSSL libcrypto.so doesn't export any ENGINE_* symbols (~120 lines).
>  > Application is normally built.
>
>  Removing symbols is an ABI break, so would imply the need for
>  an SONAME version bump. This is not normally something that
>  downstreams should ever touch though - it is an upstream
>  decision when to bump their SONAME version.
>
>  Should we not preserve the ENGINE_* symbols, but turn
>  their impl into either a no-op, or reporting a runtime
>  error, as appropriate for each API.
>  
All 100+ symbols? I don't think providing non-working stubs would be a good
idea...

-- 
Dmitry Belyavskiy

[Daniel P. Berrangé]

On Wed, Mar 20, 2024 at 02:35:21PM +0100, Dmitry Belyavskiy wrote:
>  Dear Daniel,
>  
>  On Wed, Mar 20, 2024 at 1:44 PM Daniel P. Berrangé <berrange(a)redhat.com&gt;
>  wrote:
>  
>  > On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
>  > > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  > >
>  > > This is a proposed Change for Fedora Linux.
>  > > This document represents a proposed Change. As part of the Changes
>  > > process, proposals are publicly announced in order to receive
>  > > community feedback. This proposal will only be implemented if approved
>  > > by the Fedora Engineering Steering Committee.
>  > >
>  > > == Summary ==
>  > > We disable support of engines in OpenSSL
>  > >
>  > > == Owner ==
>  > > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  > > * Email: dbelyavs(a)redhat.com
>  > >
>  > > == Detailed Description ==
>  > > We are going to build OpenSSL without engine support. Engines are not
>  > > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  > > The engine functionality we are aware of (PKCS#11, TPM) is either
>  > > covered by providers or will be covered soon.
>  >
>  > "will be covered soon"
>  >
>  > ... so lets wait until that work is actually complete before
>  > removing this from openssl, otherwise there's a window of
>  > brokenness in Fedora where the old feature is removed and
>  > the new feature is not ready.
>  >
>  
>  I am not going to land this change until the tpm2 provider is landed in
>  Fedora. 
Lets explicitly note that as a blocking /  pre-requisite dependency
for landing this change then.

>  > > == Benefit to Fedora ==
>  > > We get rid of deprecated functionality and enforce using up-to-date
>  > > API. Engine support is deprecated in OpenSSL upstream, and after
>  > > provider migration caused some deficiencies with engine support. No
>  > > new features will be added to the engine. So we reduce the maintenance
>  > > burden and potentially attack surface.
>  >
>  > What is upstream's intention with the 'engine' feature deprecation ?
>  >
>  > Are they going actively remove this functionality after some
>  > period of deprecation ? If so what's upstream timeframe, and
>  > should Fedora just wait for that, rather than jumping the
>  > gun ?
>  >
>  
>  As I understand, upstream is going to remove engines but it wouldn't happen
>  before OpenSSL 4.0 
That makes sense, as it solves the ELF ABI / SONAME change
issue with removing the APIs.

>  I don't think Fedora should wait for that. We definitely want to
> land
>  no-engine in RHEL10 so Fedora should be ready for that. 
Fedora shouldn't neccessarily be rushed into something just because
RHEL wants to do it prematurely due to RHEL's long lifecycle.

>  > > == Upgrade/compatibility impact ==
>  > > OpenSSL engines will no longer be supported. Engines will not be
>  > > supported in openssl configuration files (presumably silently
>  > > ignored). Users will have to reconfigure systems to providers if they
>  > > use engines.
>  > >
>  > >
>  > > == How To Test ==
>  > > OpenSSL libcrypto.so doesn't export any ENGINE_* symbols (~120 lines).
>  > > Application is normally built.
>  >
>  > Removing symbols is an ABI break, so would imply the need for
>  > an SONAME version bump. This is not normally something that
>  > downstreams should ever touch though - it is an upstream
>  > decision when to bump their SONAME version.
>  >
>  > Should we not preserve the ENGINE_* symbols, but turn
>  > their impl into either a no-op, or reporting a runtime
>  > error, as appropriate for each API.
>  >
>  
>  All 100+ symbols? I don't think providing non-working stubs would be a good
>  idea... 
Intentionally forking the upstream ABI is worse IMHO.

Consider you've built your own app on Fedora 39 that uses these
symbols, and now upgrade to F40. RPM will consider the dependency
still satisfied, as the SONAME hasn't changed on libcrypto. The
app throws linker errors at some point due to the missing symbols.

Another alternative is to continue providing fully functional engine
symbols, but remove the header files so in practice you can't compile
something new that uses it. This is still forking the API, but at least
has not forked the ELF ABI, so the upgrade doesn't explode.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Fabio Valentini]

On Wed, Mar 20, 2024 at 3:06 PM Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote:
>  On Wed, Mar 20, 2024 at 02:35:21PM +0100, Dmitry Belyavskiy wrote: 
(...)

>  > As I understand, upstream is going to remove engines but it
> wouldn't happen
>  > before OpenSSL 4.0
>
>  That makes sense, as it solves the ELF ABI / SONAME change
>  issue with removing the APIs.
>
>  > I don't think Fedora should wait for that. We definitely want to land
>  > no-engine in RHEL10 so Fedora should be ready for that.
>
>  Fedora shouldn't neccessarily be rushed into something just because
>  RHEL wants to do it prematurely due to RHEL's long lifecycle. 
I fully agree here. Just because something is desirable for RHEL x+1
doesn't mean that it's desirable for Fedora y+1.
Also, hasn't CentOS Stream 10 already been branched off of
ELN-is-Fedora-40, meaning it would be too late if done in Fedora 41
anyway?
Or is this just about OpenSSL maintainers not wanting to have to keep
maintaining Engine support in Fedora while it will be phased out in
RHEL sooner?

Fabio

[Dmitry Belyavskiy]

Dear Fabio,

On Wed, Mar 20, 2024 at 3:18 PM Fabio Valentini <decathorpe(a)gmail.com&gt;
wrote:

>  On Wed, Mar 20, 2024 at 3:06 PM Daniel P. Berrangé
> <berrange(a)redhat.com&gt;
>  wrote:
>  >
>  > On Wed, Mar 20, 2024 at 02:35:21PM +0100, Dmitry Belyavskiy wrote:
>
>  (...)
>
>  > > As I understand, upstream is going to remove engines but it wouldn't
>  happen
>  > > before OpenSSL 4.0
>  >
>  > That makes sense, as it solves the ELF ABI / SONAME change
>  > issue with removing the APIs.
>  >
>  > > I don't think Fedora should wait for that. We definitely want to land
>  > > no-engine in RHEL10 so Fedora should be ready for that.
>  >
>  > Fedora shouldn't neccessarily be rushed into something just because
>  > RHEL wants to do it prematurely due to RHEL's long lifecycle.
>
>  I fully agree here. Just because something is desirable for RHEL x+1
>  doesn't mean that it's desirable for Fedora y+1.
>  Also, hasn't CentOS Stream 10 already been branched off of
>  ELN-is-Fedora-40, meaning it would be too late if done in Fedora 41
>  anyway?
>  
I agree that it's not strictly necessary for F41. I'm pretty sure that if
we want it in F42 we have to start now.
At least the feedback is valuable and we can take it into account earlier.

Or is this just about OpenSSL maintainers not wanting to have to keep
>  maintaining Engine support in Fedora while it will be phased out in
>  RHEL sooner?
>  
Yes. We already have quite bad experience of maintaining slightly different
versions of OpenSSL and I'd like to keep the versions as close as possible.

-- 
Dmitry Belyavskiy

[Petr Pisar]

V Wed, Mar 20, 2024 at 02:05:52PM +0000, Daniel P. Berrangé napsal(a):
>  Consider you've built your own app on Fedora 39 that uses these
>  symbols, and now upgrade to F40. RPM will consider the dependency
>  still satisfied, as the SONAME hasn't changed on libcrypto. The
>  app throws linker errors at some point due to the missing symbols.
>  
>  Another alternative is to continue providing fully functional engine
>  symbols, but remove the header files so in practice you can't compile
>  something new that uses it. This is still forking the API, but at least
>  has not forked the ELF ABI, so the upgrade doesn't explode.
>   
Another option is remove the symbols, change soname, and rebuild reverse
dependencies.

-- Petr

[Daniel P. Berrangé]

On Wed, Mar 20, 2024 at 03:27:34PM +0100, Petr Pisar wrote:
>  V Wed, Mar 20, 2024 at 02:05:52PM +0000, Daniel P. Berrangé
> napsal(a):
>  > Consider you've built your own app on Fedora 39 that uses these
>  > symbols, and now upgrade to F40. RPM will consider the dependency
>  > still satisfied, as the SONAME hasn't changed on libcrypto. The
>  > app throws linker errors at some point due to the missing symbols.
>  > 
>  > Another alternative is to continue providing fully functional engine
>  > symbols, but remove the header files so in practice you can't compile
>  > something new that uses it. This is still forking the API, but at least
>  > has not forked the ELF ABI, so the upgrade doesn't explode.
>  > 
>  Another option is remove the symbols, change soname, and rebuild reverse
>  dependencies. 
Changing soname is something I don't think distros should ever do. ELF
soname designation belongs to the upstream project maintainers.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Neal Gompa]

On Wed, Mar 20, 2024 at 11:24 AM Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote:
>  On Wed, Mar 20, 2024 at 03:27:34PM +0100, Petr Pisar wrote:
>  > V Wed, Mar 20, 2024 at 02:05:52PM +0000, Daniel P. Berrangé napsal(a):
>  > > Consider you've built your own app on Fedora 39 that uses these
>  > > symbols, and now upgrade to F40. RPM will consider the dependency
>  > > still satisfied, as the SONAME hasn't changed on libcrypto. The
>  > > app throws linker errors at some point due to the missing symbols.
>  > >
>  > > Another alternative is to continue providing fully functional engine
>  > > symbols, but remove the header files so in practice you can't compile
>  > > something new that uses it. This is still forking the API, but at least
>  > > has not forked the ELF ABI, so the upgrade doesn't explode.
>  > >
>  > Another option is remove the symbols, change soname, and rebuild reverse
>  > dependencies.
>
>  Changing soname is something I don't think distros should ever do. ELF
>  soname designation belongs to the upstream project maintainers.
>  
I agree with this. It was a royal pain to get us to stop doing that
with OpenSSL 1.1, I don't want to go back to having to field bug
reports about broken OpenSSL sonames again.

While it is technically out of scope to discuss CentOS Stream 10 here,
I am not sure it is wise to drop the engines API there either. It will
result in tremendous problems for consumers and while deprecated,
OpenSSL 4.0 (which removes deprecated APIs) has no currently planned
release date: https://github.com/openssl/openssl/milestone/24

Even if we're being generous and saying it'll arrive in 3 years,
that's still far enough away that we're talking about Fedora 46 (!!)
and RHEL 11.

The amount of damage and breakage for third-parties by disabling
engine support is unconscionable, in my opinion.

From the Fedora perspective, I just see no reason to do this anytime soon.




--
真実はいつも一つ！/ Always, there's only one truth!

[Dmitry Belyavskiy]

Dear Daniel,

On Wed, Mar 20, 2024 at 3:06 PM Daniel P. Berrangé <berrange(a)redhat.com&gt;
wrote:

>  On Wed, Mar 20, 2024 at 02:35:21PM +0100, Dmitry Belyavskiy wrote:
>  > Dear Daniel,
>  >
>  > On Wed, Mar 20, 2024 at 1:44 PM Daniel P. Berrangé <berrange(a)redhat.com&gt;
>  > wrote:
>  >
>  > > On Fri, Mar 08, 2024 at 08:37:19PM +0000, Aoife Moloney wrote:
>  > > > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  > > >
>  > > > This is a proposed Change for Fedora Linux.
>  > > > This document represents a proposed Change. As part of the Changes
>  > > > process, proposals are publicly announced in order to receive
>  > > > community feedback. This proposal will only be implemented if
>  approved
>  > > > by the Fedora Engineering Steering Committee.
>  > > >
>  > > > == Summary ==
>  > > > We disable support of engines in OpenSSL
>  > > >
>  > > > == Owner ==
>  > > > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  > > > * Email: dbelyavs(a)redhat.com
>  > > >
>  > > > == Detailed Description ==
>  > > > We are going to build OpenSSL without engine support. Engines are not
>  > > > FIPS compatible and corresponding API is deprecated since OpenSSL
>  3.0.
>  > > > The engine functionality we are aware of (PKCS#11, TPM) is either
>  > > > covered by providers or will be covered soon.
>  > >
>  > > "will be covered soon"
>  > >
>  > > ... so lets wait until that work is actually complete before
>  > > removing this from openssl, otherwise there's a window of
>  > > brokenness in Fedora where the old feature is removed and
>  > > the new feature is not ready.
>  > >
>  >
>  > I am not going to land this change until the tpm2 provider is landed in
>  > Fedora.
>
>  Lets explicitly note that as a blocking /  pre-requisite dependency
>  for landing this change then.
>  
Fair point, will do.


>  > > Removing symbols is an ABI break, so would imply the need
> for
>  > > an SONAME version bump. This is not normally something that
>  > > downstreams should ever touch though - it is an upstream
>  > > decision when to bump their SONAME version.
>  > >
>  > > Should we not preserve the ENGINE_* symbols, but turn
>  > > their impl into either a no-op, or reporting a runtime
>  > > error, as appropriate for each API.
>  > >
>  >
>  > All 100+ symbols? I don't think providing non-working stubs would be a
>  good
>  > idea...
>
>  Intentionally forking the upstream ABI is worse IMHO.
>
>  Consider you've built your own app on Fedora 39 that uses these
>  symbols, and now upgrade to F40. RPM will consider the dependency
>  still satisfied, as the SONAME hasn't changed on libcrypto. The
>  app throws linker errors at some point due to the missing symbols.
>
>  Another alternative is to continue providing fully functional engine
>  symbols, but remove the header files so in practice you can't compile
>  something new that uses it. This is still forking the API, but at least
>  has not forked the ELF ABI, so the upgrade doesn't explode.
>  
I agree that this option is on the table.
-- 
Dmitry Belyavskiy

[Joe Orton]

On Wed, Mar 20, 2024 at 02:05:52PM +0000, Daniel Berrange wrote:
>  Another alternative is to continue providing fully functional engine
>  symbols, but remove the header files so in practice you can't compile
>  something new that uses it. This is still forking the API, but at least
>  has not forked the ELF ABI, so the upgrade doesn't explode. 
This is a really good idea, I hope Daniel's comment is not lost here.

In fact no need to remove the header files - adding the required:

#define OPENSSL_NO_ENGINE 

into <openssl/configuration-%{arch}.h> will make the OpenSSL API act as 
if it was built with the no-engine option - this would not be an API 
fork since it's one of many configurations supported upstream.

It will have the desired effect of disabling ENGINE support across most 
of Fedora in the next mass-rebuild. Or at least we can easily track down 
the places where the detection isn't perfect, they will break at compile 
time.

Regards, Joe

[Clemens Lang]

Hi,

>  On 20. Mar 2024, at 18:11, Joe Orton <jorton(a)redhat.com&gt;
> wrote:
>  
>  On Wed, Mar 20, 2024 at 02:05:52PM +0000, Daniel Berrange wrote:
> > Another alternative is to continue providing fully functional engine
> > symbols, but remove the header files so in practice you can't compile
> > something new that uses it. This is still forking the API, but at least
> > has not forked the ELF ABI, so the upgrade doesn't explode.
>  
>  This is a really good idea, I hope Daniel's comment is not lost here. 
It isn’t, we are very much aware of this possibility and have already discussed it
before.

The ENGINE API is a source of subtle bugs especially when used together with providers,
though, so we would really prefer to disable it completely rather than keeping it around.
PKEY objects backed by an ENGINE use a number of different lesser tested code paths in the
OpenSSL source code, which lead to all sorts of weird an inconsistent behavior.


>  On 20. Mar 2024, at 16:07, Zbigniew Jędrzejewski-Szmek
> <zbyszek(a)in.waw.pl&gt; wrote:
>  
>  Sorry, but the idea to drop symbols without changing the SONAME
>  must be rejected. If upstream plans to drop the symbols for 4.0, then
>  that is OK, assuming that the SONAME is bumped then. 

This does already not match reality. OpenSSL provides a number of symbols that are
dependent on compile-time options. Take a look at [1, 2]. Every symbol in there that is
tagged with EXIST::FUNCTION followed by a string that is not DEPRECATEDIN_3_0 is only
present when the associated configure-time option is enabled.

For example, an OpenSSL configured with no-des will not provide the DES_xcbc_encrypt
symbol. The assumption that one specific SOVERSION of OpenSSL will always contain the same
symbols on all operating systems is thus incorrect. Obviously this is not a change that
can be made during the lifetime of a specific Fedora release, but ahead of a mass rebuild,
it should be doable to land a change such as this without changing the SONAME, considering
the problem you want to avoid with the SONAME bump *already* exists between different
builds of OpenSSL with different configuration.


[1]: https://github.com/openssl/openssl/blob/master/util/libssl.num
[2]: https://github.com/openssl/openssl/blob/master/util/libcrypto.num


-- 
Clemens Lang
RHEL Crypto Team
Red Hat

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 20, 2024 at 07:15:56PM +0100, Clemens Lang wrote:
>  Hi,
>  
>  > On 20. Mar 2024, at 18:11, Joe Orton <jorton(a)redhat.com&gt; wrote:
>  > 
>  > On Wed, Mar 20, 2024 at 02:05:52PM +0000, Daniel Berrange wrote:
>  >> Another alternative is to continue providing fully functional engine
>  >> symbols, but remove the header files so in practice you can't compile
>  >> something new that uses it. This is still forking the API, but at least
>  >> has not forked the ELF ABI, so the upgrade doesn't explode.
>  > 
>  > This is a really good idea, I hope Daniel's comment is not lost here.
>  
>  It isn’t, we are very much aware of this possibility and have already discussed it
> before.
>  
>  The ENGINE API is a source of subtle bugs especially when used together with providers,
> though, so we would really prefer to disable it completely rather than keeping it around.
> PKEY objects backed by an ENGINE use a number of different lesser tested code paths in the
> OpenSSL source code, which lead to all sorts of weird an inconsistent behavior.
>  
>  
>  > On 20. Mar 2024, at 16:07, Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl&gt;
> wrote:
>  > 
>  > Sorry, but the idea to drop symbols without changing the SONAME
>  > must be rejected. If upstream plans to drop the symbols for 4.0, then
>  > that is OK, assuming that the SONAME is bumped then.
>  
>  
>  This does already not match reality. OpenSSL provides a number of symbols that are
> dependent on compile-time options. Take a look at [1, 2]. Every symbol in there that is
> tagged with EXIST::FUNCTION followed by a string that is not DEPRECATEDIN_3_0 is only
> present when the associated configure-time option is enabled.
>  
>  For example, an OpenSSL configured with no-des will not provide the DES_xcbc_encrypt
> symbol. The assumption that one specific SOVERSION of OpenSSL will always contain the same
> symbols on all operating systems is thus incorrect. Obviously this is not a change that
> can be made during the lifetime of a specific Fedora release, but ahead of a mass rebuild,
> it should be doable to land a change such as this without changing the SONAME, considering
> the problem you want to avoid with the SONAME bump *already* exists between different
> builds of OpenSSL with different configuration. 
I expect the following behaviour from Fedora (or any distro that
cares about stability for users):
- For a given SONAME, symbols are only ever added, never removed.
- Ideally, symbol versioning is used.
- ABI stability is also maintained for a given SONAME.
- If support for a given symbol needs to be removed, it can be replaced
  by a stub that is noop or returns an error, as appropriate.
  This approach was discussed earlier in the thread.
  (This can be useful e.g. for functions that implement some optional
   functionality and programs using those functions can still work
   even if those specific functions stop working. For example, a
   library that implements a bunch of compression methods, and one
   of them starts returning an error, and the programs will fall back
   to a different compression method, is a case where this could be
   useful.)
- SONAME changes happen only if absolutely necessary.

This means that if I compile a program against a library in Fedora,
it'll continue to function indefinitely, in the sense of the dynamic
linker loading the program without fuss.

If the SONAME changes, I'll need to either recompile or install the
older version of the library with the old SONAME. Dependent packages,
both from the distro and external, can use rpm dependencies to prevent
upgrades that'd break programs.

Obviously, the items in the list above are much easier if the library
upstream does the right things.

So please, don't ever change compilation options in Fedora in a way
that would cause symbols to disappear. Certainly not during one release.
But even doing this at a release boundary is bad.

Zbyszek

[Gary Buhrmaster]

On Wed, Mar 20, 2024 at 1:36 PM Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt; wrote:

>  As I understand, upstream is going to remove engines but it wouldn't happen before
> OpenSSL 4.0
>  I don't think Fedora should wait for that. We definitely want to land no-engine in
> RHEL10 so Fedora should be ready for that.
>  
What is the targeted time frame for release of OpenSSL 4.0?

Last I knew it was not soon(ish).  And (for Fedora), I would
expect for a number of years distros are likely going to
have to ship both openssl 3.x and openssl 4.x libraries for
compatibility (just as many still ship openssl 1.1), and
engine support may be a compatibility requirement.

I would think the OpenSSL 4.0 timeframe is when
engine support should be dropped (I believe the
3.x headers already warn about deprecation, and
people have the option to start porting code now,
although many are likely to not do so until it breaks).

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 20, 2024 at 02:35:21PM +0100, Dmitry Belyavskiy wrote:
>  > > == Benefit to Fedora ==
>  > > We get rid of deprecated functionality and enforce using up-to-date
>  > > API. Engine support is deprecated in OpenSSL upstream, and after
>  > > provider migration caused some deficiencies with engine support. No
>  > > new features will be added to the engine. So we reduce the maintenance
>  > > burden and potentially attack surface.
>  >
>  > What is upstream's intention with the 'engine' feature deprecation ?
>  >
>  > Are they going actively remove this functionality after some
>  > period of deprecation ? If so what's upstream timeframe, and
>  > should Fedora just wait for that, rather than jumping the
>  > gun ?
>  >
>  
>  As I understand, upstream is going to remove engines but it wouldn't happen
>  before OpenSSL 4.0
>  I don't think Fedora should wait for that. We definitely want to land
>  no-engine in RHEL10 so Fedora should be ready for that. 
Sorry, but the idea to drop symbols without changing the SONAME
must be rejected. If upstream plans to drop the symbols for 4.0, then
that is OK, assuming that the SONAME is bumped then.

We can try to rebuild distro packages, but we do not control everything
that is built by users. Removing symbols without bumping SONAME will
break user programs.

>  > Should we not preserve the ENGINE_* symbols, but turn
>  > their impl into either a no-op, or reporting a runtime
>  > error, as appropriate for each API.
>  
>  All 100+ symbols? I don't think providing non-working stubs would be a good
>  idea... 
If we were to do this, this would be the least bad option.
It's not that much work to generate a 100 stubs of the form
  ENGINE* ENGINE_by_id(const char*) { return NULL; }

But I don't think we should do the removal at all.

Zbyszek

[Jun Aruga (he / him)]

On Wed, Mar 20, 2024 at 2:36 PM Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt; wrote:
>  
...
>> > == Detailed Description ==
>> > We are going to build OpenSSL without engine support. Engines are not
>> > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>> > The engine functionality we are aware of (PKCS#11, TPM) is either
>> > covered by providers or will be covered soon.
>
>  
>> "will be covered soon"
>
>  
>> ... so lets wait until that work is actually complete
before
>> removing this from openssl, otherwise there's a window of
>> brokenness in Fedora where the old feature is removed and
>> the new feature is not ready.
>  
>  
> I am not going to land this
change until the tpm2 provider is landed in Fedora.
> But the affected packages must start prepare to this change as early as possible.

Hi Dmitry,
Could you provide the upstream OpenSSL project's issue ticket(s) or
pull-request(s) about the feature adding or updating the providers to
cover all the functionalities that engines have?
I would like to track the progress of the work.

Jun

-- 
Jun | He - Him | Timezone: UTC+1 or 2, Czech Republic
See <https://www.worldtimebuddy.com/czech-republic-prague-to-utc> for
the timezone.

[Dmitry Belyavskiy]

Dear Jun,



On Thu, Mar 21, 2024 at 11:04 AM Jun Aruga (he / him) <jaruga(a)redhat.com&gt;
wrote:

>  On Wed, Mar 20, 2024 at 2:36 PM Dmitry Belyavskiy
> <dbelyavs(a)redhat.com&gt;
>  wrote:
>  >
>  ...
>  >> > == Detailed Description ==
>  >> > We are going to build OpenSSL without engine support. Engines are not
>  >> > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  >> > The engine functionality we are aware of (PKCS#11, TPM) is either
>  >> > covered by providers or will be covered soon.
>  >>
>  >> "will be covered soon"
>  >>
>  >> ... so lets wait until that work is actually complete before
>  >> removing this from openssl, otherwise there's a window of
>  >> brokenness in Fedora where the old feature is removed and
>  >> the new feature is not ready.
>  >
>  >
>  > I am not going to land this change until the tpm2 provider is landed in
>  Fedora.
>  > But the affected packages must start prepare to this change as early as
>  possible.
>
>  Hi Dmitry,
>  Could you provide the upstream OpenSSL project's issue ticket(s) or
>  pull-request(s) about the feature adding or updating the providers to
>  cover all the functionalities that engines have?
>  I would like to track the progress of the work.
>  
I'm quite surprised.
I'm pretty sure that providers cover all the functionalities that engines
have.
(It doesn't mean that for each an every engine exists a 1:1 replacing
provider, but it's a question to the authors of these engines)

If you are aware of any deficiencies, could you please let upstream or me
know?

-- 
Dmitry Belyavskiy

[Zbigniew Jędrzejewski-Szmek]

On Thu, Mar 21, 2024 at 12:15:43PM +0100, Dmitry Belyavskiy wrote:
>  Dear Jun,
>  
>  
>  
>  On Thu, Mar 21, 2024 at 11:04 AM Jun Aruga (he / him) <jaruga(a)redhat.com&gt;
>  wrote:
>  
>  > On Wed, Mar 20, 2024 at 2:36 PM Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt;
>  > wrote:
>  > >
>  > ...
>  > >> > == Detailed Description ==
>  > >> > We are going to build OpenSSL without engine support. Engines are not
>  > >> > FIPS compatible and corresponding API is deprecated since OpenSSL
> 3.0.
>  > >> > The engine functionality we are aware of (PKCS#11, TPM) is either
>  > >> > covered by providers or will be covered soon.
>  > >>
>  > >> "will be covered soon"
>  > >>
>  > >> ... so lets wait until that work is actually complete before
>  > >> removing this from openssl, otherwise there's a window of
>  > >> brokenness in Fedora where the old feature is removed and
>  > >> the new feature is not ready.
>  > >
>  > >
>  > > I am not going to land this change until the tpm2 provider is landed in
>  > Fedora.
>  > > But the affected packages must start prepare to this change as early as
>  > possible.
>  >
>  > Hi Dmitry,
>  > Could you provide the upstream OpenSSL project's issue ticket(s) or
>  > pull-request(s) about the feature adding or updating the providers to
>  > cover all the functionalities that engines have?
>  > I would like to track the progress of the work.
>  >
>  
>  I'm quite surprised.
>  I'm pretty sure that providers cover all the functionalities that engines
>  have.
>  (It doesn't mean that for each an every engine exists a 1:1 replacing
>  provider, but it's a question to the authors of these engines)
>  
>  If you are aware of any deficiencies, could you please let upstream or me
>  know? 
https://lists.fedoraproject.org/archives/list/[e-mail].o...
?

Zbyszek

[Dmitry Belyavskiy]

Dear Zbyszek,

On Thu, Mar 21, 2024 at 12:41 PM Zbigniew Jędrzejewski-Szmek <
zbyszek(a)in.waw.pl&gt; wrote:

>  On Thu, Mar 21, 2024 at 12:15:43PM +0100, Dmitry Belyavskiy wrote:
>  

>  > > Hi Dmitry,
>  > > Could you provide the upstream OpenSSL project's issue ticket(s) or
>  > > pull-request(s) about the feature adding or updating the providers to
>  > > cover all the functionalities that engines have?
>  > > I would like to track the progress of the work.
>  > >
>  >
>  > I'm quite surprised.
>  > I'm pretty sure that providers cover all the functionalities that engines
>  > have.
>  > (It doesn't mean that for each an every engine exists a 1:1 replacing
>  > provider, but it's a question to the authors of these engines)
>  >
>  > If you are aware of any deficiencies, could you please let upstream or me
>  > know?
>
>
>
> https://lists.fedoraproject.org/archives/list/[e-mail].o...
>  ?
>  
Unfortunately I don't agree. It doesn't contain any details: what was
tested and what has failed

-- 
Dmitry Belyavskiy

[Luca Boccassi]

For an example of missing critical functionality, see this comment:
https://github.com/systemd/systemd/pull/29539#issuecomment-1760243611

Aside from that, trying to use the pkcs11 and tpm2 providers just ended up with
unintelligible errors being vomited on the console. No, I did not keep a copy of those
tests, and no, I am not going to try again, as I most definitely do not have the time nor
the interest to become an openssl or pkcs11 expert. This stuff needs to "just
work" by default, without spending days tinkering, and engines do that just fine,
because they have been developed for years. Providers are just too new, and those that
exist need a few more years as optional features for enthusiasts and experts to adopt and
fix all the bugs, add all the needed features, and make sure they work out of the box. We
are not there yet, and jumping the gun is not going to help anybody.

[Jun Aruga (he / him)]

On Thu, Mar 21, 2024 at 12:16 PM Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt; wrote:
>  Dear Jun,
>
>
>
>  On Thu, Mar 21, 2024 at 11:04 AM Jun Aruga (he / him) <jaruga(a)redhat.com&gt; wrote:
> >
> > On Wed, Mar 20, 2024 at 2:36 PM Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt; wrote:
> > >
> > ...
> > >> > == Detailed Description ==
> > >> > We are going to build OpenSSL without engine support. Engines are not
> > >> > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
> > >> > The engine functionality we are aware of (PKCS#11, TPM) is either
> > >> > covered by providers or will be covered soon.
> > >>
> > >> "will be covered soon"
> > >>
> > >> ... so lets wait until that work is actually complete before
> > >> removing this from openssl, otherwise there's a window of
> > >> brokenness in Fedora where the old feature is removed and
> > >> the new feature is not ready.
> > >
> > >
> > > I am not going to land this change until the tpm2 provider is landed in Fedora.
> > > But the affected packages must start prepare to this change as early as
> possible.
> >
> > Hi Dmitry,
> > Could you provide the upstream OpenSSL project's issue ticket(s) or
> > pull-request(s) about the feature adding or updating the providers to
> > cover all the functionalities that engines have?
> > I would like to track the progress of the work.
>
>
>  I'm quite surprised.
>  I'm pretty sure that providers cover all the functionalities that engines have.
>  (It doesn't mean that for each an every engine exists a 1:1 replacing provider, but
> it's a question to the authors of these engines)
>
>  If you are aware of any deficiencies, could you please let upstream or me know? 
Hi Dmitry,
Sorry. Maybe I used the terminology "functionality" incorrectly.
I am talking about some features that engines provided are missing in
providers. I see the following issue tickets.

* https://github.com/ruby/openssl/issues/722
>  The Engine API was deprecated in OpenSSL 3 and there seems to be
no alternatives for it at the moment using Provider API. The providers
can only be loaded, but there seems to be no way to load keys using an
uri (for ex. pkcs11 uri scheme)
* https://github.com/ruby/openssl/issues/723
>  GOST engine 
-- 
Jun | He - Him | Timezone: UTC+1 or 2, Czech Republic
See <https://www.worldtimebuddy.com/czech-republic-prague-to-utc> for
the timezone.

[Clemens Lang]

Hi Jan,

>  On 21. Mar 2024, at 14:28, Jun Aruga (he / him)
> <jaruga(a)redhat.com&gt; wrote:
>  
>  
>  * https://github.com/ruby/openssl/issues/722
> > The Engine API was deprecated in OpenSSL 3 and there seems to be
>  no alternatives for it at the moment using Provider API. The providers
>  can only be loaded, but there seems to be no way to load keys using an
>  uri (for ex. pkcs11 uri scheme) 

As I understand that ticket, the functionality exists in OpenSSL, but ruby OpenSSL module
does not expose it.

In any case, some providers are also providing workarounds for this problem. See for
example https://github.com/latchset/pkcs11-provider/pull/328, which allows the PKCS11
provider to work everywhere where a simple PEM private key file is currently supported.
With this, the Ruby OpenSSL module has all the time in the world to make the transition.


-- 
Clemens Lang
RHEL Crypto Team
Red Hat

[Dmitry Belyavskiy]

Dear Jun,

On Thu, Mar 21, 2024 at 2:29 PM Jun Aruga (he / him) <jaruga(a)redhat.com&gt;
wrote:

>  On Thu, Mar 21, 2024 at 12:16 PM Dmitry Belyavskiy
> <dbelyavs(a)redhat.com&gt;
>  wrote:
>  >
>  > Dear Jun,
>  >
>  >
>  >
>  > On Thu, Mar 21, 2024 at 11:04 AM Jun Aruga (he / him) <jaruga(a)redhat.com&gt;
>  wrote:
>  >>
>  >> On Wed, Mar 20, 2024 at 2:36 PM Dmitry Belyavskiy <dbelyavs(a)redhat.com&gt;
>  wrote:
>  >> >
>  >> ...
>  >> >> > == Detailed Description ==
>  >> >> > We are going to build OpenSSL without engine support. Engines are
>  not
>  >> >> > FIPS compatible and corresponding API is deprecated since OpenSSL
>  3.0.
>  >> >> > The engine functionality we are aware of (PKCS#11, TPM) is either
>  >> >> > covered by providers or will be covered soon.
>  >> >>
>  >> >> "will be covered soon"
>  >> >>
>  >> >> ... so lets wait until that work is actually complete before
>  >> >> removing this from openssl, otherwise there's a window of
>  >> >> brokenness in Fedora where the old feature is removed and
>  >> >> the new feature is not ready.
>  >> >
>  >> >
>  >> > I am not going to land this change until the tpm2 provider is landed
>  in Fedora.
>  >> > But the affected packages must start prepare to this change as early
>  as possible.
>  >>
>  >> Hi Dmitry,
>  >> Could you provide the upstream OpenSSL project's issue ticket(s) or
>  >> pull-request(s) about the feature adding or updating the providers to
>  >> cover all the functionalities that engines have?
>  >> I would like to track the progress of the work.
>  >
>  >
>  > I'm quite surprised.
>  > I'm pretty sure that providers cover all the functionalities that
>  engines have.
>  > (It doesn't mean that for each an every engine exists a 1:1 replacing
>  provider, but it's a question to the authors of these engines)
>  >
>  > If you are aware of any deficiencies, could you please let upstream or
>  me know?
>
>  Hi Dmitry,
>  Sorry. Maybe I used the terminology "functionality" incorrectly.
>  I am talking about some features that engines provided are missing in
>  providers. I see the following issue tickets.
>
>  * https://github.com/ruby/openssl/issues/722
>    > The Engine API was deprecated in OpenSSL 3 and there seems to be
>  no alternatives for it at the moment using Provider API. The providers
>  can only be loaded, but there seems to be no way to load keys using an
>  uri (for ex. pkcs11 uri scheme)
>  
I believe the pkcs11-provider is already capable of it.


>  * https://github.com/ruby/openssl/issues/723
>    > GOST engine
>  
I can say a lot about it in private. TL;DR - it's almost abandoned for many
reasons.

-- 
Dmitry Belyavskiy

[Ali Erdinc Koroglu]

On 08/03/2024 22:37, Aoife Moloney wrote:
>  Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  
>  This is a proposed Change for Fedora Linux.
>  This document represents a proposed Change. As part of the Changes
>  process, proposals are publicly announced in order to receive
>  community feedback. This proposal will only be implemented if approved
>  by the Fedora Engineering Steering Committee.
>  
>  == Summary ==
>  We disable support of engines in OpenSSL
>  
>  == Owner ==
>  * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  * Email: dbelyavs(a)redhat.com
>  
>  == Detailed Description ==
>  We are going to build OpenSSL without engine support. Engines are not
>  FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  The engine functionality we are aware of (PKCS#11, TPM) is either
>  covered by providers or will be covered soon.
>  
>  == Feedback ==
>  
>  
>  == Benefit to Fedora ==
>  We get rid of deprecated functionality and enforce using up-to-date
>  API. Engine support is deprecated in OpenSSL upstream, and after
>  provider migration caused some deficiencies with engine support. No
>  new features will be added to the engine. So we reduce the maintenance
>  burden and potentially attack surface.
>  
>  It follows the approach planned for CentOS 10. 
Hi,
We're providing the Intel QuickAssist Technology OpenSSL Engine (QAT_Engine)* in
Fedora and RHEL.
I think we shouldn't rush things to have no-engine environment.

* :
https://www.redhat.com/en/blog/accelerated-encryption-4th-gen-intelr-xeon...

-- 
Ali Erdinc Koroglu
Intel, SSE | Linux OS Systems Engineering

[Alexander Sosedkin]

On Wed, Mar 20, 2024 at 6:52 PM Ali Erdinc Koroglu
<aekoroglu(a)linux.intel.com&gt; wrote:
>  On 08/03/2024 22:37, Aoife Moloney wrote:
>  > Wiki - https://fedoraproject.org/wiki/Changes/OpensslNoEngine
>  >
>  > This is a proposed Change for Fedora Linux.
>  > This document represents a proposed Change. As part of the Changes
>  > process, proposals are publicly announced in order to receive
>  > community feedback. This proposal will only be implemented if approved
>  > by the Fedora Engineering Steering Committee.
>  >
>  > == Summary ==
>  > We disable support of engines in OpenSSL
>  >
>  > == Owner ==
>  > * Name: [[User:Dbelyavs| Dmitry Belyavskiy]]
>  > * Email: dbelyavs(a)redhat.com
>  >
>  > == Detailed Description ==
>  > We are going to build OpenSSL without engine support. Engines are not
>  > FIPS compatible and corresponding API is deprecated since OpenSSL 3.0.
>  > The engine functionality we are aware of (PKCS#11, TPM) is either
>  > covered by providers or will be covered soon.
>  >
>  > == Feedback ==
>  >
>  >
>  > == Benefit to Fedora ==
>  > We get rid of deprecated functionality and enforce using up-to-date
>  > API. Engine support is deprecated in OpenSSL upstream, and after
>  > provider migration caused some deficiencies with engine support. No
>  > new features will be added to the engine. So we reduce the maintenance
>  > burden and potentially attack surface.
>  >
>  > It follows the approach planned for CentOS 10.
>
>  Hi,
>  We're providing the Intel QuickAssist Technology OpenSSL Engine (QAT_Engine)* in
> Fedora and RHEL.
>  I think we shouldn't rush things to have no-engine environment.
>
>  * :
> https://www.redhat.com/en/blog/accelerated-encryption-4th-gen-intelr-xeon...
QAT can be built with --enable-qat_provider:
https://github.com/intel/QAT_Engine/blob/1d248f28a10123f3a681b9910283d6e6...