F30 System-Wide Change Proposal: Fully remove deprecated and unsafe functions from libcrypt
https://fedoraproject.org/wiki/Changes/FullyRemoveDeprecatedAndUnsafeFunctio...
== Summary == This change is about removing binary support for deprecated and unsafe functions and bumping libcrypt.so to libcrypt.so.2.
== Owner == * Name: [[User:besser82 | Björn Esser]] besser82@fedoraproject.org
== Detailed Description == In Fedora 28 we replaced glibc's libcrypt with the fully binary compatible libcrypt library from the external libxcrypt project. There are certain interfaces (encrypt, encrypt_r, setkey, setkey_r) that are mandatory by POSIX or various other standards.
At the time those interfaces have been implemented they internally relied on using the DES encryption algorithm, that today is widely considered unsecure and insufficient for applications which require sane data encryption. There are various recommendations new written code should not use them anymore.
For keeping binary backwards compatibility we made sure the libcrypt library still provides those interfaces for existing applications, but code cannot be compiled against them anymore. For that reason there is no application left in Fedora that uses them. With the latest release of glibc, its upstream developers made sure those interfaces cannot be compiled against their libcrypt anymore, too.
Some users may use software from third-parties that may still use those interfaces silently and possibly sacrificing the security of the user's sensitive data silently.
For that reason those interfaces should genrally not be available by default for existing third-party applications in Fedora anymore. Fedora users should be aware whether they use applications that utilize secure encryption algorithms or not.
To accomplish this we are going to bump the shared object name of libcrypt.so from 1 to 2 and remove all of the functions that are considered unsafe. The maintain POSIX or otherwise compatibility, we keep providing a compatibility library (libcrypt.so.1) in a separated package, that can be installed by the user.
== Benefit to Fedora == Most of the benefits have already been explained in the detailed description above.
For those given reasons the key benefit for Fedora is to provide their users a default installation that ensures no third-party software can use unsafe encryption algorithms silently. If a user still wants to use such applications (or really requires the usage of those few POSIX interfaces otherwise), it is required for them to install an additional compatibility-package prior to using such an application.
== Scope == * Proposal owners: Build the libxcrypt package with bumped so-name, enable compatibility library for third-party applications and rebuild the packages that require libcrypt.so.1. After all packages have been rebuilt strip all hashing methods but the ones provided by former glibc libcrypt from the compatibility library. * Other developers: No involvement needed, but testing their packages still work as expected. * Release engineering: [https://pagure.io/releng/issue/8023 #8023] * List of deliverables: This change does not block any deliverable. * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact == The library with the bumped so-name will be pulled automatically on upgrades by autogenerated rpm dependencies. Users of third-party applications linked against libcrypt.so.1 my need to install the libxcrypt-compat package.
== How To Test == There is no real testing needed, as the common interfaces of libcrypt.so.2 do not differ from the ones of libcrypt.so.1, just the legacy bits, which are not used by any application inside Fedora anymore, are removed. Thus basic testing (does it start?) of the applications using the crypt() (or crypt_gensalt()) interfaces is sufficient.
== User Experience == No user-visible impacts, but maybe the need for manually installing the libxcrypt-compat package for some third-party applications.
== Dependencies == There are about 187 pacakges that require libcrypt.so.1, which need to be rebuild at some point (e.g. during mass-rebuild). There is no immediate rebuild needed as we still provide a reduced, but still sufficient, libcrypt.so.1 library for POSIX compatibility.
== Contingency Plan == # Change the symlink from the libxcrypt-devel package to point to the libcrypt.so.1 library again. # Rebuild all packages, that already require libcrypt.so.2 # Fully revert the so-name bump.
== Documentation == The version of the libxcrypt package included with Fedora 30 now ships the libcrypt.so2 library and does not provide the legacy API functions that have been provided by glibc's libcrypt.so.1. The removed functions by name are encrypt, encrypt_r, setkey, setkey_r, and fcrypt.
If you are using a third-party application that links against those functions, or that is linked against glibc's libcrypt, you may need to install the libxcrypt-compat package manually.
All existing binary executables linked against glibc's libcrypt should work unmodified with the libcrypt.so.1 library supplied by the libxcrypt-compat package.
== Release Notes == See the paragraph about documentation above.
On 02. 01. 19 22:14, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/FullyRemoveDeprecatedAndUnsafeFunctio...
== Summary == This change is about removing binary support for deprecated and unsafe functions and bumping libcrypt.so to libcrypt.so.2.
== Owner ==
- Name: [[User:besser82 | Björn Esser]] besser82@fedoraproject.org
== Detailed Description == In Fedora 28 we replaced glibc's libcrypt with the fully binary compatible libcrypt library from the external libxcrypt project. There are certain interfaces (encrypt, encrypt_r, setkey, setkey_r) that are mandatory by POSIX or various other standards. ...
Any chance you would have this ready in some copr? I'd like to test if Python builds and passes the tests, the replacement in F28 wasn't without problems IIRC.
Am Donnerstag, den 10.01.2019, 23:34 +0100 schrieb Miro Hrončok:
On 02. 01. 19 22:14, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/FullyRemoveDeprecatedAndUnsafeFunctio...
== Summary == This change is about removing binary support for deprecated and unsafe functions and bumping libcrypt.so to libcrypt.so.2.
== Owner ==
- Name: [[User:besser82 | Björn Esser]] besser82@fedoraproject.org
== Detailed Description == In Fedora 28 we replaced glibc's libcrypt with the fully binary compatible libcrypt library from the external libxcrypt project. There are certain interfaces (encrypt, encrypt_r, setkey, setkey_r) that are mandatory by POSIX or various other standards. ...
Any chance you would have this ready in some copr? I'd like to test if Python builds and passes the tests, the replacement in F28 wasn't without problems IIRC.
A COPR with builds of bumped libxcrypt so-name and all recently supported versions of Python is here [1].
The problems in F28 were about `crypt(3)` not being prototyped in `unistd.h` anymore. This change doesn't touch any includes or similar stuff, so I do not expect any fallout.
[1] https://copr.fedorainfracloud.org/coprs/besser82/xcrypt_soname/
Am Freitag, den 11.01.2019, 15:13 +0100 schrieb Björn 'besser82' Esser:
Am Donnerstag, den 10.01.2019, 23:34 +0100 schrieb Miro Hrončok:
On 02. 01. 19 22:14, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/FullyRemoveDeprecatedAndUnsafeFunctio...
== Summary == This change is about removing binary support for deprecated and unsafe functions and bumping libcrypt.so to libcrypt.so.2.
== Owner ==
- Name: [[User:besser82 | Björn Esser]] <
besser82@fedoraproject.org>
== Detailed Description == In Fedora 28 we replaced glibc's libcrypt with the fully binary compatible libcrypt library from the external libxcrypt project. There are certain interfaces (encrypt, encrypt_r, setkey, setkey_r) that are mandatory by POSIX or various other standards. ...
Any chance you would have this ready in some copr? I'd like to test if Python builds and passes the tests, the replacement in F28 wasn't without problems IIRC.
A COPR with builds of bumped libxcrypt so-name and all recently supported versions of Python is here [1].
The problems in F28 were about `crypt(3)` not being prototyped in `unistd.h` anymore. This change doesn't touch any includes or similar stuff, so I do not expect any fallout.
[1] https://copr.fedorainfracloud.org/coprs/besser82/xcrypt_soname/
FYI, the build failures in the COPR are *NOT* releated to libxcrypt nor Python's crypt module.
On 11. 01. 19 16:21, Björn 'besser82' Esser wrote:
Any chance you would have this ready in some copr? I'd like to test if Python builds and passes the tests, the replacement in F28 wasn't without problems IIRC.
A COPR with builds of bumped libxcrypt so-name and all recently supported versions of Python is here [1].
The problems in F28 were about `crypt(3)` not being prototyped in `unistd.h` anymore. This change doesn't touch any includes or similar stuff, so I do not expect any fallout.
[1] https://copr.fedorainfracloud.org/coprs/besser82/xcrypt_soname/
FYI, the build failures in the COPR are *NOT* releated to libxcrypt nor Python's crypt module.
Excellent! Thanks.
On Wed, Jan 02, 2019 at 04:14:59PM -0500, Ben Cotton wrote:
== Detailed Description ==
<snip helpful background>
At the time those interfaces have been implemented they internally relied on using the DES encryption algorithm, that today is widely considered unsecure and insufficient for applications which require sane data encryption. There are various recommendations new written code should not use them anymore.
<big thumbs up> DES has been proven insecure against brute force for over two decades.
<snip a lot of good engineering to make it hard to keep using these broken functions>
Some users may use software from third-parties that may still use those interfaces silently and possibly sacrificing the security of the user's sensitive data silently.
For that reason those interfaces should genrally not be available by default for existing third-party applications in Fedora anymore. Fedora users should be aware whether they use applications that utilize secure encryption algorithms or not.
To accomplish this we are going to bump the shared object name of libcrypt.so from 1 to 2 and remove all of the functions that are considered unsafe. The maintain POSIX or otherwise compatibility, we keep providing a compatibility library (libcrypt.so.1) in a separated package, that can be installed by the user.
But this happens all the time with other libraries, so I doubt an uninformed user will understand you did it on purpose unless you do something more.
How do we plan to describe the package in the summary/description? And if they don't look at that, what clue will the user get that they might want to re-think the install of the compat package?
Maybe this package should be named "libxcrypt-compat-insecure-read-description-before-install"? Or at least *something* that screams "wait, maybe I should look into this more, this isn't standard operating procedure..."?
== User Experience == No user-visible impacts, but maybe the need for manually installing the libxcrypt-compat package for some third-party applications.
This seems a little problematic given the motivation behind this change.
== Documentation == The version of the libxcrypt package included with Fedora 30 now ships the libcrypt.so2 library and does not provide the legacy API functions that have been provided by glibc's libcrypt.so.1. The removed functions by name are encrypt, encrypt_r, setkey, setkey_r, and fcrypt.
If you are using a third-party application that links against those functions, or that is linked against glibc's libcrypt, you may need to install the libxcrypt-compat package manually.
All existing binary executables linked against glibc's libcrypt should work unmodified with the libcrypt.so.1 library supplied by the libxcrypt-compat package.
And I object to nothing in this section informing the user that "those interfaces ... possibly sacrific[e] the security of the user's sensitive data silently." Especially since it appears that this will the wording that goes into the release notes.
== Release Notes == See the paragraph about documentation above.
See objections above.
Am Dienstag, den 15.01.2019, 23:16 -0500 schrieb Scott Schmit:
On Wed, Jan 02, 2019 at 04:14:59PM -0500, Ben Cotton wrote:
== Detailed Description ==
<snip helpful background>
At the time those interfaces have been implemented they internally relied on using the DES encryption algorithm, that today is widely considered unsecure and insufficient for applications which require sane data encryption. There are various recommendations new written code should not use them anymore.
<big thumbs up> DES has been proven insecure against brute force for over two decades.
<snip a lot of good engineering to make it hard to keep using these broken functions>
Some users may use software from third-parties that may still use those interfaces silently and possibly sacrificing the security of the user's sensitive data silently.
For that reason those interfaces should genrally not be available by default for existing third-party applications in Fedora anymore. Fedora users should be aware whether they use applications that utilize secure encryption algorithms or not.
To accomplish this we are going to bump the shared object name of libcrypt.so from 1 to 2 and remove all of the functions that are considered unsafe. The maintain POSIX or otherwise compatibility, we keep providing a compatibility library (libcrypt.so.1) in a separated package, that can be installed by the user.
But this happens all the time with other libraries, so I doubt an uninformed user will understand you did it on purpose unless you do something more.
How do we plan to describe the package in the summary/description? And if they don't look at that, what clue will the user get that they might want to re-think the install of the compat package?
Maybe this package should be named "libxcrypt-compat-insecure-read-description-before-install"? Or at least *something* that screams "wait, maybe I should look into this more, this isn't standard operating procedure..."?
== User Experience == No user-visible impacts, but maybe the need for manually installing the libxcrypt-compat package for some third-party applications.
This seems a little problematic given the motivation behind this change.
== Documentation == The version of the libxcrypt package included with Fedora 30 now ships the libcrypt.so2 library and does not provide the legacy API functions that have been provided by glibc's libcrypt.so.1. The removed functions by name are encrypt, encrypt_r, setkey, setkey_r, and fcrypt.
If you are using a third-party application that links against those functions, or that is linked against glibc's libcrypt, you may need to install the libxcrypt-compat package manually.
All existing binary executables linked against glibc's libcrypt should work unmodified with the libcrypt.so.1 library supplied by the libxcrypt-compat package.
And I object to nothing in this section informing the user that "those interfaces ... possibly sacrific[e] the security of the user's sensitive data silently." Especially since it appears that this will the wording that goes into the release notes.
== Release Notes == See the paragraph about documentation above.
See objections above.
Please have a look at this separate change proposal [1]. It is discussed here [2].
Basically the named unsafe functions are subject to be changed in the compat library to some no-function stubs which still guarantee to be compliant to POSIX and other standarts, so "Average Joe" users do not face that security problem even when installing the compat package.
[1] https://fedoraproject.org/wiki/Changes/libcrypt_so_1_Let_encrypt_encrypt_r_s... [2] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
On Wed, Jan 16, 2019 at 12:05:46PM +0100, Björn 'besser82' Esser wrote:
Am Dienstag, den 15.01.2019, 23:16 -0500 schrieb Scott Schmit:
On Wed, Jan 02, 2019 at 04:14:59PM -0500, Ben Cotton wrote:
== Documentation == The version of the libxcrypt package included with Fedora 30 now ships the libcrypt.so2 library and does not provide the legacy API functions that have been provided by glibc's libcrypt.so.1. The removed functions by name are encrypt, encrypt_r, setkey, setkey_r, and fcrypt.
If you are using a third-party application that links against those functions, or that is linked against glibc's libcrypt, you may need to install the libxcrypt-compat package manually.
All existing binary executables linked against glibc's libcrypt should work unmodified with the libcrypt.so.1 library supplied by the libxcrypt-compat package.
And I object to nothing in this section informing the user that "those interfaces ... possibly sacrific[e] the security of the user's sensitive data silently." Especially since it appears that this will the wording that goes into the release notes.
Please have a look at this separate change proposal [1]. It is discussed here [2].
Basically the named unsafe functions are subject to be changed in the compat library to some no-function stubs which still guarantee to be compliant to POSIX and other standarts, so "Average Joe" users do not face that security problem even when installing the compat package.
FWIW, if that change is required for this one to acheive its aims, then this one should cite the other as a dependency or be the same change.
If that change is also accepted, then that mitigates my concerns.
If it is not accepted (for whatever reason), then as written this change will not achieve its purpose without something more.
(Process aside, I am glad you put some thought into the follow-through, it just wasn't evident from this change proposal's writeup.)
[1] https://fedoraproject.org/wiki/Changes/libcrypt_so_1_Let_encrypt_encrypt_r_s... [2] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Almost all packages (see the list at the very end of this mail), that needed a rebuild for the change [1] have been rebuilt and are linking against `libcrypt.so.2` now.
Before and during the rebuild I've applied small patches to several packages to fixup unrelated FTBFS.
The cockpit and community-mysql packages were rebuilt with testsuites temporarily disabled, as they were FTBFS with testsuite even before starting the rebuilds for the change in libxcrypt for other reasons.
The mesos and trafficserver packages are FTBFS since Fedora 25 and have NOT been rebuilt successfully as expected.
Please test your package *BEFORE* the mass-rebuild in about two weeks from today and report problems / bugs related to this change on Bugzilla against the libxcrypt package of the Fedora component. Your bug should also block rhbz# 1666033, so I can track everything much easier in one place.
Björn aka besser82
[1] https://fedoraproject.org/wiki/Changes/FullyRemoveDeprecatedAndUnsafeFunctio...
***
Rebuilt packages:
389-ds-base accountsservice adanaxisgpl apg apr aprsd apr-util asterisk axmail bacula BitchX burp calamares ccrypt cherokee ckermit claws-mail clisp cockpit collectd community-mysql compat-guile18 conserver couchdb crack crossfire cups-filters cvs cyrus-sasl dcap deepin-control-center deepin-daemon dovecot dpsearch drgeo dropbear epic exim freeradius freeradius-client gambas3 gauche gnumeric gpsdrive graphviz groonga gsequencer gsi-openssh guile guile22 haproxy heimdal httpd hylafax+ idzebra inn ipsec-tools ircd-ratbox isdn4k-utils jabberd jd john kuser lcgdm ldapvi libapreq2 libcli libguestfs librep libreswan libuser libwvstreams lighttpd lua-posix makepasswd mariadb maxima mod_gnutls mod_perl mod_psgi mokutil monit nbdkit net-snmp newscache nginx ocserv opendchub openldap opensmtpd openssh openwsman pam pam_krb5 pam_mysql parrot passwdqc percona-xtrabackup perl php pidgin pl plasma-user-manager postgresql postgresql-pgpool-II ppp proftpd pure-ftpd pypy3 pypy python2 python26 python3 python34 python35 python36 q quagga rapidsvn ratbox-services root rsh ruby rudiments rwhoisd samba sawfish sawfish-pager screen sendmail shadow-utils slim slock squid stress-ng subversion systemd tcsh thttpd tlock tog-pegasus util-linux uwsgi vile virtualbox-guest-additions whois wt x11vnc xlockmore xorg-x11-xdm xrootd xscreensaver ypserv yp-tools
-
Ben Cotton -
Björn 'besser82' Esser -
Miro Hrončok -
Scott Schmit