F35 Change: Broken RPATH will fail rpmbuild (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/Broken_RPATH_will_fail_rpmbuild
== Summary ==
Enable broken RPATH detection
[https://docs.fedoraproject.org/en-US/packaging-guidelines/#_brp_buildroot...
buildroot policy] script by default. This will make the RPM build fail
once a broken RPATH was detected within a binary or a shared library
file. An opt-out mechanism will be provided as well.
== Owner ==
* Name: [[User:cstratak| Charalampos Stratakis]]
* Email: cstratak AT redhat.com
== Detailed Description ==
The dynamic linker and loader (ld.so) is responsible for resolving
runtime dependencies of executables and shared library files through a
search hierarchy. However some packages (usually through their
upstream buildsystems) contain a hard-coded path within their binaries
or .so files, by using the -R or -rpath flag during compilation, which
is called an RPATH. By utilizing RPATH, ELF files can point to
directories to be included in the search path, on runtime, to resolve
their dependencies.
While RPATH can be used for non-standard directories, such as a place
containing private libraries of the project, when it points to a value
already provided by the search path of ld.so, it changes the hierarchy
of the search by placing the system defaults first.
(a) DT_RPATH -> (b) LD_LIBRARY_PATH -> (c) DT_RUNPATH -> (d) cache
(/etc/ld.so.cache) -> (e) system defaults
This could present a variety of issues, such as LD_LIBRARY_PATH
overrides not working, incomplete dependency resolution, loading of
wrong libraries etc. In general, changing the default search hierarchy
could lead to unforeseen bugs and issues - in a similar manner as
adding /usr/lib64 to LD_LIBRARY_PATH.
Another problem of a hardcoded RPATH is security. When an ELF object
contains an RPATH pointed to a directory not managed by the system,
where some malicious actor has write permissions to, it's relatively
easy to execute arbitrary code.
Performance can be also affected, since probing explicitly e.g.
/usr/lib64 through RPATH adds extra open/openat system calls to the
process startup.
In Fedora the use of such RPATH is
[https://docs.fedoraproject.org/en-US/packaging-guidelines/#_beware_of_rpath
forbidden], but it was never enforced. This change intends to ratify
that by executing `/usr/lib/rpm/check-rpaths` during rpmbuild, after
%install, and fail the build if an RPATH entry was detected. This
change will not affect RPATH's pointing to private libraries.
=== Definition of a broken RPATH ===
This change will use the
[https://github.com/rpm-software-management/rpm/blob/master/scripts/check-...
rpm script] for checking the broken RPATH's.
The categories are:
* standard RPATHs (e.g. `/usr/lib` or `/usr/lib64`); such RPATHs are a
minor issue but are introducing redundant searchpaths without
providing a benefit. They can also cause errors in multilib
environments.
* invalid RPATHs; these are RPATHs which are neither absolute nor
relative filenames and can therefore be a SECURITY risk
* insecure RPATHs; these are relative RPATHs which are a SECURITY risk
* the special `$ORIGIN` RPATHs are appearing after other RPATHs; this
is just a minor issue but usually unwanted
* the RPATH is empty; there is no reason for such RPATHs and they
cause unneeded work while loading libraries
* an RPATH references `..` of an absolute path; this will break the
functionality when the path before `..` is a symlink
=== Opting out ===
A standard opt-out mechanism is provided, same as the other
[https://docs.fedoraproject.org/en-US/packaging-guidelines/#_brp_buildroot...
buildroot policy scripts], if the script provides incorrect results
for your package. Simply add `%define __brp_check_rpaths %{nil}` on
top of your SPEC. According to the guidelines, any package that
disables a BRP script this way, MUST also note the reason in an
accompanying comment.
== Feedback ==
The change [https://pagure.io/packaging-committee/issue/886 has been
proposed] a long time ago through FPC and the general consensus is
that it needs to be done along with an overhaul of the Fedora
documentation in regards to RPATH.
An [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
email thread] was also started on Fedora devel regarding this change.
There have been multiple requests in the past to enable that check, as
well as various attempts to remove RPATH's from packages in the
distro. [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
0] [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
1][https://lists.fedoraproject.org/archives/list/devel@lists.fedoraprojec...
2][https://lists.fedoraproject.org/archives/list/devel@lists.fedoraprojec...
3]
As for other distributions, Debian [https://wiki.debian.org/RpathIssue
discourages] the use of RPATH, openSUSE
[https://en.opensuse.org/openSUSE:Packaging_checks#Beware_of_Rpath
forbids it] by running the check from rpmlint after every package
build and Arch and Gentoo point out to possible insecure usage at
their respective documentation pages.
Also there has been a relevant [https://bugs.python.org/issue36659
discussion] in the upstream python community.
== Benefit to Fedora ==
The main benefit of this change is avoiding bugs that might stem from
hardcoded RPATH values which include but not limited to: loading of
wrong or malicious code, wrong dependency resolution of object files
etc. There will also be a performance increase by derived from
avoiding unnecessary system calls required for handling RPATH's.
In addition the RPATH related packaging guidelines will be enforced by
the build system.
== Scope ==
* Proposal owners: Merge the
[https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/132
pull request] to redhat-rpm-config to enable running the check-rpaths
script after %install.
* Other developers: After merging the changes to redhat-rpm-config the
affected package maintainers that will see their packages' builds
fail, will need to review their usage of RPATH and either remove it or
workaround the issue. The packages currently failing to build due to
RPATH issues, so far, are listed in the wiki page
* Release engineering: This change doesn't require coordination with
rel-eng, as any issues will be caught during the regular mass rebuild
of packages.
* Policies and guidelines: TODO: The guidelines will be overhauled to
take into account accepted usage or RPATH, clarification of the policy
and ways to opt-out.
FPC ticket: https://pagure.io/packaging-committee/issue/886
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
There will be no visible impact to non-packagers. Packagers will need
to fix their packages if an broken RPATH entry was detected, as a
broken RPATH will make the rpmbuild fail either in koji or locally.
== How To Test ==
* Mock build testing:
Initiate a Fedora rawhide mock chroot and install the modified
redhat-rpm-config:
mock -r fedora-rawhide-x86_64 --install
https://download.copr.fedorainfracloud.org/results/cstratak/rpath/fedora-...
Build your package with the --no-clean option:
mock -r fedora-rawhide-x86_64 --no-clean <srpm>
* Local testing: Building rpm's locally should already reveal the
issue as the .rpmmacros file defines the RPATH check. Sample of a
vanilla .rpmmacros file:
%_topdir %(echo $HOME)/rpmbuild
%__arch_install_post \
[ "%{buildarch}" = "noarch" ] || QA_CHECK_RPATHS=1 ; \
case "${QA_CHECK_RPATHS:-}" in [1yY]*) /usr/lib/rpm/check-rpaths ;; esac \
/usr/lib/rpm/check-buildroot
* rpmlint test: The binary-or-shlib-defines-rpath test from the
BinariesCheck module will reveal RPATH usage.
rpmlint -c BinariesCheck <binary rpm>
e.g.: rpmlint -c BinariesCheck -v audiofile-0.3.6-27.fc34.x86_64.rpm
audiofile.x86_64: I: checking
audiofile.x86_64: E: binary-or-shlib-defines-rpath
/usr/bin/sfconvert ['/usr/lib64']
audiofile.x86_64: E: binary-or-shlib-defines-rpath
/usr/bin/sfinfo ['/usr/lib64']
1 packages and 0 specfiles checked; 2 errors, 0 warnings.
== User Experience ==
N/A (While this is a system wide change the impact is not visible to
regular users, only to packagers as described above).
== Dependencies ==
The change depends on modifying redhat-rpm-config.
== Contingency Plan ==
* Contingency mechanism: The change to redhat-rpm-config will be reverted
* Contingency deadline: Beta freeze
* Blocks release? No
== Documentation ==
TODO:
The documentation of the packaging guidelines will be updated to
reflect the changes that this change brings. Updated guidelines will
be https://docs.fedoraproject.org/en-US/packaging-guidelines/#_beware_of_rpath
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
2 years, 12 months
Offering strongswan for (co)maintaining
by Petr Menšík
Hello,
strongswan and NetworkManager-strongswan packages were passed to me from
previous maintainer. I admit I have little experience with them and do
not run any service based on them. Because IPSsec is quite complex
technology, I am looking for help with its maintenance. I was always
using OpenVPN based solutions myself, so I guess I am not the best
person as main admin. I would like to transfer main admin to anyone
doing a good job, not not immediately. That is why I haven't orphaned it
already.
I would like to keep commit access for a while, but I would share at
least commit access with anyone willing to improve those packages.
Especially someone using they (almost) everyday would be ideal maintainer.
Regards,
Petr
--
Petr Menšík
PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB
2 years, 12 months
aom soname bump
by Robert-André Mauchin
Hi,
AOM AV1 version 3 has been published a while ago and I am planning to
update it in Fedora. Due to a CVE affecting versions prior to 2021-04-07
(CVE-2021-30473), I plan to push this update to ALL STABLE RELEASES:
https://bugzilla.redhat.com/show_bug.cgi?id=1961376
I intend to wait until 3.1.1 which fix a few compilation bugs, and wait
for jpeg-xl to be packaged to benefit from the new butteraugli tuning.
Affected packages as follows:
avidemux rpmfusion-free
cinelerra-gg rpmfusion-free
ffmpeg rpmfusion-free
go-avif fedora (should be retired)
gstreamer1-plugins-bad-free fedora
libavif fedora
libheif rpmfusion-free
rust-aom-sys fedora
seamonkey fedora
vlc rpmfusion-free
xine-lib rpmfusion-free
I will rebuild the affected packages using my PP.
Best regards,
Robert-André
p.s.: Leigh, I might need your help not to fuck up the repo in rpmfusion
like last time (if you explain me your process to keep the branches in
sync it would be nice).
3 years
Intent to orphan maven-verifier-plugin
by Ben Cotton
I adopted maven-verifier-plugin because it was a transitive dependency
for condor back in December 2019. I did essentially nothing with it.
It is no longer a part of the dependency chain and is about to fall
victim to the Javapocalypse[1]. So I plan on orphan it at the end of
the week.
If anyone wants to pick it up (and thus adopt the dependencies
jackson-bom, jackson-parent, jackson-core, apache-ivy, spice-parent,
hawtjni, jackson-databind, jackson-annotations, aopalliance,
weld-parent, jakarta-server-pages, jansi-native, jakarta-interceptors,
snakeyaml), let me know before then and I'll just hand it over to you
instead.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
3 years
F35 Change: Make btrfs the default file system for Fedora Cloud
(System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/FedoraCloudBtrfsByDefault
== Summary ==
For cloud installs of Fedora, we want to provide advanced file system
features to users in a transparent fashion. Thus, we are changing the
file system for the Cloud Edition to Btrfs so we can leverage its
features and capabilities to improve the quality of experience for
Cloud users.
== Owners ==
* Name: [[User:Davdunc|David Duncan]], [[User:Chrismurphy|Chris
Murphy]], [[User:Josef|Josef Bacik]], [[User:Salimma|Michel Alexandre
Salim]], [[User:Dcavalca|Davide Cavalca]], [[User:Ngompa|Neal Gompa]],
[[User:Dustymabe|Dusty Mabe]], [[User:Malmond|Matthew Almond]]
* Email: davdunc(a)amazon.com, chrismurphy(a)fedoraproject.org,
josef(a)toxicpanda.com, michel(a)michel-slm.name, dcavalca(a)fb.com,
ngompa13(a)gmail.com, dusty(a)dustymabe.com, malmond(a)fb.com
* Products: Fedora Cloud Edition
* Responsible WGs: Fedora Cloud WG
== Detailed Description ==
Fedora Cloud Edition will switch to using Btrfs for its images. The
configuration for the Cloud Edition will match the setup used on the
desktop variants, as this has been very well-tested with production
deployments across multiple Fedora Linux releases now.
This includes the same subvolume layout that is used on the desktop
variants [[Changes/BtrfsByDefault|as introduced in Fedora Linux 33]],
as well as transparent Zstd compression
[[Changes/BtrfsTransparentCompression|as introduced in Fedora Linux
34]].
== Feedback ==
== Benefit to Fedora ==
The benefits are similar to
[[Changes/BtrfsByDefault#Benefit_to_Fedora|the ones for Fedora desktop
variants]]; however, there are specific benefits for Fedora Cloud:
* Adds support to Fedora Cloud for [[Changes/RPMCoW|the Change to
introduce support for Copy-on-Write enhancements to improve
performance to package management]]
* Adds the ability to logically separate contents of the volume
without dividing up the available space
** Transparent compression: significantly reduces write amplification
and improves effective I/O throughput
** Reflinks and snapshots improve efficiency for use cases like
containers (CRI-O, containerd, and Podman support both)
* Storage devices can be flaky, resulting in data corruption; Btrfs
can help mitigate this
** Everything is checksummed and verified on every read
** Corrupt data results in EIO (input/output error), instead of
resulting in application confusion, and isn’t replicated into backups
and archives
* Improves system responsiveness under pressure
** Btrfs has been tested in production to have proper IO isolation
capability via cgroups2
** Completes the resource control picture: memory, cpu, IO isolation
* File system resize
** Online shrink and grow are cornerstones of the Btrfs design
* Complex storage setups are… complicated
** Simple and comprehensive command interface. One master command
** Simpler to boot, all code is in the kernel, no initramfs complexities
** Simple and efficient file system replication, including incremental
backups, with <code>btrfs send</code> and <code>btrfs receive</code>
== Scope ==
* Proposal owners:
** Submit PRs for Cloud Edition kickstarts to produce disk images using Btrfs.
* Release engineering: [https://pagure.io/releng/issue/10129 #10129]
* Policies and guidelines: N/A
* Trademark approval: N/A
== Upgrade/compatibility impact ==
Change will not affect upgrades.
== How To Test ==
Once the change lands in Rawhide, spin up the images in AWS, GCP, and
KVM/OpenStack to test to see systems boot and run.
== User Experience ==
* Mostly transparent.
* Space savings and extend hardware life, via compression.
* Utilities for used and free space are expected to behave the same.
No special commands are required.
* More detailed information can be revealed by <code>btrfs</code>
specific commands.
* <code>cp</code> command will create reflink copies
[https://src.fedoraproject.org/rpms/coreutils/c/5d08d14b/ by default.]
== Dependencies ==
None.
== Contingency Plan ==
* Contingency mechanism: Owner will revert changes back to the
previous ext4 configuration
* Contingency deadline: Beta freeze
* Blocks release? Yes
* Blocks product? Cloud
== Documentation ==
Strictly speaking, no extra documentation is required reading for users.
== Release Notes ==
The default file system on the cloud is now Btrfs, following the
desktop change in Fedora Linux 33. Fedora Server, IoT, and CoreOS are
still specifically excluded.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
3 years
Fedora for WSL
by Greg Hellings
I may be hair-brained to do this, but I've put together an installer for
Fedora on WSL.
It mostly follows the procedures that had been much talked about in a blog
post about running Fedora 33 in the WSL2, but it uses the direct installer
rather than the manual side-loads. It also will install Fedora 34.
Obviously, it's not released into the Windows Store as that requires more
than just some technical bit wrangling. But if you're feeling adventurous,
and you are sometimes relegated to the world of Windows but want to bring
your Fedora along, you can find it here:
https://github.com/greg-hellings/FedoraWSL
Am I crazy? Yep.
Does it work? "Works in dev."
Anything to watch out for? It uses the trustywolf/wslu COPR repository for
installing WSL integration packages. Otherwise it's just raw Fedora bits.
Feel free to flame me if this was a terrible idea.
--Greg
3 years
Heads Up: openexr 3.0 coming to Rawhide
by Richard Shaw
I'm currently in the process of testing all deps in my COPR. Assuming no
major issues are found I'll setup a side tag to perform all the builds. I
expect to be done by this coming weekend.
Affected packages are:
alembic
aqsis
bcd
blender
calligra
CTL
darktable
Field3D
freeimage
gegl04
gimp
gmic
gstreamer1-plugins-bad-free
hugin
ImageMagick
kdebase3
kdelibs
kde-runtime
kf5-kimageformats
kio-extras
krita
luminance-hdr
luxcorerender
opencv
OpenImageIO
OpenSceneGraph
openshadinglanguage
openvdb
pfstools
povray
prusa-slicer
synfig
synfigstudio
vigra
vips
YafaRay
Thanks,
Richard
3 years
RPMLint 2.0 released!
by Neal Gompa
Hey all,
Nearly four years and *754 commits* since rpmlint 1.10, we are
releasing rpmlint 2.0.0!
This new release has a _lot_ of new features, but here are the most notable:
* RPMLint now is a "normal" Python application and now supports being
imported like a standard Python module! This means that all the normal
use-cases for RPMLint are still supported, but now you can make it a
part of larger Python-based applications or services.
* RPMLint uses a declarative TOML-based syntax for configuring RPMLint
policy instead of Python code.
* RPMLint now has an override system for the descriptions shown for
various checks, so that distributions who want to give specific policy
information can do so without patching the code.
* RPMLint includes _many more checks_! Nearly all of the generally
useful checks created by the openSUSE community have been merged into
the tree, so distributions can now benefit from a wider offering of
checks to implement policy enforcement.
* RPMLint is Python 3 only and now supports Python 3.6 and newer.
* RPMLint is now built and installed like a standard Python
application using setuptools.
I want to specifically thank Tomáš Chvátal, Martin Liska, Kristyna
Streitova, Dirk Mueller, Miroslav Suchý, Ondřej Súkup, thisisshub, and
Miro Hrončok as top contributors to make this release happen!
Full author list with number of commits:
309 Tomáš Chvátal
197 Martin Liska
47 Dirk Mueller
26 Kristyna Streitova
24 Neal Gompa (ニール・ゴンパ)
24 marxin
21 Neal Gompa
21 Ondřej Súkup
14 thisisshub
11 Miro Hrončok
9 Kristýna Streitová
8 Miroslav Suchý
6 Markéta Calábková
5 Ville Skyttä
4 Ben Greiner
4 Frank Schreiner
4 Van de Bugger
3 David Greaves
3 Matwey V. Kornilov
2 Daniel Mach
2 Matthias Gerstner
1 Cathy Hu
1 Ludwig Nussel
1 MeggyCal
1 Petr Menšík
1 Stefan Brüns
1 Steve Kowalik
1 Werner Fink
1 Wolfgang Stöggl
1 Yanko Kaneti
1 tpgxyz
--
真実はいつも一つ！/ Always, there's only one truth!
3 years
hamcrest update to 2.2 in Fedora rawhide
by Mikolaj Izdebski
Hello,
Next week I'm going to update package hamcrest in Fedora rawhide from
version 1.3 to version 2.2.
The proposed update contains an API change that can affect packages
depending on hamcrest. You may need to rebuild your packages to keep
working with updated hamcrest.
The update has already been checked into dist-git and a Koji build has
already been done, but Bodhi update has not been submitted yet. Bodhi
update is expected after a week, to comply with Updates Policy that
mandates a notification one week in advance before submitting an API
changing update.
Current NVR in rawhide: hamcrest-1.3-31.fc34
Updated NVR: hamcrest-2.2-3.fc35
Build link: https://koji.fedoraproject.org/koji/buildinfo?buildID=1748364
Packages depending on hamcrest that are possibly affected by this update:
* eclipse, maintained by lef jerboaa dbhole rgrunber jjohnstn
akurtakov ebaron oliver mbooth arobinso
* freemarker, maintained by filiperosset
* hamcrest, maintained by akurtakov mizdebsk jerboaa
* hdf, maintained by sagitter orion
* hdf5, maintained by deji sagitter orion ignatenkobrain
* icedtea-web, maintained by jvanek dbhole omajid
* jmock, maintained by orphan
* openas2, maintained by sdgathman
* py4j, maintained by raphgro
--
Mikolaj Izdebski
3 years
📦 Advice on packaging azure-cli
by Major Hayden
👋🏻 Hello there,
I'm eager to package Azure's CLI tools for Fedora that would allow users
to manage their Microsoft Azure cloud infrastructure. This would help
with CI/CD, information security, monitoring, and of course, deployments.
However, these cloud tools are a bit tricky to package. There are a few
python components in the main repository[0] that make up the CLI itself:
azure-cli
azure-cli-core
azure-cli-telemetry
Those seem fairly straightforward, but things get complicated because
those three depend on *plenty* of little SDK components from another
repository[1]. That repository contains over 220 SDK components that all
release independently from the same git repository. They also have
dependencies on some other bits of python that are already packaged for
Fedora, such as cffi, cryptography, and PyYAML.
To make things a bit more complicated, the core CLI tools have strict
dependencies on specific versions of the SDK components. For example:
'azure-mgmt-datamigration~=4.1.0',
'azure-mgmt-deploymentmanager~=0.2.0',
'azure-mgmt-devtestlabs~=4.0',
'azure-mgmt-dns~=8.0.0',
At first glance, that doesn't seem too bad since the versions of the
core CLI tools and SDK versions move in tandem in the repositories. SDK
components are constantly moving forward, but the CLI releases are tied
to specific SDK component versions which do not change after release.
The dependency tree is fairly long[2], but it's also fairly standard
between most of the SDKs.
I have several questions here:
1) Should I make separate Fedora packages/specs for each CLI
component and the SDK components? The SDK components look
nearly identical from a packaging standpoint (no executables
there, just libraries in each). If so, that would be about
80-100 packages to make and maintain.
2) Should I 'vendor' the SDK components into a single package and
set it as a dependency of the core CLI tools?
3) Should I dynamically generate spec files for the SDK components
based on the requirements from the main CLI tools?
4) Am I making this much, much more difficult than it really is? 🤦🏻♂️
Thanks in advance for any guidance you have. 🤗
[0] https://github.com/Azure/azure-cli
[1] https://github.com/Azure/azure-sdk-for-python
[2] https://gist.github.com/major/821e2f90c8a2b6111a42e35503caa3ad
--
Major Hayden
3 years
