F39 Change Proposal: Allow Removal of tzdata (System-Wide)
by Aoife Moloney
https://fedoraproject.org/wiki/Changes/AllowRemovalOfTzdata
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Allow the removal of tzdata especially on containers in order to minimize size.
== Owner ==
* Name: Patsy Griffin (Franklin)
* Email: patsy(a)redhat.com
== Detailed Description ==
This change will allow the removal of tzdata. When tzdata is removed,
the system will default to UTC. In order to reduce overhead, many
container installations now remove the data associated with tzdata but
cannot fully remove the package due to dependencies by other packages.
This results in confusion regarding the expected timezone info.
In order for this to work, we need packages that use tzdata at run
time to switch from Require'ing tzdata to Recommend'ing tzdata. These
packages should also gracefully handle instances where tzdata has been
removed. For example, this would require recognizing that tzdata is
not present and providing an appropriate error, such as recommending
that the user install tzdata.
== Feedback ==
In June of 2021, we proposed creating a new tzdata sub-package that
would only provide the UTC timezone. As part of the discussion around
this proposal, it was recommended that we completely remove tzdata. We
appreciated this input and welcome additional feedback.
== Benefit to Fedora ==
This change will allow tzdata to be removed from containers without
leaving inconsistent package remnants.
== Scope ==
* Proposal owners: No changes are needed to tzdata.
* Other developers: Some packages need to change their spec files from
`Requires: tzdata` to `Recommends: tzdata`. It would be beneficial if
all packages switched in this way, but it is not required. Supporting
optional tzdata installation for as many workloads as possible allows
those workloads to minimize their container image size.
List of packages which need to be changed:
* glibc (glibc-common)
* gcc (libstdc++)
* python3.XX (3.9, 3.10, 3.11, 3.12)
List of packages which would be beneficial to be changed:
* python3-dateutil
* python3-pytz
* libical
Upon acceptance of the change request we will file bugs to fix each of
these packages for Fedora 39.
* Release engineering: No changes needed.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives:
== Upgrade/compatibility impact ==
If tzdata is already installed, then it continues to be installed on the system.
Following the Fedora Weak Dependencies Policy dnf will treat the
Recommends on tzdata as if it were a Requires and tzdata will always
be installed in a default system
(https://docs.fedoraproject.org/en-US/packaging-guidelines/WeakDependencies/)
However, the recommends will allow tzdata to be correctly uninstalled
in a container build file rather than having to use ‘rm -rf’ to delete
the zone files to recover space.
== How To Test ==
Language runtimes were installed and A/B tests carried out with tzdata
present and tzdata removed. The intent of these tests was to ensure
that the system can use the language frameworks without tzdata present
and that when the data was required that meaningful errors were
presented to the user.
Packages tested were C (glibc), C++ (libstdc++), Python (Python 3.11),
using their time and date APIs. This testing led to the correction of
the libstdc++ implementation as noted here:
https://gcc.gnu.org/cgit/gcc/commit/?id=4abd5bc600193e821fbc41995a0b8d9ea...
Developers can test this by installing Rawhide and uninstalling tzdata
and verifying their package operates as expected.
If tzdata cannot be uninstalled then we recommend filling a bug
against the package that Requires: tzdata and having a discussion with
the maintainer to make tzdata optional e.g. Recommends: tzdata.
== User Experience ==
The user can remove tzdata to minimize the container size.
With tzdata removed the system is UTC only.
== Dependencies ==
Fixing glibc, gcc, and python3.xx at a minimum to make tzdata
Recommends instead of Requires.
== Contingency Plan ==
* Contingency mechanism: Back out the change. Don’t do it.
* Contingency deadline: Can be backed out at the last minute since we
are only dealing with conversions from Requires to Recommends.
* Blocks release? No
== Documentation ==
Document that the tzdata package may be removed if needed to reduce
space on containers. With tzdata removed, the system will default to
UTC.
== Release Notes ==
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
4 hours, 21 minutes
zlib-ng as a compat replacement for zlib
by Richard W.M. Jones
The background to this is I've spent far too long trying to optimize
the conversion of qcow2 files to raw files. Most existing qcow2 files
that you can find online are zlib compressed, including the qcow2
images provided by Fedora. Each cluster in the file is separately
compressed as a zlib stream, and qemu uses zlib library functions to
decompress them. When downloading and decompressing these files, I
measured 40%+ of the total CPU time is doing zlib decompression.
[You don't need to tell me how great Zstd is, qcow2 supports this for
compression also, but it is not widely used by existing content.]
I was looking around for alternative zlib implementations and there
are several. This AWS blog is a decent summary:
https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-com...
We already package zlib-ng in Fedora:
https://src.fedoraproject.org/rpms/zlib-ng/blob/rawhide/f/zlib-ng.spec
https://github.com/zlib-ng/zlib-ng
In my test zlib-ng is about 40% faster.
Sadly zlib-ng is not compiled with the ZLIB_COMPAT option. What this
means in practice is that the zlib functions have different names
(eg. zng_inflateInit instead of inflateInit). It is not a drop-in
replacement for zlib and software would need to be adjusted to use it.
However there is this bug / RFE to package the compat library.
https://bugzilla.redhat.com/show_bug.cgi?id=2145239
It literally replaces /usr/lib64/libz.so.1, so pretty high stakes. It
has the appropriate Provides/Conflicts.
Anyway, 40% (or whatever, but significant) performance improvement
sounds good for a very widely used operation.
So I'd like to ask Fedora ...
What do we think about the opt in approach of adding a patch similar
to the one proposed in bug 2145239, where I think you could "simply"
install zlib-ng to get better performance with existing software?
("simply" because it seems high risk of going wrong)
What about replacing zlib with zlib-ng?
Other ideas ...? Does anyone know what other distros are doing?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
2 days, 20 hours
Potential changes to systemd RPM macros
by Andrea Bolognani
Hi,
this requires a bit of background, so please bear with me :)
Earlier this year, the libvirt project changed its RPM package to be
more granular[1] as a step towards a future where the legacy
monolithic daemon (libvirtd) is completely gone. This change has
happened upstream, but the Fedora package uses the same spec file so
it was quickly reflected in Rawhide.
As part of this change virtproxyd, a service which has historically
been part of the libvirt-daemon package, has been moved to the
newly-introduced libvirt-daemon-proxy package.
More recently we have realized that, despite our care in trying to
ensure smooth upgrades, splitting the package this way has
unfortunately caused unintended consequences[2][3]. Specifically,
since the libvirt-daemon-proxy package is a completely new one as far
as RPM is concerned, all its systemd units will get presets applied
during %post. In at least two completely valid deployment scenarios,
this results in some amount of brokenness due to local configuration
changes made by the admin getting discarded.
In order to avoid this problem, I have implemented a new set of RPM
macros[4][5] that base the decision of whether presets should be
applied for a systemd unit not on whether the package that contains
them is being newly installed, but rather on whether the unit itself
existed on the system before package installation. This should
address not only the scenario of services being split off to separate
packages, but also the one where an existing package starts shipping
additional services.
Now, since this is clearly not a libvirt-specific issue, I believe
this approach should be adopted across Fedora by way of these macros
(or comparable ones) being added to systemd. Packages would have to
migrate from the old macros over time, and at some point it should be
possible to get rid of them entirely.
Note that openSUSE already has its own distro-specific RPM macros for
dealing with systemd units, and in fact the approach that I have
implemented is partially inspired by theirs. Ideally, we'd be able to
collaborate with them to create a single set of macros that lives in
systemd and satisfies the needs of both Fedora and openSUSE.
The problem is that Fedora 39 and RHEL 9.3 are fast approaching and,
if we don't do anything about this issue before then, a subset of
libvirt users will see their deployments broken after upgrade. In the
interest of avoiding that, I would prefer to get the libvirt-specific
version of the macros merged as soon as possible, and focus on
upstreaming the work all the way to systemd as a follow-up.
Does this plan sound reasonable to the Fedora community? Are there
any serious concerns regarding the approach taken for the macros that
would cause them to be considered a complete no-go? Any scenarios
that I might have missed while implementing them?
Thanks for your patience in reading this far :) and looking forward
to your feedback!
[1] https://listman.redhat.com/archives/libvir-list/2023-January/237059.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=2210058
[3] https://listman.redhat.com/archives/libvir-list/2023-June/240226.html
[4] https://listman.redhat.com/archives/libvir-list/2023-July/240723.html
[5] https://listman.redhat.com/archives/libvir-list/2023-July/240729.html
--
Andrea Bolognani / Red Hat / Virtualization
1 week, 2 days
Re: TSS maintainer volunteer
by Petr Pisar
V Thu, Feb 16, 2023 at 07:29:07PM +0000, Kenneth Goldman napsal(a):
> I think I followed all those steps - identifying the package, announcing that
> I want to be the packager, making an account, etc.
>
> What's next?
Submit an updated tss2 package for a package review. As far as I can see,
there is no review opened for tss2 now
<https://bugzilla.redhat.com/buglist.cgi?component=Package%20Review&produc...>.
How to do it is described at
<https://docs.fedoraproject.org/en-US/package-maintainers/Package_Review_P...>.
Especially pay attention to:
If you are not member of the packager group, you need a sponsor. Add
FE-NEEDSPONSOR to the bugs being blocked by your review request.
> Does someone approve me?
Based on the FE-NEEDSPONSOR blocker someone from sponsors should notice your
review request and start to communicate with you in the review request in
Bugzilla. (If that does not happen, approach you a sponsor of your choice as
recommended at
<https://docs.fedoraproject.org/en-US/package-maintainers/How_to_Get_Spons...>)
Once the sponsor finds your package looks good and you understand how to
maintain a package, he/she will sponsor you, i.e. adds you into a packagers
group. Then you will be able to continue from this item on the
Package_Review_Process document:
When your package passes the review you should use fedpkg to request a Git
repository for it.
> Move a git repo somewhere?
For the purpose of the package review, you need to publish the spec file and
the SRPM file somewhere on the Internet. (Once you become a packager, you can
also use <https://fedorapeople.org/> server for that purpose.)
Once the package review passes, the offical git repository (called dist-git in
Fedora) for the tss2 package will be reopened with completing this item:
Request a Git repository for the package
Then you will commit the new spec file into the reopen dist-git repository.
-- Petr
1 week, 6 days
Bootstrapping package with circular dependencies in koji
by Jaroslav Skarvada
Hi,
I need to bootstrap package which has bootstrap support written
according to the [1]. I am able to bootstrap it locally (rpmbuild,
mock, ...) with the "--with bootstrap" or "-D '_with_bootstrap 1'". Is
there support for it in koji? E.g. something like:
koji build SIDE-TAG PACKAGE --bootstrap? Or do I have to manually do:
1. patch:
- %bcond_with bootstrap
+ %bcond_without bootstrap
2. koji build SIDE-TAG SCM
3. update&build the circular dep
4. unpatch:
- %bcond_without bootstrap
+ %bcond_with bootstrap
5. release bump
6. koji build SIDE-TAG SCM
Or is there some better way?
thanks & regards
Jaroslav
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping
2 weeks
Donate 1 minute of your time to test upgrades from F38 to F39
by Miroslav Suchý
Do you want to make Fedora 39 better? Please spend 1 minute of your time and try to run:
# Run this only if you use default Fedora modules
# next time you run any DNF command default modules will be enabled again
sudo dnf module reset '*'
dnf --releasever=39 --setopt=module_platform_id=platform:f39 \
--enablerepo=updates-testing \
$(rpm -q fedora-repos-modular >/dev/null && echo --enablerepo=updates-testing-modular) \
--assumeno distro-sync
This command does not replace `dnf system-upgrade`, but it will reveal potential problems.
You may also run `dnf upgrade` before running this command.
The `--assumeno` will just test the transaction, but does not make the actual upgrade.
In case you hit dependency issues, please report it against the appropriate package.
Or against fedora-obsolete-packages if that package should be removed in Fedora 39. Please check existing reports
against fedora-obsolete-packages first:
https://red.ht/2kuBDPu
and also there is already bunch of "Fails to install" (F39FailsToInstall) reports:
https://bugzilla.redhat.com/buglist.cgi?bug_id=2168845&bug_id_type=anddep...
Two notes:
* you may want to run the same command with dnf5 to help test new dnf.
* this command found zero issues on my personal system - great work all everybody!
Thank you
Miroslav
2 weeks
Adding Passim as a Fedora 40 feature?
by Richard Hughes
Hi all,
I was thinking of adding Passim as a default-installed and
default-enabled dep of fwupd in the Fedora 40 release. Before I create
lots of unnecessary drama, is there any early feedback on what's
described in https://github.com/hughsie/passim/blob/main/README.md
please.
The tl;dr: is I want to add a mDNS server that reshares the public
firmware update metadata from the LVFS on your LAN. The idea is that
rather than 25 users in an office downloading the same ~2MB file from
the CDN every day, the first downloads from the CDN and the other 24
download from the first machine. All machines still download the
[tiny] jcat file from the CDN still so we know the SHA256 to search
for and verify.
The backstory is that as the fwupd grows and grows (to ChromeOS,
FreeBSD, Windows and macOS) we need to scale things up a couple of
orders of magnitude. This isn't specific to firmware stuff, although I
think it makes a great testcase which we could add dnf or ostree
content to in the future. Comments and questions are most welcome.
Thanks,
Richard.
2 weeks, 1 day
Re: [PATCH] fedora: Replace pasta hard links by copies, mangle
Build-IDs
by Richard W.M. Jones
On Wed, Aug 23, 2023 at 03:59:39PM +0200, Stefano Brivio wrote:
> On Wed, 23 Aug 2023 14:53:27 +0100
> "Richard W.M. Jones" <rjones(a)redhat.com> wrote:
>
> > On Wed, Aug 23, 2023 at 03:48:44PM +0200, Stefano Brivio wrote:
> > > The hard link trick didn't actually fix the issue with SELinux file
> > > contexts properly: as opposed to symbolic links, SELinux now
> > > correctly associates types to the labels that are set -- except that
> > > those labels are now shared, so we can end up (depending on how
> > > rpm(8) extracts the archives) with /usr/bin/passt having a
> > > pasta_exec_t context.
> > >
> > > This got rather confusing as running restorecon(8) seemed to fix up
> > > labels -- but that's simply toggling between passt_exec_t and
> > > pasta_exec_t for both links, because each invocation will just "fix"
> > > the file with the mismatching context.
> > >
> > > Replace the hard links with copies. AppArmor's attachment, instead,
> > > works with hard links, and if there's no LSM, we can keep symbolic
> > > links, so keep symbolic links in the Makefile.
> > >
> > > With copies, rpmbuild(8) will warn about duplicate Build-IDs in the
> > > same package. Mangle them in pasta binaries by summing one to the
> > > last byte, modulo one byte, using xxd (provided by vim-common) and
> > > disable the automatic rehashing by find-debuginfo(1) -- we already
> > > have per-release Build-IDs thanks to $VERSION passed on 'make'.
> >
> > Right, this ^ was going to be my comment. RPM doesn't like having two
> > identical copies of a file.
>
> In which other way, though? cpio(1) is fine with it, and I tried to
> install the package on both ext4 and xfs -- the only warning I got was
> the (semi-reasonable) one from rpmbuild about duplicate Build-IDs.
I'm fairly sure I've seen an error when you have two identical files;
it might only happen in Koji. Anyway, hacking the binary is surely
wrong, but let's hear the opinions of the Fedora / SELinux developers.
Rich.
> > Hacking the binary to "fix" the problem doesn't sound like a solution.
> >
> > I'm CC-ing Fedora-devel-list to find out we can properly fix this issue.
> >
> > Rich.
> >
> > > Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
> > > ---
> > > contrib/fedora/passt.spec | 27 ++++++++++++++++++++++-----
> > > 1 file changed, 22 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/contrib/fedora/passt.spec b/contrib/fedora/passt.spec
> > > index d0c6895..51bf5a8 100644
> > > --- a/contrib/fedora/passt.spec
> > > +++ b/contrib/fedora/passt.spec
> > > @@ -9,6 +9,10 @@
> > >
> > > %global git_hash {{{ git_head }}}
> > > %global selinuxtype targeted
> > > +# Different Build-IDs for passt and pasta: don't let find-debuginfo touch them
> > > +%undefine _unique_build_ids
> > > +%global _no_recompute_build_ids 1
> > > +
> > >
> > > Name: passt
> > > Version: {{{ git_version }}}
> > > @@ -19,7 +23,7 @@ Group: System Environment/Daemons
> > > URL: https://passt.top/
> > > Source: https://passt.top/passt/snapshot/passt-%{git_hash}.tar.xz
> > >
> > > -BuildRequires: gcc, make, checkpolicy, selinux-policy-devel
> > > +BuildRequires: gcc, make, checkpolicy, selinux-policy-devel, binutils, vim-common
> > > Requires: (%{name}-selinux = %{version}-%{release} if selinux-policy-%{selinuxtype})
> > >
> > > %description
> > > @@ -56,15 +60,28 @@ This package adds SELinux enforcement to passt(1) and pasta(1).
> > > %install
> > >
> > > %make_install DESTDIR=%{buildroot} prefix=%{_prefix} bindir=%{_bindir} mandir=%{_mandir} docdir=%{_docdir}/%{name}
> > > -# The Makefile creates symbolic links for pasta, but we need hard links for
> > > +# The Makefile creates symbolic links for pasta, but we need actual copies for
> > > # SELinux file contexts to work as intended. Same with pasta.avx2 if present.
> > > -ln -f %{buildroot}%{_bindir}/passt %{buildroot}%{_bindir}/pasta
> > > +#
> > > +# To avoid duplicate Build-IDs in the same package, we increase the last byte of
> > > +# the value for pasta binaries by one (modulo one byte). Note that we already
> > > +# have differentiated Build-IDs per release, courtesy of $VERSION, so we don't
> > > +# need find-debuginfo(1) to recalculate them.
> > > +rm %{buildroot}%{_bindir}/pasta
> > > +objcopy --dump-section .note.gnu.build-id=%{buildroot}/build_id %{buildroot}%{_bindir}/passt
> > > +printf '\x'$(printf %02x $(( ( 0x$(xxd -ps -s 35 %{buildroot}/build_id) + 1 ) % 0xff )) ) | dd of=%{buildroot}/build_id seek=35 bs=1 count=1 conv=notrunc
> > > +objcopy --update-section .note.gnu.build-id=%{buildroot}/build_id %{buildroot}%{_bindir}/passt %{buildroot}%{_bindir}/pasta
> > > +rm %{buildroot}/build_id
> > > +
> > > %ifarch x86_64
> > > -ln -f %{buildroot}%{_bindir}/passt.avx2 %{buildroot}%{_bindir}/pasta.avx2
> > > +rm %{buildroot}%{_bindir}/pasta.avx2
> > > +objcopy --dump-section .note.gnu.build-id=%{buildroot}/build_id %{buildroot}%{_bindir}/passt.avx2
> > > +printf '\x'$(printf %02x $(( ( 0x$(xxd -ps -s 35 %{buildroot}/build_id) + 1 ) % 0xff )) ) | dd of=%{buildroot}/build_id seek=35 bs=1 count=1 conv=notrunc
> > > +objcopy --update-section .note.gnu.build-id=%{buildroot}/build_id %{buildroot}%{_bindir}/passt.avx2 %{buildroot}%{_bindir}/pasta.avx2
> > > +rm %{buildroot}/build_id
> > >
> > > ln -sr %{buildroot}%{_mandir}/man1/passt.1 %{buildroot}%{_mandir}/man1/passt.avx2.1
> > > ln -sr %{buildroot}%{_mandir}/man1/pasta.1 %{buildroot}%{_mandir}/man1/pasta.avx2.1
> > > -install -p -m 755 %{buildroot}%{_bindir}/passt.avx2 %{buildroot}%{_bindir}/pasta.avx2
> > > %endif
> > >
> > > pushd contrib/selinux
> > > --
> > > 2.39.2
>
> --
> Stefano
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
2 weeks, 1 day
