Fedora & Containers
by Michal Schorm
Hello,
Will anybody be able to explain to me the current state of the
containers & containerization in Fedora, please?
I have some questions, but the more I searched for whom & where to
ask, the more confused I became.
--
1) There ́s an IRC on freenode, '#fedora-containers' channel.
The TOPIC set contains the following message:
"The place for container runtimes and application containers.
Forums @ https://discussion.fedoraproject.org/c/containers,
visit the site @ https://containers.fedoraproject.org
(website operational at some point in August)."
However no link mentioned are accessible.
2) There is 'Fedora Container & Tools Documentation' [1]
It is only half-filled with information, some part missing entirely.
Even parts as 'Container Guidelines' [2].
Btw the wiki gladly redirects there ^ [3].
Btw there are some scratch notes about the guidelines from damn 2017
[4], which google offers me rather than the actual guidelines (because
they don't exist)
3) So ... who is in charge of it? Whom to contact? How? Where?
Does Fedora count with Containers or does it already thrown it overboard?
4) The container I am maintainer of (in pagure in 'container'
namespace) is not branched automatically. The last branch is 'f30',
but now I want to update it and add the missing branches. I have no
idea how should I do it though, since (1) (2) are such a mess.
I can try 'git push', but anyway, it won't solve the issue, it's not
branched automatically.
5) The Dockerfile (are we still using this name? shouldn't it be
'containerfile' now, with podman?)
starts with line like:
" FROM registry.fedoraproject.org/f30/....."
How can I substitute the Fedora release version with a variable, so I
can share the same content amongst branches for different Fedora
releases?
[1] https://docs.fedoraproject.org/en-US/containers/
[2] https://docs.fedoraproject.org/en-US/containers/guidelines/guidelines/
[3] https://fedoraproject.org/wiki/Container:Guidelines
[4] https://fedoraproject.org/wiki/Talk:Container:Guidelines
--
Michal Schorm
Software Engineer
Core Services - Databases Team
Red Hat
--
4 years, 1 month
Seeking co-maintainer for libffi package
by Anthony Green
Hello -- I'm the current Fedora maintainer for libffi, as well as the
upstream author/maintainer. I'm looking for help with libffi
packaging. Specifically, we need to roll out a new ABI-breaking
release (required for ARM64 and Intel CET support), and I don't have
the volunteer time available to create the requisite compat packages,
monitor all of the rebuilds, etc. It would be best if this person
had some Fedora packaging experience under their belt, given some of
the complexities involved and the criticality of libffi to the whole
platform.
Thanks!
AG
--
Anthony Green <green(a)redhat.com>
Senior Principal Solutions Architect, Financial Services
4 years, 1 month
Include non-RPM content in buildroot
by Martin Sehnoutka
Hi,
before I write the proposal itself I just want to stress the fact that
it isn’t my intention to change the current packaging workflow and
definitely not the user experience. Also if you have C or Python
packages it would not affect your work at all (I’m not familiar with all
interpreted languages in Fedora, but I guess it is similar to Python and
therefore it is not affected by the problems I am going to describe).
First of all, let me describe the problem I see in our Fedora ecosystem
with relation to Go and Rust language ecosystems. More specifically in
the relation between RPM buildroot and packages in these ecosystems.
Both of these languages follow the idea that packages should be small
and only have a limited set of features. Developers then use a lot of
these packages to write the final executable that is meant for end-users
[1]. Also both of these languages use static linking of the final
binaries meaning that Fedora users don’t install RPM packages of these
libraries as they are already baked inside of the binary [2].
The 1st problem is that if we want to build RPMs of the final
executables the way we do now, we need to package all these small
libraries into RPM even though they are just build dependencies and
users never install RPMs of these libraries. Many of these RPMs are
automatically generated from the upstream packages meaning that we don’t
do anything except for unpacking the upstream package (e.g. plain
tarball in case of crates.io) and then we package the same into RPM.
This process is unfortunately not fully automated and therefore requires
a certain amount of human effort.
To sum up the previous paragraph, I don’t think it is necessary to
repackage upstream tarball into a downstream RPM.
The 2nd problem is present only in the Rust ecosystem (as far as my
knowledge goes). Cargo, the official package manager for Rust, can
handle the scenario where an executable depends on a single library in
two different versions [3]. Dnf, on the other hand, will not install two
versions of the same RPM package. What we do now is, that we patch the
affected executables and libraries to only use the newest versions
everywhere. This is again an additional maintenance cost and we create
differences from upstream, because these patches are not necessarily
merged. See this as an example:
https://src.fedoraproject.org/rpms/rust-bstr/blob/master/f/rust-bstr.spec...
https://github.com/BurntSushi/bstr/pull/23
To sum up the 2nd problem: we are using dnf instead of the upstream
package manager to install dependencies. These two approaches can be
incompatible (and they are in case of Rust).
The 3rd problem I see is that issues like this are not going away, they
are only going to get worse as other ecosystems emerge. e.g. if the
Swift language became popular (for Linux development) it would again
have it’s own package manager and probably its own set of issues in
relation to our build system. (I mention Swift specifically because it
is a compiled language that have stable ABI only for std, other
libraries are statically linked)
Now, I’d like to point out that Go and Rust packaging works well in
Fedora due to the enormous effort of certain people and I very much
admire the work they do. But on the other hand I’m afraid where the
ecosystem would go without them. This is where we get to the situation
in the enterprise derivatives, specifically RHEL and CentOS. Their
solution to this problem is not to package all libraries separately but
to bundle all libraries directly into source RPMs of each executable. So
the bundling is not present only on the binary file level, but also on
the source RPM level. Go went even further in this case and it is common
to bundle all the dependencies as a source code directly in the upstream
repository. See this repo as an example:
https://github.com/containers/libpod/tree/master/vendor
It is fair to say, that my first motivation was the current state of
packaging in RHEL but I’d prefer to discuss this in Fedora first.
The proposal itself is fairly simple: Let’s stop packaging all Go and
Rust libraries into RPM and install them to the buildroot in the
upstream format instead.
The specific implementation is up for a discussion but I think it is
logical to start by asking what features do we want from the build
system? My answers are:
* We want to know what exactly was used to build the RPM to ensure
integrity. This is possible with upstream tarballs as well as with RPMs.
Just store a hash of the tarball.
* There should be no maintenance cost. If we would avoid modifying the
upstream package it would be the ideal case.
* It should be possible to patch the library in case of severe issues
like CVEs. This is a contradiction to the previous point, but it could
be solved by unpacking the upstream package into a git repository and
creating a new, modified package in upstream format from it.
* It should be possible to run the build locally in exactly the same
way Koji does it. This is just about exposing our “registry” of packages
to the public Internet.
Finally, the implementation could be something like this:
A service like release-monitoring could monitor the upstream projects
for new releases (e.g. NPM has a RSS feed). Every time there is a new
release it would automatically synchronize our own Fedora-specific
registry, which would in turn be accessible in buildroot. Then the RPM
macros like “cargo_build” could be modified to use this registry.
If we wanted to have the possibility to patch these libraries we could
synchronize the upstream release into our git repository and generate
packages in upstream format from it.
What do you think about the way we build RPM packages for Go and Rust?
Do you see other solutions or is it ok the way it is? (Please do not
suggest to “fix” the upstream communities)
Thanks for your opinions!
Martin
FAQ:
* Does it mean you want to get rid of -devel subpackages for C libraries?
No. First of all C language does not have a single package manager and
it is still common to install dependencies using the distro specific
package manager. But packaging into RPM also comes with the advantage of
having shared libraries so in this case it has a positive effect on the
end-user experience. Rust and Go libraries on the other hand are mostly
useless for end-users.
* I prefer to install development libraries using the distro package
manager instead of the upstream package manager. Why do you want to
change it?
This is the case for C development, but it is different in most of the
modern languages. For example we ship Python packages, but we discourage
people from using them for their own development. Python developers are
instead encouraged to use pip to create a virtual environment and use
upstream packages. In the case of Rust, using the system libraries would
be much harder than using the upstream package manager.
* Does it mean you want to get rid of Rust packages like ripgrep?
No. The end user experience stays unchanged. Packages that we ship to
Fedora users must be in RPM format, it is only the buildroot that changes.
Notes:
[1] Please don’t discuss if it is a good idea or not. This is a
discussion for upstream of those languages.
[2] Again, please don’t discuss static vs. dynamic linking here.
[3] https://stephencoakley.com/2019/04/24/how-rust-solved-dependency-hell
4 years, 1 month
Unannounced SONAME bump in cantor: libcantorlibs.so.23 → 24
by Fabio Valentini
Hi everybody,
The recent update from cantor 19.08 to cantor 12.12 in both fedora 32
and rawhide bumped the SONAME of a shared library as mention in
$SUBJECT (maintainers in CC).
At least LabPlot still needs to be rebuilt on both f32 and rawhide
(maintainers in CC).
Fabio
4 years, 1 month
Downgrading from rawhide
by Christophe de Dinechin
Is there any documented procedure to safely downgrade from rawhide to
the latest release?
I tried
# dnf update --releasever=32 fedora-release
# dnf distro-sync --allowerasing --skip-broken
Does something like that have any chance of working? At first sight, it
seems to be somewhat successful.
--
Cheers,
Christophe de Dinechin (IRC c3d)
4 years, 1 month
