Upstream tip wanted: CI service for Big Endian acrhes
by Miro Hrončok
Recently I've reported some Big Endian related test failures to an
upstream project [0].
I was asked by an upstream project maintainer, whether I know some free
Continuous Integration services where they can easily run their
testsuite on Big Endian.
Any tips?
* Upstream uses Travis CI to test on x86_64 Linux (Ubuntu)
* Upstream uses AppVeyor to test on Microsoft Windows
* It's a pure Python project, noarch, but some changes need to be done
when loading/saving binary data (LE) with NumPy on BE system.
What I've considered:
* COPR (but there is no big endian arch)
* (Ab)using Koji (I guess that would be considered a bad practice?)
* using QUEMU on Travis CI [1]
Any better tips? Thanks
[0] https://github.com/mikedh/trimesh/issues/249
[1]
https://developer.ibm.com/linuxonpower/2017/07/28/travis-multi-architectu...
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
3 years
Server Side Public License (SSPL) v1
by Tom Callaway
After review, Fedora has determined that the Server Side Public License
v1 (SSPL) is not a Free Software License.
It is the belief of Fedora that the SSPL is intentionally crafted to be
aggressively discriminatory towards a specific class of users.
Additionally, it seems clear that the intent of the license author is to
cause Fear, Uncertainty, and Doubt towards commercial users of software
under that license. To consider the SSPL to be "Free" or "Open Source"
causes that shadow to be cast across all other licenses in the FOSS
ecosystem, even though none of them carry that risk.
It is also worth nothing that while there is a draft for a "v2" of the SSPL:
A) It is not final.
B) It is not in use anywhere at this time (as far as we know).
C) The intent of the v2 draft text is not changed from the v1 license
currently in use.
We have updated our "Bad License" list to include SSPLv1. No software
under that license may be included in Fedora (including EPEL and COPRs).
Thanks,
Tom Callaway
Fedora Legal
3 years, 2 months
Re: NeuroFedora review swaps
by Ankur Sinha
On Mon, Jan 28, 2019 15:47:00 +0100, J. Scheurich wrote:
>
> > I'd like to get this package reviewed please:
> >
> > - python-pyscaffold: https://bugzilla.redhat.com/show_bug.cgi?id=1669913#
> >
> > Would anyone like to swap reviews?
>
> I would review it for wdune sponsoring.
Sorry---I'm not current with the wdune scenario. I assumed you meant
that you'd review it unofficially as part of the work to get sponsored
to the packagers group:
https://fedoraproject.org/wiki/How_to_get_sponsored_into_the_packager_gro...
I'm not a sponsor yet so I cannot sponsor you to the group myself, but
once you've done a few reviews, a sponsor will be happy to take a look
at them and guide you through the sponsorship process.
If you've submitted a review ticket for wdune already, I will be happy
to review it and provide comments.
--
Thanks,
Regards,
Ankur Sinha
https://ankursinha.in
Time zone: Europe/London
3 years, 3 months
Reproducible builds/bootstrap
by Pablo Greco
I'm starting to work on a project to make Fedora fully reproducible and bootstrappable from scratch.
I know it is a long term plan and still working on the steps, but it would be good to know the current status, if there is an internal interest in this, if someone is already working (or planning to).
Thanks for the info.
Pablo
3 years, 6 months
Fedora 33 System-Wide Change proposal: systemd-resolved
by Ben Cotton
https://fedoraproject.org/wiki/Changes/systemd-resolved
== Summary ==
Enable systemd-resolved by default. glibc will perform name resolution
using nss-resolve rather than nss-dns.
== Owner ==
* Name: [[User:catanzaro| Michael Catanzaro]]
* Email: <mcatanzaro(a)redhat.com>
== Detailed Description ==
We will enable systemd-resolved by default.
# We will change the
[https://src.fedoraproject.org/rpms/fedora-release/blob/f4cc5b6ce095bb4233...
fedora-release presets] to enable systemd-resolved instead of disable
it.
# systemd-libs currently has
[https://src.fedoraproject.org/rpms/systemd/blob/bb79fb73875f8e71841a1ee8e...
a %post scriplet] to enable nss-myhostname and nss-systemd by either
(a) modifying authselect's user-nsswitch.conf template, if authselect
is in use, or (b) directly modifying /etc/nsswitch.conf otherwise. We
will work with the systemd maintainers to enable nss-resolve here as
well.
# We will work with the authselect maintainers to update authselect's
minimal and nis profiles to enforce nss-resolve. These profiles modify
the hosts line of /etc/resolv.conf. (By default, Fedora uses
authselect's sssd profile, which does not modify the hosts line and
therefore does not have this problem.)
# We will remove our downstream patch to systemd that prevents systemd
from symlinking /etc/resolv.conf to
/run/systemd/resolve/stub-resolv.conf on new installs. For system
upgrades, a systemd-libs %post scriptlet will be used to reassign the
symlink during upgrade. Upon detecting this symlink, NetworkManager
will automatically enable its systemd-resolved support and configure
split DNS as appropriate.
systemd-resolved has been enabled by default in Ubuntu since Ubuntu
16.10, but please note we are doing this differently than Ubuntu has.
Ubuntu does not use nss-resolve. Instead, Ubuntu uses the traditional
nss-dns provided by glibc upstream, so glibc on Ubuntu continues to
read /etc/resolv.conf, as is traditional. This extra step is not
useful and not recommended by upstream. We want to follow upstream
recommendations in using nss-resolve instead.
If you do not wish to use systemd-resolved, then manual intervention
will be required to disable it:
* Modify /etc/authselect/user-nsswitch.conf and remove `resolve
[!UNAVAIL=return]` from the hosts line. Run `authselect
apply-changes`. (If you have disabled authselect, then edit
/etc/nsswitch.conf directly.)
* Disable and stop systemd-resolved.service.
* Restart the NetworkManager service. NetworkManager will then create
a traditional /etc/resolv.conf. (If you are not using NetworkManager,
you may create your own /etc/resolv.conf.)
== Benefit to Fedora ==
=== Standardization ===
Fedora will continue its history of enabling new systemd-provided
services whenever it makes sense to do so. Standardizing on upstream
systemd services is beneficial to the broader Linux ecosystem in
addition to Fedora, since standardizing reduces behavior differences
between different Linux distributions. Sadly, Fedora is no longer
leading in this area. Ubuntu has enabled systemd-resolved by default
since Ubuntu 16.10, so by the time Fedora 33 is released, we will be
three years behind Ubuntu here.
=== resolvectl ===
`resolvectl` has several useful functions (e.g. `resolvectl status` or
`resolvectl query`) that will be enabled out-of-the-box.
=== Caching ===
systemd-resolved caches DNS queries for a short while. This can
[https://gitlab.gnome.org/GNOME/glib/-/merge_requests/682#note_441846
dramatically] improve performance for applications that do not already
manually cache their own DNS results. (Generally, only web browsers
bother with manually caching DNS results.)
=== Split DNS ===
When systemd-resolved is enabled, users who use multiple VPNs at the
same time will notice that DNS requests are now sent to the correct
DNS server by default. Previously, this scenario would result in
embarrassing "DNS leaks" and, depending on the order that the VPN
connections were established, possible failure to resolve private
resources. These scenarios will now work properly. For example,
consider the scenario of Distrustful Denise, who (quite reasonably)
does not trust her ISP. Denise uses a public VPN service,
public-vpn.example.com, to hide her internet activity from her ISP,
but she also needs to use her employer's corporate VPN,
corporation.example.com, in order to access internal company resources
while working from home. Using the Network panel in System Settings,
Denise has configured her employer's VPN to "use this connection only
for resources on its network." Distrustful Denise expects that her
employer's VPN will receive all traffic for corporation.example.com,
and no other traffic. And while this mostly works in Fedora 32, she
discovers that it does not work properly for DNS:
* If Denise connects to public-vpn.example.com first and
corporation.example.com second, she is unable to access internal
company resources. All DNS requests are sent to
public-vpn.example.com's DNS server, so she is unable to resolve names
for internal company websites.
* If Denise connects to corporation.example.com first and
public-vpn.example.com second, then she is able to access internal
company resources. However, it only works because ''all'' her DNS
requests are sent to corporation.example.com's DNS server. Sadly for
Distrustful Denise, her employer discovers that she has been making
some embarrassing DNS requests that she had expected to go through
public-vpn.example.com instead.
Whichever VPN Denise connects to first receives all DNS requests
because glibc's nss-dns module is not smart enough to split the
requests. /etc/resolv.conf is just a list of DNS servers to try in
order, and NetworkManager has no plausible way to decide which to list
first, because both ways are broken, so it just prefers whichever was
connected first. And if one server fails to respond, then the next
server in the list will be tried, likely resulting in a DNS leak. In
contrast, when systemd-resolved is enabled, it will send each DNS
request only to the correct DNS server. The DNS server that will be
used for each tun interface can be inspected using the resolvectl
tool.
Migrating away from /etc/resolv.conf will also avoid an annoying
footgun with this legacy file: only the first three listed nameservers
are respected. All further nameservers are silently ignored.
NetworkManager adds a warning comment when writing more than three
nameservers to this file, but it cannot do any better than that.
=== DNS over TLS ===
systemd-resolved supports DNS over TLS (different from DNS over
HTTPS). Although this feature will not initially be enabled by
default, using systemd-resolved will enable us to turn on DNS over TLS
in a future Fedora release, providing improved security if the user's
DNS server supports DNS over TLS.
== Scope ==
* Proposal owners: We will update Fedora presets to enable
systemd-resolved by default.
* Other developers: This change requires coordination with the systemd
and authselect maintainers.
* Release engineering: [https://pagure.io/releng/issue/9367 #9367]
* Policies and guidelines: none required
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
systemd-resolved will be enabled automatically when upgrading to
Fedora 33. After upgrade, /etc/resolv.conf will be managed by systemd
and symlinked to /run/systemd/resolve/stub-resolv.conf. '''glibc will
no longer look at /etc/resolv.conf when performing name resolution.'''
Instead, glibc will communicate directly with systemd-resolved via
nss-resolve. systemd adds a large warning comment to the top of
/etc/resolv.conf to warn system administrators that changes to this
file will be ignored; however, scripts that edit this file manually
will break. Because this file is usually managed by NetworkManager,
impact to Fedora users will be limited to users who have manually
disabled NetworkManager; such users are expected to be experienced
system administrators who should be comfortable adapting to the change
(or disabling systemd-resolved).
Any applications that bypass glibc and read /etc/resolv.conf directly
will still work because /etc/resolv.conf will point to
systemd-resolved's stub resolver running on 127.0.0.53. Nevertheless,
/etc/resolv.conf is provided only for compatibility purposes, and
applications should prefer to use either glibc or the systemd-resolved
D-Bus API instead; see systemd-resolved(8) for details.
In short, '''applications that read /etc/resolv.conf will continue to
work as before.''' Applications that write to it will no longer work
as expected, but this only previously worked if NetworkManager is
disabled, a non-default configuration. It remains possible to disable
systemd-resolved if desired. Otherwise, any custom system
administration scripts that manage /etc/resolv.conf will need to be
updated.
=== DNSSEC ===
systemd-resolved's DNSSEC support is known to cause compatibility
problems with certain network access points. Per recommendation from
the systemd developers, we will change the default value of this
setting in Fedora from the upstream default `DNSSEC=allow-downgrade`
to `DNSSEC=no` by building systemd with the build option
`-Ddefault-dnssec=no`. The upstream default attempts to use DNSSEC if
it is working, but automatically disable it otherwise, allowing
man-in-the-middle attackers to disable DNSSEC. Sadly, even the
allow-downgrade setting suffers known compatibility problems. Because
Fedora is not prepared to handle an influx of DNSSEC-related bug
reports, we will disable this feature altogether. We anticipate that
enabling DNSSEC by default will not be possible in the foreseeable
future, or perhaps ever. Instead, enabling DNS over TLS (when
supported by the DNS server) seems likely in the near future.
=== Multicast DNS ===
systemd-resolved's multicast DNS support conflicts with Avahi. Per
recommendation from the systemd developers, we will change the default
value of this setting in Fedora from the upstream default
`MulticastDNS=yes` to `MulticastDNS=resolve`. Multicast DNS resolving
will be enabled, but responding will be disabled. This will require
adding a new systemd build option to control the default value of the
MulticastDNS setting, similar to the existing `default-dnssec` and
`default-dns-over-tls` build options.
== How To Test ==
Load any website in a web browser. If you succeed, then name
resolution probably works.
Try using `resolvectl status` and, for example, `resolvectl query
fedoraproject.org`, to see how they work and sanity-check their
output.
Users who use multiple VPNs at the same time are encouraged to test
DNS in a multiple VPN scenario, to ensure that DNS requests are sent
to the expected DNS server.
== User Experience ==
See the Benefit to Fedora section, above, for direct benefits to users
who use multiple VPNs at the same time.
Users will be able to use the resolvectl tool and the functionality it provides.
/etc/resolv.conf will now be managed by systemd rather than by
NetworkManager. As before, this file must not be modified directly
when it is managed.
== Dependencies ==
In Fedora, /etc/nsswitch.conf is managed by authselect. By default,
authselect uses the sssd profile to generate configuration compatible
with sssd. In this mode of operation, it does not modify the hosts
line in /etc/nsswitch.conf. This is also true if using the winbind
profile instead of the sssd profile. However, authselect's minimal and
nis profiles do modify the hosts line. These authselect profiles must
be updated to enable nss-resolved. If you are using authselect in one
of these modes, it will not be possible to cleanly disable
systemd-resolved because the hosts line in /etc/nsswitch.conf will be
clobbered whenever 'authselect apply-changes' is run. If you wish to
disable systemd-resolved and you are using authselect in one of these
modes, then you should stop using authselect. This is not expected to
cause many problems because virtually all Fedora users will be using
the default sssd profile.
We do not need to directly make any changes to the /etc/nsswitch.conf
shipped by glibc. Changes will be applied in the systemd-libs %post
scriptlet.
== Contingency Plan ==
The contingency plan, in the unlikely event of unexpected problems, is
simply to revert any changes and not enable systemd-resolved.
* Contingency deadline: beta freeze
* Blocks release? No
* Blocks product? No
== Documentation ==
* systemd-resolved is documented in several manpages: resolvectl(1),
resolved.conf(5), nss-resolve(8), systemd-resolved(8).
* [https://wiki.archlinux.org/index.php/Systemd-resolved Arch Wiki
documentation]
* [https://wiki.gnome.org/Projects/NetworkManager/DNS NetworkManager
DNS documentation]
== Release Notes ==
systemd-resolved is now enabled by default. systemd-resolved provides
a system-level DNS cache that can substantially improve performance
for applications that do not cache their own DNS results, allows
correct handling of split DNS scenarios such as when multiple VPNs are
in use, and will allow Fedora to enable DNS over TLS in the future.
/etc/resolv.conf will now be managed by systemd-resolved rather than
by NetworkManager. /etc/resolv.conf will no longer be read when
performing name resolution using glibc; however, it is still provided
for compatibility with applications that manually read this file to
perform name resolution. Writing to /etc/resolv.conf will no longer
work as expected.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 6 months
Fedora 32 system-wide change proposal: reduce installation media size
by improving the compression ratio of SquashFS filesystem
by Bohdan Khomutskyi
Summary
Improve compression ratio of SquashFS filesystem on the installation media.
Owner
Name: Bohdan Khomutskyi
Email: bkhomuts(a)redhat.com
Current status
Targeted release: I propose this change for Fedora 32
Last updated: Jan 5 2020
Pagure.io issue: https://pagure.io/releng/issue/9127
I was unable to create an article in Fedora wiki system.
Detailed Description
As of Fedora 31, the LiveOS/squashfs.img file on the installation image, is
compressed with default settings of mksquashfs. The standard configuration
is set to XZ algorithm with block size of 128k and BCJ filter enabled.
Those parameters can be adjusted which will lead to a better compression
ratio and/or reduction of the CPU usage at build time.
This is simple to achieve. Recently, Lorax has gotten support[1] for
adjusting the compression options for mksquashfs via the configuration
file. The file should be altered as following:
[compression]
bcj = yes
args = -b 1M -Xdict-size 1M -no-recovery
Where -b 1M and -Xdict-size 1M are block and dictionary sizes respectively.
Could be adjusted.
Benefit to Fedora
-
Reduction of the installation media size and the cost of storing and
distributing Fedora.
-
Reduction of the CPU usage at build time. Depending on which compression
parameters chosen.
-
See a graphical detail at https://pagure.io/releng/issue/9127.
Scope
-
Proposal owners:
The build environment should have support for adjusting the Lorax
configuration file.. Lorax is a program that produces the
LiveOS/squashfs.img file on the installation media.
One of the way to allow for such customization, is to add a feature in
Pungi, to allow for passing -c option to Lorax.
-
Release engineering: #9127 <https://pagure.io/releng/issue/9127>
-
Policies and guidelines: N/A
-
Trademark approval: N/A
Upgrade/compatibility impact
-
This change comes at a cost of higher memory usage during the
installation. Based on my personal estimations, this should not be the
issue. Since the decompression should require up to 1MiB per thread.
User Experience
-
Increasing the block size on the current configuration with EXT4 file
system, should increase latency while accessing the EXT4 filesystem. The
exact impact is to be evaluated.
-
The impact of latency will be reduced, if the plain SquashFS option is
be choosen.
Dependencies
-
N/A
Contingency Plan
-
N/A
Documentation
https://pagure.io/releng/issue/9127.
mksquashfs(1)
Release notes See also
https://pagure.io/releng/issue/8646
--
Bohdan Khomutskyi
Release Configuration Management engineer
Red Hat
3 years, 8 months
Fedora 32 System-Wide Change proposal (late): Enable EarlyOOM
by Ben Cotton
https://fedoraproject.org/wiki/Changes/EnableEarlyoom
== Summary ==
Install earlyoom package, and enable it by default. This will cause
the kernel oomkiller to trigger sooner, but will not affect which
process it chooses to kill off. The idea is to recover from out of
memory situations sooner, rather than the typical complete system hang
in which the user has no other choice but to force power off.
== Owner ==
* Name: [[User:chrismurphy| Chris Murphy]]
* Email: bugzilla(a)colorremedies.com
== Detailed Description ==
Workstation working group has discussed "better interactivity in
low-memory situations" for some months. In certain use cases,
typically compiling, if all RAM and swap are completely consumed,
system responsiveness becomes so abysmal that a reasonable user can
consider the system "lost", and resorts to forcing a power off. This
is objective a very bad UX. The broad discussion of this problem, and
some ideas for near term and long term solutions, is located here:
Recent long discussions on "Better interactivity in low-memory situations"<br>
https://pagure.io/fedora-workstation/issue/98<br>
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...<br>
Fedora editions and spins, have the in-kernel OOM (out-of-memory)
manager enabled. The manager's concern is keeping the kernel itself
functioning. It has no concern about user space function or
interactivity. This proposed change attempts to improve the user
experience, in the short term, by triggering the in-kernel process
killing mechanism, sooner. Instead of the system becoming completely
unresponsive for tens of minutes, hours or days, the expectation is an
offending process (determined by oom_score, same as now) will be
killed off within seconds or a few minutes. This is an incremental
improvement in user experience, but admittedly still suboptimal. There
is additional work on-going to improve the user experience further.
Workstation working group discussion specific to enabling earlyoom by default
https://pagure.io/fedora-workstation/issue/119
Other in-progress solutions:<br>
https://gitlab.freedesktop.org/hadess/low-memory-monitor<br>
Background information on this complicated problem:<br>
https://www.kernel.org/doc/gorman/html/understand/understand016.html<br>
https://lwn.net/Articles/317814/<br>
== Benefit to Fedora ==
There are two major benefits to Fedora:
* improved user experience by more quickly regaining control over
one's system, rather than having to force power off in low-memory
situations where there's aggressive swapping. Once a system becomes
unresponsive, it's completely reasonable for the user to assume the
system is lost, but that includes high potential for data loss.
* reducing forced poweroff as the main work around will increase data
collection, improving understanding of low memory situations and how
to handle them better
== Scope ==
* Proposal owners:
a. Modify {{code|https://pagure.io/fedora-comps/blob/master/f/comps-f32.xml.in}}
to include earlyoom package for Workstation.<br>
b. Modify {{code|https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/80-workstation.preset}}
to include:
<pre>
# enable earlyoom by default on workstation
enable earlyoom.service
</pre>
* Other developers:
Restricted to Workstation edition, unless other editions/spins want to opt-in.
* Release engineering: [https://pagure.io/releng/issues #9141] (a
check of an impact with Release Engineering is needed) <!-- REQUIRED
FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines: N/A
* Trademark approval: N/A
== Upgrade/compatibility impact ==
earlyoom.service will be enabled on upgrade. An upgraded system should
exhibit the same behaviors as a clean installed system.
== How To Test ==
* Fedora 30/31 users can test today, any edition or spin:<br>
{{code|sudo dnf install earlyoom}}<br>
{{code|sudo systemctl enable --now earlyoom}}
And then attempt to cause an out of memory situation. Examples:<br>
{{code|tail /dev/zero}}<br>
{{code|https://lkml.org/lkml/2019/8/4/15}}
* Fedora Workstation 32 (and Rawhide) users will see this service is
already enabled. It can be toggled with {{code|sudo systemctl
start/stop earlyoom}} where start means earlyoom is running, and stop
means earlyoom is not running.
== User Experience ==
The most egregious instances this change is trying to mitigate:
a. RAM is completely used
b. Swap is completely used
c. System becomes unresponsive to the user as swap thrashing has ensued
--> earlyoom disabled, the user often gives up and forces power off
(in my own testing this condition lasts >30 minutes with no kernel
triggered oom killer and no recovery)
--> earlyoom enabled, the system likely still becomes unresponsive but
oom killer is triggered in much less time (seconds or a few minutes,
in my testing, after less than 10% RAM and 10% swap is remaining)
earlyoom starts sending SIGTERM once both memory and swap are below
their respective PERCENT setting, default 10%. It sends SIGKILL once
both are below their respective KILL_PERCENT setting, default 5%.
The package includes configuration file /etc/default/earlyoom which
sets option {{code|-r 60}} causing a memory report to be entered into
the journal every minute.
== Dependencies ==
earlyoom package has no dependencies
== Contingency Plan ==
* Contingency mechanism: Owner will revert all changes
* Contingency deadline: Final freeze
* Blocks release? No
* Blocks product? No
== Documentation ==
{{code|man earlyoom}}<br><br>
https://www.kernel.org/doc/gorman/html/understand/understand016.html
== Release Notes ==
Earlyoom service is enabled by default, which will cause kernel
oom-killer to trigger sooner. To revert to previous behavior:<br>
{{code|sudo systemctl disable earlyoom.service}}
And to customize see {{code|man earlyoom}}.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
3 years, 8 months
