Ars claims: Fedora 32 is sluggish

[Michael Catanzaro]

Hi,

Has anybody investigated Jim Salter's claims that Fedora 32 is slow to 
launch applications? Recent article:

https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...

"in my experience, Fedora 32 is noticeably, demonstrably more sluggish 
to launch applications than Ubuntu is in general."

Original article:

https://arstechnica.com/gadgets/2020/05/linux-distro-review-fedora-workstati...

Would be good to know, for starters, whether this difference is real 
and measurable.

Michael

[Nathanael D. Noblet]

On Wed, 2021-02-03 at 10:53 -0600, Michael Catanzaro wrote:
> Hi,
>
> Has anybody investigated Jim Salter's claims that Fedora 32 is slow
> to 
> launch applications? Recent article:
>
> https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
>
> "in my experience, Fedora 32 is noticeably, demonstrably more
> sluggish 
> to launch applications than Ubuntu is in general."
>
> Original article:
>
> https://arstechnica.com/gadgets/2020/05/linux-distro-review-fedora-workstati...
>
> Would be good to know, for starters, whether this difference is real 
> and measurable.

I haven't used Ubuntu in a long time. However I do know that I remember
creating Ubuntu VM a few years ago and was blown away at how fast it
went from grub to a desktop login compared to Fedora. Other than that
Fedora seems fast enough to me. Just a single datapoint...

[Fabio Valentini]

On Wed, Feb 3, 2021 at 5:54 PM Michael Catanzaro  wrote:
> Hi,
>
> Has anybody investigated Jim Salter's claims that Fedora 32 is slow to
> launch applications? Recent article:
>
> https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
>
> "in my experience, Fedora 32 is noticeably, demonstrably more sluggish
> to launch applications than Ubuntu is in general."
>
> Original article:
>
> https://arstechnica.com/gadgets/2020/05/linux-distro-review-fedora-workstati...
>
> Would be good to know, for starters, whether this difference is real
> and measurable.

I don't know if this helps as a data point, but until 32, Fedora
always "won" one of the bottom spots in Phronix distro comiparison
benchmarks.

However, starting with Fedora 33, at least on some hardware, we now
outperform Ubuntu 20.20, Manjaro, and even Clear Linux - at least on
average:
https://www.phoronix.com/scan.php?page=article&item=fedora-wins-tgl

Maybe I can provide some anecdata: In my experience, Ubuntu based VMs
always felt way more responsive and faster than Fedora based VMs (I
use them regularly to do QA for my Pantheon packages, since Fedora 2x
or so.) But today, cold booting a Fedora 33 or 34 VM is more than
twice as fast (~25s) compared to an Ubuntu based (elementary OS 5 /
ubuntu 18.04.x LTS) VM (~68s), and when launching Firefox after a cold
boot, Fedora 33 was almost exacty twice as fast (~25s vs. ~50s),
again. For both of those "tests", Fedora 32 was a bit slower than
either newer Fedora or elementary OS.

Even if those are not "scientific" measurements, I did not expect the
differences to be this big, but apparently, we're doing something
right. :)
Maybe switching on LTO by default did help - it was introduced with Fedora 33.

The "Fedora is slow and sluggish" review you linked specifically
mentions Fedora 32.
Apparently the author has not tried Fedora since (there's no review of
Fedora 33 on Ars Technica).
And I would be curious to see whether upgrading to Fedora 33 would
affect the result of the "launch Chromium snap" test in our favor ;)

Fabio

[Artem Tim]

I did my own various tests on Ryzen (Zen 2) 4 month ago or so on Fedora 32/33 vs Clear Linux which known to be "fastest" distro and in almost all benchmarks Fedora was equal or even faster a little bit then Clear Linux. In rare cases Clear Linux was faster within the margin of error. Have no idea what they mean by "sluggish".

[Stephen John Smoogen]

On Wed, 3 Feb 2021 at 14:13, Artem Tim  wrote:
> I did my own various tests on Ryzen (Zen 2) 4 month ago or so on Fedora
> 32/33 vs Clear Linux which known to be "fastest" distro and in almost all
> benchmarks Fedora was equal or even faster a little bit then Clear Linux.
> In rare cases Clear Linux was faster within the margin of error. Have no
> idea what they mean by "sluggish".

Sluggish can mean all kinds of things to different people so it is not a
useful metric term. Network latency, disk drive latency, and startup times
are the usual culprits. However other items like background colour and
font look and feel can give people the feeling something is 'sluggish' or
'slower' versus another system because a mouse will 'feel' like its slower
than in another due to human brain wiring than actual facts. As such, I
would just take it as a figure of speech which would need careful testing
to see what was going on before seeing what might be the cause.
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Matthew Miller]

On Wed, Feb 03, 2021 at 10:53:32AM -0600, Michael Catanzaro wrote:
> Has anybody investigated Jim Salter's claims that Fedora 32 is slow
> to launch applications? Recent article:
>
> https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
>
> "in my experience, Fedora 32 is noticeably, demonstrably more
> sluggish to launch applications than Ubuntu is in general."

I genuinely wonder if this is due to the launch animation. I know that
subjectively for myself using the Impatience to triple the speed makes my
desktop feel more snappy.

[Nathanael D. Noblet]

On Wed, 2021-02-03 at 14:48 -0500, Matthew Miller wrote:
> I genuinely wonder if this is due to the launch animation. I know
> that
> subjectively for myself using the Impatience to triple the speed
> makes my
> desktop feel more snappy.

Sorry, what is 'the Impatience'? How does it improve the desktop
performance?

[Matthew Miller]

On Thu, Feb 04, 2021 at 10:33:37AM -0700, Nathanael D. Noblet wrote:
> > I genuinely wonder if this is due to the launch animation. I know that
> > subjectively for myself using the Impatience to triple the speed makes
> > my desktop feel more snappy.
>
> Sorry, what is 'the Impatience'? How does it improve the desktop
> performance?

It is this extension: https://github.com/timbertson/gnome-shell-impatience

It does not _actually_ improve performance. It gives you a slider which can
be used to double or triple (or whatever) the speed of the animations, which
has — to my easily-tricked brain — the effect of making the desktop _seem_
more responsive. Try it and see if it works for you too!

[clime]

On Thu, 4 Feb 2021 at 19:36, Matthew Miller 
wrote:
> On Thu, Feb 04, 2021 at 10:33:37AM -0700, Nathanael D. Noblet wrote:
> > > I genuinely wonder if this is due to the launch animation. I know that
> > > subjectively for myself using the Impatience to triple the speed makes
> > > my desktop feel more snappy.
> >
> > Sorry, what is 'the Impatience'? How does it improve the desktop
> > performance?
>
> It is this extension: https://github.com/timbertson/gnome-shell-impatience
>
> It does not _actually_ improve performance. It gives you a slider which can
> be used to double or triple (or whatever) the speed of the animations,
> which
> has — to my easily-tricked brain — the effect of making the desktop _seem_
> more responsive. Try it and see if it works for you too!

Thanks, it helps even for going to overlay and back and for switching
workspaces.
> --
> Matthew Miller
>
> Fedora Project Leader
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[clime]

On Thu, 4 Feb 2021 at 21:12, clime  wrote:
> On Thu, 4 Feb 2021 at 19:36, Matthew Miller 
> wrote:
> > On Thu, Feb 04, 2021 at 10:33:37AM -0700, Nathanael D. Noblet wrote:
> > > > I genuinely wonder if this is due to the launch animation. I know that
> > > > subjectively for myself using the Impatience to triple the speed makes
> > > > my desktop feel more snappy.
> > >
> > > Sorry, what is 'the Impatience'? How does it improve the desktop
> > > performance?
> >
> > It is this extension:
> > https://github.com/timbertson/gnome-shell-impatience
> >
> > It does not _actually_ improve performance. It gives you a slider which
> > can
> > be used to double or triple (or whatever) the speed of the animations,
> > which
> > has — to my easily-tricked brain — the effect of making the desktop _seem_
> > more responsive. Try it and see if it works for you too!
>
> Thanks, it helps even for going to overlay and back and for switching
> workspaces.

I now have more time to procrastinate, which is great!
> > --
> > Matthew Miller
> >
> > Fedora Project Leader
> > _______________________________________________
> > devel mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/[e-mail]

[Nathanael D. Noblet]

On Thu, 2021-02-04 at 13:25 -0500, Matthew Miller wrote:
> On Thu, Feb 04, 2021 at 10:33:37AM -0700, Nathanael D. Noblet wrote:
> > > I genuinely wonder if this is due to the launch animation. I know
> > > that
> > > subjectively for myself using the Impatience to triple the speed
> > > makes
> > > my desktop feel more snappy.
> >
> > Sorry, what is 'the Impatience'? How does it improve the desktop
> > performance?
>
> It is this extension: 
> https://github.com/timbertson/gnome-shell-impatience
>
> It does not _actually_ improve performance. It gives you a slider
> which can
> be used to double or triple (or whatever) the speed of the
> animations, which
> has — to my easily-tricked brain — the effect of making the desktop
> _seem_
> more responsive. Try it and see if it works for you too!

Oh my goodness that's hilarious and all kinds of awesome. The name of
the extension is great.

[Akashdeep Dhar]

Hey Matthew,

Could you please try switching to a much stronger CPU governor like `performance` and see if it helps?

You could use this tool https://github.com/t0xic0der/switcheroo and run `sudo ./switcheroo -setn performance` to do so.

Regards,
Akashdeep Dhar

[Alexander Ploumistos]

On Wed, Feb 3, 2021 at 8:48 PM Matthew Miller  wrote:
> On Wed, Feb 03, 2021 at 10:53:32AM -0600, Michael Catanzaro wrote:
> > Has anybody investigated Jim Salter's claims that Fedora 32 is slow
> > to launch applications? Recent article:
> >
> > https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
> >
> > "in my experience, Fedora 32 is noticeably, demonstrably more
> > sluggish to launch applications than Ubuntu is in general."
>
> I genuinely wonder if this is due to the launch animation. I know that
> subjectively for myself using the Impatience to triple the speed makes my
> desktop feel more snappy.

I do not particularly enjoy self-flagellation, but I think that
besides some conscious system-wide choices that could measurably
affect performance for everyone (e.g. compiler flags), we suffer from
a lack of interest towards bugs that happen behind the scenes and
impact a subset of our users. Even in cases when upstream is aware of
an issue and a fix is promptly made available, in some cases it never
finds its way to a supported Fedora version or that happens with a
considerable lag. In the last 4 or so years I remember issues with
tracker, gnome-shell, mutter/clutter and friends on specific GPUs,
default or popular shell extensions and dbus services. A recent bug
which is also related to the topic of Jim Salter's article is
#1916652. SELinux and Flatpak are supposed to be first-class citizens
in Fedora, yet we get bugs like that. There has been a flatpak update
in the interim, but it didn't address the problem. And while on a
16-core system this barely registers (unless someone has SELinux
Troubleshooter installed), I reproduced it on a dual-core Celeron and
it took almost five minutes to get the system to a usable state.

[Roberto Ragusa]

On 2/4/21 9:52 PM, Alexander Ploumistos wrote:
> considerable lag. In the last 4 or so years I remember issues with
> tracker, gnome-shell, mutter/clutter and friends on specific GPUs,
> default or popular shell extensions and dbus services. A recent bug

If tracker is enabled the performance drop after booting is huge.

rpm -e tracker-miners

[Alexander Ploumistos]

On Fri, Feb 5, 2021 at 10:03 AM Roberto Ragusa  wrote:
> On 2/4/21 9:52 PM, Alexander Ploumistos wrote:
> > considerable lag. In the last 4 or so years I remember issues with
> > tracker, gnome-shell, mutter/clutter and friends on specific GPUs,
> > default or popular shell extensions and dbus services. A recent bug
>
> If tracker is enabled the performance drop after booting is huge.

There was a time when it worked really well and the performance
penalty of indexing was worth it. I relied a lot on full-text PDF
indexing to locate specific articles among the hundreds on my computer
from the overview in gnome-shell. Right now, even typing the file name
of a file that was not recently accessed yields nothing but tremendous
amounts of heat from the CPU no matter how long one waits… Nowadays I
just dump all the articles in Zotero and sort them from inside the
program as I go along.

[Jeremy Linton]

Hi,

On 2/5/21 3:03 AM, Roberto Ragusa wrote:
> On 2/4/21 9:52 PM, Alexander Ploumistos wrote:
> > considerable lag. In the last 4 or so years I remember issues with
> > tracker, gnome-shell, mutter/clutter and friends on specific GPUs,
> > default or popular shell extensions and dbus services. A recent bug
>
> If tracker is enabled the performance drop after booting is huge.
>
> rpm -e tracker-miners

I regularly have problems with runaway processes on fedora. Frequently 
the  first indication of a problem are fans on a laptop running when the 
machine is idle. Just yesterday, clean install of F33, enabled Plasma, 
and korgac sat there for a few minutes at 100% until I killed it and 
told it never to start again. There isn't anything like a couple runaway 
processes to make the whole machine "sluggish".

Fedora suffers from having a lot of things starting by default that are 
only used by a fraction of the user base. If it were smarter about 
starting/installing things provided by systemd/gnome/kde/etc I suspect 
that not only would it utilize less resources, but there security 
benefits of simply not having much of this running all the time would be 
clearer too. To pick on a couple. iscsi and avahi, are very important 
for a subset of the users, but frankly i suspect they are trivial 
fraction of the overall userbase. But they show up in security audits 
(lynis) simply because systemd-analyze itself reports them as insecure.

Runaway processes are also a bit of an abrt failure in that there isn't 
a good automated way to report them. A package flag to the effect "this 
package doesn't contain anything which should consume 100% cpu for > 10 
seconds" could be set on a wide range of these services to at least 
notify the upstream developers that there might be something wrong. Of 
course this would require yet another always on monitoring daemon.

[Viktor Ashirov]

On Wed, Feb 3, 2021 at 5:54 PM Michael Catanzaro 
wrote:
> Hi,
>
> Has anybody investigated Jim Salter's claims that Fedora 32 is slow to
> launch applications? Recent article:
>
> https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
>
> "in my experience, Fedora 32 is noticeably, demonstrably more sluggish
> to launch applications than Ubuntu is in general."
>
> Original article:
>
> https://arstechnica.com/gadgets/2020/05/linux-distro-review-fedora-workstati...
>
> Would be good to know, for starters, whether this difference is real
> and measurable.

This was bugging me for a while. I also noticed that Fedora 32 is a bit
slower than it used to be. Compilation time of a project that I'm working
on went from ~35-36 seconds to ~47-48. At first I thought that it's just
another round of CPU vulnerabilities mitigations that introduced a
performance drop. But after some digging I found that the default CPU
governor was switched from 'ondemand' to 'schedutil' in Fedora kernel 5.9.7:
https://src.fedoraproject.org/rpms/kernel/c/73c86ebaee23df8310b903c1dab2176d...
(see configs/fedora/generic/CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL)

I switched it back using cpupower from kernel-tools:
$ sudo cpupower frequency-set --governor ondemand

And confirmed that my compilation time went back to the previous ~35
seconds.
In the end I switched the governor to 'performance' and shaved another 5
seconds. And gnome-shell no longer feels sluggish, switching tabs in the
browser is also instant.
To make the change permanent I used settings in /etc/sysconfig/cpupower and
enabled cpupower service:
$ sudo systemctl enable --now cpupower.service

The change of the default CPU governor looks pretty significant to me, but
I couldn't find any discussions about it.
> Michael
>
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Ondrej Mosnacek]

On Thu, Feb 11, 2021 at 10:14 AM Viktor Ashirov  wrote:
> On Wed, Feb 3, 2021 at 5:54 PM Michael Catanzaro  wrote:
> > Hi,
> >
> > Has anybody investigated Jim Salter's claims that Fedora 32 is slow to
> > launch applications? Recent article:
> >
> > https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
> >
> > "in my experience, Fedora 32 is noticeably, demonstrably more sluggish
> > to launch applications than Ubuntu is in general."
> >
> > Original article:
> >
> > https://arstechnica.com/gadgets/2020/05/linux-distro-review-fedora-workstati...
> >
> > Would be good to know, for starters, whether this difference is real
> > and measurable.
>
> This was bugging me for a while. I also noticed that Fedora 32 is a bit slower than it used to be. Compilation time of a project that I'm working on went from ~35-36 seconds to ~47-48. At first I thought that it's just another round of CPU vulnerabilities mitigations that introduced a performance drop. But after some digging I found that the default CPU governor was switched from 'ondemand' to 'schedutil' in Fedora kernel 5.9.7:
> https://src.fedoraproject.org/rpms/kernel/c/73c86ebaee23df8310b903c1dab2176d...
> (see configs/fedora/generic/CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL)
>
> I switched it back using cpupower from kernel-tools:
> $ sudo cpupower frequency-set --governor ondemand
>
> And confirmed that my compilation time went back to the previous ~35 seconds.
> In the end I switched the governor to 'performance' and shaved another 5 seconds. And gnome-shell no longer feels sluggish, switching tabs in the browser is also instant.
> To make the change permanent I used settings in /etc/sysconfig/cpupower and enabled cpupower service:
> $ sudo systemctl enable --now cpupower.service
>
> The change of the default CPU governor looks pretty significant to me, but I couldn't find any discussions about it.

CCing the Fedora kernel list and Justin. At the ARK tree level, the
change was introduced in this commit, with no explanation:
https://gitlab.com/cki-project/kernel-ark/commit/9d69ad49ab90db607e25a99eacb...

Justin, do you remember the reason for the change? Can/should it be reverted?

--
Ondrej Mosnacek
Software Engineer, Linux Security - SELinux kernel
Red Hat, Inc.

[Peter Robinson]

On Thu, Feb 11, 2021 at 1:03 PM Ondrej Mosnacek  wrote:
> On Thu, Feb 11, 2021 at 10:14 AM Viktor Ashirov  wrote:
> > On Wed, Feb 3, 2021 at 5:54 PM Michael Catanzaro  wrote:
> > > Hi,
> > >
> > > Has anybody investigated Jim Salter's claims that Fedora 32 is slow to
> > > launch applications? Recent article:
> > >
> > > https://arstechnica.com/gadgets/2021/02/ubuntu-core-20-adds-secure-boot-with...
> > >
> > > "in my experience, Fedora 32 is noticeably, demonstrably more sluggish
> > > to launch applications than Ubuntu is in general."
> > >
> > > Original article:
> > >
> > > https://arstechnica.com/gadgets/2020/05/linux-distro-review-fedora-workstati...
> > >
> > > Would be good to know, for starters, whether this difference is real
> > > and measurable.
> >
> > This was bugging me for a while. I also noticed that Fedora 32 is a bit slower than it used to be. Compilation time of a project that I'm working on went from ~35-36 seconds to ~47-48. At first I thought that it's just another round of CPU vulnerabilities mitigations that introduced a performance drop. But after some digging I found that the default CPU governor was switched from 'ondemand' to 'schedutil' in Fedora kernel 5.9.7:
> > https://src.fedoraproject.org/rpms/kernel/c/73c86ebaee23df8310b903c1dab2176d...
> > (see configs/fedora/generic/CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL)
> >
> > I switched it back using cpupower from kernel-tools:
> > $ sudo cpupower frequency-set --governor ondemand
> >
> > And confirmed that my compilation time went back to the previous ~35 seconds.
> > In the end I switched the governor to 'performance' and shaved another 5 seconds. And gnome-shell no longer feels sluggish, switching tabs in the browser is also instant.
> > To make the change permanent I used settings in /etc/sysconfig/cpupower and enabled cpupower service:
> > $ sudo systemctl enable --now cpupower.service
> >
> > The change of the default CPU governor looks pretty significant to me, but I couldn't find any discussions about it.
>
> CCing the Fedora kernel list and Justin. At the ARK tree level, the
> change was introduced in this commit, with no explanation:
> https://gitlab.com/cki-project/kernel-ark/commit/9d69ad49ab90db607e25a99eacb...
>
> Justin, do you remember the reason for the change? Can/should it be reverted?

It was upstream changes, the Intel maintainer changed it in [1] if
X86_INTEL_PSTATE state was selected in late March which would make
sense in the timg, and also changed for arm arches [2] in July.

If that change was made upstream I'm assuming it was assumed that
performance should be equivalent or better than the other option, I
suspect we should engage with upstream as they're probably interested
in the issues.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...

[Thorsten Leemhuis]

Am 11.02.21 um 15:28 schrieb Peter Robinson:
> On Thu, Feb 11, 2021 at 1:03 PM Ondrej Mosnacek  wrote:
> > On Thu, Feb 11, 2021 at 10:14 AM Viktor Ashirov  wrote:
> > > On Wed, Feb 3, 2021 at 5:54 PM Michael Catanzaro  wrote:
>
> […]
> > > This was bugging me for a while. I also noticed that Fedora 32 is a bit slower than it used to be. Compilation time of a project that I'm working on went from ~35-36 seconds to ~47-48. At first I thought that it's just another round of CPU vulnerabilities mitigations that introduced a performance drop. But after some digging I found that the default CPU governor was switched from 'ondemand' to 'schedutil' in Fedora kernel 5.9.7:
>
> […]
> It was upstream changes, the Intel maintainer changed it in [1] if
> X86_INTEL_PSTATE state was selected in late March which would make
> sense in the timg, and also changed for arm arches [2] in July.
>
> If that change was made upstream I'm assuming it was assumed that
> performance should be equivalent or better than the other option, I
> suspect we should engage with upstream as they're probably interested
> in the issues.

FWIW, I wonder if some changes that were merged for mainline this week
might be related:

https://git.kernel.org/torvalds/c/291009f656e8eaebbdfd3a8d99f6b190a9ce9deb
https://git.kernel.org/torvalds/c/d11a1d08a082a7dc0ada423d2b2e26e9b6f2525c
https://git.kernel.org/torvalds/c/3c55e94c0adea4a5389c4b80f6ae9927dd6a4501

But I'm not entirely sure what CPUs are affected by d11a1d08a082

HTH, CU, thl

[Akashdeep Dhar]

This was once discussed here in Ask Fedora https://ask.fedoraproject.org/t/how-to-increasing-performance-by-changing-cp... and there's an ongoing investigation regarding the same here https://pagure.io/fedora-workstation/issue/212. You would want to check both to stay posted about the updates regarding this issue.