On Fri, Aug 14, 2020 at 12:42 PM Sergio Belkin <sebelk@gmail.com> wrote:
>
> 2 comments:
> - I don't use disk-based swap, only zram.
> - It happened again, and in this case there is no Virtual Machine nor Zoom app running:
> ago 14 15:08:37 dublin.ireland.home earlyoom[888]: sending SIGTERM to process 2052260 uid 1000 "Web Content": badness 322, VmRSS 447 MiB
> ago 14 15:08:40 dublin.ireland.home earlyoom[888]: process exited after 2.7 seconds
> ago 14 15:12:15 dublin.ireland.home earlyoom[888]: mem avail: 395 of 15887 MiB ( 2.49%), swap free: 0 of 4095 MiB ( 0.00%)
> ago 14 15:12:15 dublin.ireland.home earlyoom[888]: low memory! at or below SIGTERM limits: mem 2.52%, swap 10.00%
> ago 14 15:12:15 dublin.ireland.home earlyoom[888]: sending SIGTERM to process 2055755 uid 1000 "Web Content": badness 319, VmRSS 392 MiB
> ago 14 15:12:17 dublin.ireland.home earlyoom[888]: process exited after 2.8 seconds
> ago 14 15:28:35 dublin.ireland.home earlyoom[888]: mem avail: 371 of 15887 MiB ( 2.34%), swap free: 0 of 4095 MiB ( 0.00%)
> ago 14 15:28:35 dublin.ireland.home earlyoom[888]: low memory! at or below SIGTERM limits: mem 2.52%, swap 10.00%
> ago 14 15:28:35 dublin.ireland.home earlyoom[888]: sending SIGTERM to process 2062157 uid 1000 "Web Content": badness 327, VmRSS 553 MiB
> ago 14 15:28:37 dublin.ireland.home earlyoom[888]: process exited after 2.3 seconds
What's the workload? What is filling up 4G of swap with inactive
pages such that none of them are being freed? The usual case is
available memory is well below the watermark before swap has filled.
But in this case, it's the opposite.
Good question this the output of he last lines of smem -c "name swap" -s swap -k -t
kaccess 4.1M
kded5 4.5M
cadmus 5.0M
konsole 5.4M
xdg-desktop-por 10.6M
mount.ntfs 26.1M
Xorg 30.0M
mysqld 36.6M
plasmashell 36.9M
packagekitd 239.6M
----------------------------------
547.3M
As sigterm is happening, memory available is still dropping. Some
process is still taking up more memory, but has low enough badness
that its not being terminated. It could be one of the exempt
processes.
>
> Perhaps this ps_mem snippet is useful:
> <snip
> Private + Shared = RAM used Program
> </snip>
> ....
> <snip>
> 294.8 MiB + 4.7 MiB = 299.4 MiB telegram-desktop.bin
> 290.3 MiB + 22.1 MiB = 312.4 MiB rocketchat-desktop (5)
> 323.8 MiB + 9.5 MiB = 333.2 MiB kwin_x11 (9)
> 344.5 MiB + 1.4 MiB = 345.8 MiB nextcloud
> 373.2 MiB + 21.2 MiB = 394.4 MiB spotify (5)
> 416.6 MiB + 1.3 MiB = 417.9 MiB plasma-discover
> 448.5 MiB + 16.6 MiB = 465.2 MiB MainThread
> 892.8 MiB + 444.5 KiB = 893.2 MiB packagekitd
> 1.0 GiB + 8.6 MiB = 1.0 GiB plasmashell
> 1.9 GiB + 81.0 MiB = 1.9 GiB Web Content (9)
892M for packagekit? Seems excessive. I wonder what's up with that. It
is an exempt process. On my current system it's 1/2 that amount, which
is in turn twice that of GNOME shell. Hmm.
systemctl status packagekit
● packagekit.service - PackageKit Daemon
Loaded: loaded (/usr/lib/systemd/system/packagekit.service; static; vendor preset: disabled)
Active: active (running) since Thu 2020-07-30 15:40:51 -03; 2 weeks 1 days ago
Main PID: 5393 (packagekitd)
Tasks: 12 (limit: 19015)
Memory: 903.2M
CPU: 9min 6.976s
CGroup: /system.slice/packagekit.service
├─ 5393 /usr/libexec/packagekitd
├─2019772 gpg-agent --homedir /var/cache/PackageKit/32/metadata/fedora-modular-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019790 gpg-agent --homedir /var/cache/PackageKit/32/metadata/updates-modular-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019808 gpg-agent --homedir /var/cache/PackageKit/32/metadata/google-chrome-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019819 gpg-agent --homedir /var/cache/PackageKit/32/metadata/updates-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019838 gpg-agent --homedir /var/cache/PackageKit/32/metadata/rpmfusion-free-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019857 gpg-agent --homedir /var/cache/PackageKit/32/metadata/ring-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019868 gpg-agent --homedir /var/cache/PackageKit/32/metadata/rpmfusion-free-updates-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
├─2019885 gpg-agent --homedir /var/cache/PackageKit/32/metadata/fedora-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
└─2019926 gpg-agent --homedir /var/cache/PackageKit/32/metadata/fedora-cisco-openh264-32-x86_64.tmp/gpgdir --use-standard-socket --daemon
ago 13 00:52:26 dublin.ireland.home PackageKit[5393]: uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth (only_trusted:0)
ago 13 00:52:26 dublin.ireland.home PackageKit[5393]: uid 1000 obtained auth for org.freedesktop.packagekit.system-sources-refresh
ago 13 00:53:48 dublin.ireland.home PackageKit[5393]: refresh-cache transaction /2893_cebbbeab from uid 1000 finished with success after 81694ms
ago 13 00:53:49 dublin.ireland.home PackageKit[5393]: get-updates transaction /2894_ebbbbbee from uid 1000 finished with success after 999ms
ago 13 02:10:12 dublin.ireland.home PackageKit[5393]: get-updates transaction /2895_cceeeece from uid 1000 finished with success after 208ms
ago 14 00:55:03 dublin.ireland.home PackageKit[5393]: uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth (only_trusted:0)
ago 14 00:55:03 dublin.ireland.home PackageKit[5393]: uid 1000 obtained auth for org.freedesktop.packagekit.system-sources-refresh
ago 14 00:56:36 dublin.ireland.home PackageKit[5393]: refresh-cache transaction /2896_eaabbbce from uid 1000 finished with success after 92165ms
ago 14 00:56:37 dublin.ireland.home PackageKit[5393]: get-updates transaction /2897_eebaecab from uid 1000 finished with success after 1062ms
ago 14 01:43:43 dublin.ireland.home PackageKit[5393]: get-updates transaction /2898_bcabedcb from uid 1000 finished with success after 235ms
Now I've restarted packagekit and it outputs:
● packagekit.service - PackageKit Daemon
Loaded: loaded (/usr/lib/systemd/system/packagekit.service; static; vendor preset: disabled)
Active: active (running) since Fri 2020-08-14 17:58:31 -03; 44s ago
Main PID: 2073306 (packagekitd)
Tasks: 3 (limit: 19015)
Memory: 11.2M
CPU: 100ms
CGroup: /system.slice/packagekit.service
└─2073306 /usr/libexec/packagekitd
ago 14 17:58:31 dublin.ireland.home systemd[1]: Starting PackageKit Daemon...
ago 14 17:58:31 dublin.ireland.home PackageKit[2073306]: daemon start
ago 14 17:58:31 dublin.ireland.home systemd[1]: Started PackageKit Daemon.
However swap usage is still high :
free -m
total used free shared buff/cache available
Mem: 15887 8577 1187 4587 6123 2382
Swap: 4095 3854 241
It's weird, isn't it?