1:45 p.m.
Hi, folks!
openQA tests on Rawhide are currently failing quite often, especially
on KDE, apparently because llvmpipe is crashing and taking down the
desktop environment. In the system logs, it looks like this:
2022-07-12T06:26:09.804990-04:00 fedora audit[1304]: ANOM_ABEND auid=1000 uid=1000
gid=1000 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 pid=1304
comm="llvmpipe-0" exe="
/usr/bin/kwin_wayland" sig=11 res=1
2022-07-12T06:26:09.805624-04:00 fedora kernel: show_signal_msg: 73 callbacks suppressed
2022-07-12T06:26:09.805761-04:00 fedora kernel: llvmpipe-0[1338]: segfault at 7fec83f00a30
ip 00007feca80787af sp 00007fecb4fd6850 error 4
2022-07-12T06:26:09.805831-04:00 fedora kernel: Code: 6d f4 66 44 0f 6f e5 66 44 0f 6c e0
66 0f 6d e8 41 0f 28 c3 41 0f 16 c1 66 45 0f 15 d9 41 0f 28 da 41 0f 16 d8 66 45 0f 15 d0
<66> 0f 6f 08 66 0f 6f 14 08 8d 14 09 48 63 d2 66 0f 6f 24 10 8d 34
2022-07-12T06:26:09.830632-04:00 fedora systemd[1]: Created slice
system-systemd\x2dcoredump.slice - Slice /system/systemd-coredump.
2022-07-12T06:26:09.833124-04:00 fedora audit: BPF prog-id=137 op=LOAD
2022-07-12T06:26:09.833599-04:00 fedora audit: BPF prog-id=138 op=LOAD
2022-07-12T06:26:09.833751-04:00 fedora audit: BPF prog-id=139 op=LOAD
2022-07-12T06:26:09.837271-04:00 fedora systemd[1]: Started
systemd-coredump(a)0-2189-0.service - Process Core Dump (PID 2189/UID 0).
2022-07-12T06:26:09.837920-04:00 fedora audit[1]: SERVICE_START pid=1 uid=0
auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0
msg='unit=systemd-coredump@0-2189-0 comm="systemd"
exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
2022-07-12T06:26:09.982088-04:00 fedora plasmashell[1382]: org.kde.plasma.notifications:
Failed to generate job text for job with following properties:
2022-07-12T06:26:09.992539-04:00 fedora plasmashell[1382]: org.kde.plasma.notifications:
processedFiles = 0, totalFiles = 0
2022-07-12T06:26:09.992687-04:00 fedora plasmashell[1382]: org.kde.plasma.notifications:
current file name = ""
2022-07-12T06:26:09.992735-04:00 fedora plasmashell[1382]: org.kde.plasma.notifications:
destination url = QUrl("")
2022-07-12T06:26:09.992791-04:00 fedora plasmashell[1382]: org.kde.plasma.notifications:
label1 = "", value1 = ""
2022-07-12T06:26:09.992851-04:00 fedora plasmashell[1382]: org.kde.plasma.notifications:
label2 = "", value2 = ""
2022-07-12T06:26:11.184992-04:00 fedora [2195]: json_parse on
{"type":"rpm","name":"libedit","version":"3.1-41.20210910cvs.fc36","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:36"}<C5>
failed: Invalid argument
2022-07-12T06:26:11.190411-04:00 fedora systemd-coredump[2190]: Process 1304
(kwin_wayland) of user 1000 dumped core.#012#012Module linux-vdso.so.1 with build-id
805638058c919fd779cc1e1fa412f8832069dbf3#012Module breezedecoration.so with build-id
e715bf13e75d84216d6607d4cdeafc4f1a68e95e#012Metadata for module breezedecoration.so owned
by FDO found: {#012#011"type" : "rpm",#012#011"name" :
"plasma-breeze",#012#011"version" :
"5.25.2-1.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module libxshmfence.so.1 with
build-id 930360dde486956810e2a4ab67efae781f463ced#012Metadata for module libxshmfence.so.1
owned by FDO found: {#012#011"type" : "rpm",#012#011"name" :
"libxshmfence",#012#011"version" :
"1.3-10.fc36",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:36"#012}#012#012Module libxcb-present.so.0 with
build-id e55a074882c4c9357934efcf295654ea5badcce1#012Metadata for module
libxcb-present.so.0 owned by FDO found: {#012#011"type" :
"rpm",#012#011"name" : "libxcb",#012#011"version"
: "1.13.1-9.fc36",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:36"#012}#012#012Module libxcb-dri2.so.0 with
build-id b557629bc47fea6de9edb4534378f26906bc5d80#012Metadata for module libxcb-dri2.so.0
owned by FDO found: {#012#011"type" : "rpm",#012#011"name" :
"libxcb",#012#011"version" :
"1.13.1-9.fc36",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:36"#012}#012#012Module libX11-xcb.so.1 with
build-id b395bdbc52537c00bd88a7a631dcac49b7c40934#012Metadata for module libX11-xcb.so.1
owned by FDO found: {#012#011"type" : "rpm",#012#011"name" :
"libX11",#012#011"version" :
"1.8.1-1.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module libEGL_mesa.so.0 with
build-id 733f6ea3cfd4357a9668e6b3c5a3968af4533690#012Metadata for module libEGL_mesa.so.0
owned by FDO found: {#012#011"type" : "rpm",#012#011"name" :
"mesa",#012#011"version" :
"22.1.3-1.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module libKWinNightColorPlugin.so
with build-id dc82a504bc7fdd79bf4e7d69b6fa8137cbaf644a#012Metadata for module
libKWinNightColorPlugin.so owned by FDO found: {#012#011"type" :
"rpm",#012#011"name" : "kwin",#012#011"version" :
"5.25.2-1.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module krunnerintegration.so with
build-id 9b62ad046672cd903572c0a72c38f36580de7588#012Metadata for module
krunnerintegration.so owned by FDO found: {#012#011"type" :
"rpm",#012#011"name" : "kwin",#012#011"version" :
"5.25.2-1.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module colordintegration.so with
build-id 6266591e822c25984dc2d50249713f145a8568e3#012Metadata for module
colordintegration.so owned by FDO found: {#012#011"type" :
"rpm",#012#011"name" : "kwin",#012#011"version" :
"5.25.2-1.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module libtinfo.so.6 with build-id
3316591b8bdf1e4d2cd97a12c2ec8099f01a96ba#012Metadata for module libtinfo.so.6 owned by FDO
found: {#012#011"type" : "rpm",#012#011"name" :
"ncurses",#012#011"version" :
"6.3-2.20220501.fc37",#012#011"architecture" :
"x86_64",#012#011"osCpe" :
"cpe:/o:fedoraproject:fedora:37"#012}#012#012Module libedit.so.0 with build-id
786ebbe150c63e27beb2957d717bece33431af6f#012Stack trace of thread 1338:#012#0
0x00007feca80787af n/a (n/a + 0x0)#012ELF object binary architecture: AMD x86-64
2022-07-12T06:26:11.267438-04:00 fedora systemd[1]: systemd-coredump(a)0-2189-0.service:
Deactivated successfully.
we get a core dump from the window manager process (kwin_wayland in
this case), but backtracing it the usual way does not produce anything
useful. So I've been a bit stuck on how to proceed with debugging from
here.
It looks like this started happening around a month ago in F36 (it's
also happening in Rawhide). I think it ties in with mesa-22.1.1 going
stable:
https://bodhi.fedoraproject.org/updates/FEDORA-2022-db6aea82e4
This person may be hitting the same issue:
https://www.reddit.com/r/Fedora/comments/vswyq4/fedora_36_llvmpipe_segfault/
How can we debug this and hopefully get it fixed? Can anyone give me
any pointers? Thanks!
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net