Hello,
Apologies if you find the subject line vague or misleading, I couldn't figure out what to write. I've been trying to debug a transient pipewire issue so that I can file a bug report, but I keep stumbling from one roadblock to the next.
When the problem occurs, pipewire receives a SIGSEGV and dies, e.g. kernel: pipewire[2239]: segfault at 0 ip 0000000000000000 sp 00007fc634298998 error 14 in pipewire[55d7316d0000+1000] kernel: Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
At that point, abrt tries to get gdb to process the core dump, which fails because it wants to open a sound device: audit[7970]: AVC avc: denied { read } for pid=7970 comm="gdb" name="pcmC0D0p" dev="devtmpfs" ino=594 scontext=system_u:system_r:abrt_t:s0-s0:c0.c1023 tcontext=system_u:object_r:sound_device_t:s0 tclass=chr_file permissive=0
Should gdb be able to read from a sound device?
There seems to be a problem in impl_node_process.lto_priv.0, but a) I can't reproduce it at will and b) abrt can't send a report to someone who knows what they're doing because it can't process the core dump.
Having root logged in a terminal, I tried to check the state of pipewire.service: systemctl --machine=myuser@.host --user status pipewire.service
to which I got the reply: Failed to get properties: Transport endpoint is not connected
Am I doing something wrong here? How is root supposed to control (or just check on) userspace systemd services? I've also tried the loopback address and my actual IP instead of .host, but no luck with them either. The user can run all the systemctl --user commands just fine.
By the way, what is the proper, Fedora way to restart pipewire? Is $ systemctl --user restart pipewire.service what I should be doing?
I haven't had to bother with a userspace systemd service before, so going down that rabbit hole, at some point I started messing with machinectl, and apparently, running something seemingly innocuous such as "machinectl list-images --all" triggers a SELinux denial. But not always… The message I'm seeing is this: audit[1079]: AVC avc: denied { write } for pid=1079 comm="systemd-machine" name="/" dev="dm-1" ino=2 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=dir permissive=0
setroubleshoot tells me that if I want to allow daemons to dump core, then I should enable the 'daemons_dump_core' boolean. I don't know if I want to, do I need to do that in order to dive deeper into my pipewire issue? How is listing the images on a system related to allowing demons to dump core though?
If anyone can shed some light on any of the above, please do. Again, sorry for the messy subject and the even messier post.
Best regards
On Fri, Jun 18, 2021 at 01:38:08AM +0200, Alexander Ploumistos wrote:
Hello,
Apologies if you find the subject line vague or misleading, I couldn't figure out what to write. I've been trying to debug a transient pipewire issue so that I can file a bug report, but I keep stumbling from one roadblock to the next.
When the problem occurs, pipewire receives a SIGSEGV and dies, e.g. kernel: pipewire[2239]: segfault at 0 ip 0000000000000000 sp 00007fc634298998 error 14 in pipewire[55d7316d0000+1000] kernel: Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
At that point, abrt tries to get gdb to process the core dump, which fails because it wants to open a sound device: audit[7970]: AVC avc: denied { read } for pid=7970 comm="gdb" name="pcmC0D0p" dev="devtmpfs" ino=594 scontext=system_u:system_r:abrt_t:s0-s0:c0.c1023 tcontext=system_u:object_r:sound_device_t:s0 tclass=chr_file permissive=0
Should gdb be able to read from a sound device?
It shouldn't attempt to.
There seems to be a problem in impl_node_process.lto_priv.0, but a) I can't reproduce it at will and b) abrt can't send a report to someone who knows what they're doing because it can't process the core dump.
Having root logged in a terminal, I tried to check the state of pipewire.service: systemctl --machine=myuser@.host --user status pipewire.service
to which I got the reply: Failed to get properties: Transport endpoint is not connected
The way that myuser@.host is implemented is surprisingly complex. See the original commit https://github.com/systemd/systemd/commit/1b630835df for a description.
Does it work for other users? If yes, then it looks like something is borked for that particular user. systemctl -M myuser@.host executes systemd-run -p User=myuser systemd-stdio-bridge -punix:path=${XDG_RUNTIME_DIR}/bus The error message is most likely from systemd-stdio-bridge, or from systemctl failing to connect to the bridge. Since the invocation of systemd-stdio-bridge goes through the system manager, you can look at the logs for that unit ('journalctl -b -u run-uNNNN.service'). It should show some error.
Am I doing something wrong here? How is root supposed to control (or just check on) userspace systemd services? I've also tried the loopback address and my actual IP instead of .host, but no luck with them either. The user can run all the systemctl --user commands just fine.
It sounds like selinux is getting in the way. The switch to connect to the user manager is fairly new functionality, so it seems likely that the policy hasn't caught up. Maybe 'setenforce 0' and see if that helps?
By the way, what is the proper, Fedora way to restart pipewire? Is $ systemctl --user restart pipewire.service what I should be doing?
Yeah, that should work.
I haven't had to bother with a userspace systemd service before, so going down that rabbit hole, at some point I started messing with machinectl, and apparently, running something seemingly innocuous such as "machinectl list-images --all" triggers a SELinux denial. But not always… The message I'm seeing is this: audit[1079]: AVC avc: denied { write } for pid=1079 comm="systemd-machine" name="/" dev="dm-1" ino=2 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=dir permissive=0
setroubleshoot tells me that if I want to allow daemons to dump core, then I should enable the 'daemons_dump_core' boolean. I don't know if I want to, do I need to do that in order to dive deeper into my pipewire issue? How is listing the images on a system related to allowing demons to dump core though?
If machined crashes, there should be logs about this. Did you look at the journal?
Zbyszek
Hello Zbigniew,
Thank you very much for your answers and sorry for the late reply, IRL things have been hectic. I've had the problem with pipewire once since I last wrote, but I can't figure out how to trigger it at will.
On Fri, Jun 18, 2021 at 8:36 AM Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
On Fri, Jun 18, 2021 at 01:38:08AM +0200, Alexander Ploumistos wrote:
Should gdb be able to read from a sound device?
It shouldn't attempt to.
Should I file a bug against gdb or abrt?
Having root logged in a terminal, I tried to check the state of pipewire.service: systemctl --machine=myuser@.host --user status pipewire.service
to which I got the reply: Failed to get properties: Transport endpoint is not connected
The way that myuser@.host is implemented is surprisingly complex. See the original commit https://github.com/systemd/systemd/commit/1b630835df for a description.
Does it work for other users? If yes, then it looks like something is borked for that particular user.
Forgot to check, I will try to do it later today.
systemctl -M myuser@.host executes systemd-run -p User=myuser systemd-stdio-bridge -punix:path=${XDG_RUNTIME_DIR}/bus The error message is most likely from systemd-stdio-bridge, or from systemctl failing to connect to the bridge. Since the invocation of systemd-stdio-bridge goes through the system manager, you can look at the logs for that unit ('journalctl -b -u run-uNNNN.service'). It should show some error.
There is no such service. Assuming NNNN is the uid, I have these units listed: user-runtime-dir@1000.service user@1000.service user-1000.slice
Looking at the logs for user@1000.service, there's nothing pertinent. Actually there's nothing even in the unfiltered log when I issue the command and it fails.
I haven't had to bother with a userspace systemd service before, so going down that rabbit hole, at some point I started messing with machinectl, and apparently, running something seemingly innocuous such as "machinectl list-images --all" triggers a SELinux denial. But not always… The message I'm seeing is this: audit[1079]: AVC avc: denied { write } for pid=1079 comm="systemd-machine" name="/" dev="dm-1" ino=2 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=dir permissive=0
setroubleshoot tells me that if I want to allow daemons to dump core, then I should enable the 'daemons_dump_core' boolean. I don't know if I want to, do I need to do that in order to dive deeper into my pipewire issue? How is listing the images on a system related to allowing demons to dump core though?
If machined crashes, there should be logs about this. Did you look at the journal?
Nothing in the journal about machined crashing or doing anything else other than getting the SELinux denial.
On Sun, Jun 20, 2021 at 03:19:27PM +0200, Alexander Ploumistos wrote:
Hello Zbigniew,
Thank you very much for your answers and sorry for the late reply, IRL things have been hectic. I've had the problem with pipewire once since I last wrote, but I can't figure out how to trigger it at will.
On Fri, Jun 18, 2021 at 8:36 AM Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
On Fri, Jun 18, 2021 at 01:38:08AM +0200, Alexander Ploumistos wrote:
Should gdb be able to read from a sound device?
It shouldn't attempt to.
Should I file a bug against gdb or abrt?
You *may*, but I think it's one of those cases where you'll need to figure it our on your own anyway. Until you have narrowed down what is going wrong, filing a bug is likely to only cause the maintainers to ask you for (a lot of) info.
Having root logged in a terminal, I tried to check the state of pipewire.service: systemctl --machine=myuser@.host --user status pipewire.service
to which I got the reply: Failed to get properties: Transport endpoint is not connected
The way that myuser@.host is implemented is surprisingly complex. See the original commit https://github.com/systemd/systemd/commit/1b630835df for a description.
Does it work for other users? If yes, then it looks like something is borked for that particular user.
Forgot to check, I will try to do it later today.
systemctl -M myuser@.host executes systemd-run -p User=myuser systemd-stdio-bridge -punix:path=${XDG_RUNTIME_DIR}/bus The error message is most likely from systemd-stdio-bridge, or from systemctl failing to connect to the bridge. Since the invocation of systemd-stdio-bridge goes through the system manager, you can look at the logs for that unit ('journalctl -b -u run-uNNNN.service'). It should show some error.
There is no such service. Assuming NNNN is the uid,
No, NNNN is a (pseudorandom) number assigned to the service. Every time you execute 'systemctl -M myuser@.host', this invokes 'systemd-run', which in turns causes 'systemd' to start a new 'run-uNNNN.service' unit.
Looking at the logs for user@1000.service, there's nothing pertinent.
This is not part of user@1000.service.
Actually there's nothing even in the unfiltered log when I issue the command and it fails.
That seems wrong. Maybe you're not seeing all logs because you don't have enough privileges?
I haven't had to bother with a userspace systemd service before, so going down that rabbit hole, at some point I started messing with machinectl, and apparently, running something seemingly innocuous such as "machinectl list-images --all" triggers a SELinux denial. But not always… The message I'm seeing is this: audit[1079]: AVC avc: denied { write } for pid=1079 comm="systemd-machine" name="/" dev="dm-1" ino=2 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=dir permissive=0
setroubleshoot tells me that if I want to allow daemons to dump core, then I should enable the 'daemons_dump_core' boolean. I don't know if I want to, do I need to do that in order to dive deeper into my pipewire issue? How is listing the images on a system related to allowing demons to dump core though?
If machined crashes, there should be logs about this. Did you look at the journal?
Nothing in the journal about machined crashing or doing anything else other than getting the SELinux denial.
The usual method to debugging those is to set permissive mode and then check if the issue still reproduces.
Zbyszek
-
Alexander Ploumistos -
Zbigniew Jędrzejewski-Szmek