Creating cores in f25

[Steve Dickson]

Hello,

How do I get f25 to create cores, these days?

I'm getting the segfault 
   [55108.290610] rpc.gssd[13264]: segfault at 0 ip 000055dc90af9dde sp 00007f9fb73cb7c0
error 4 in rpc.gssd[55dc90af3000+14000]

but no core so those address are meaningless. 

Everything in the kernel is set:
f25# sysctl -a | grep kernel.core        
kernel.core_pattern = |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t %P %I
kernel.core_pipe_limit = 4
kernel.core_uses_pid = 1

The abrtd daemon is up and running
f25# ps ax | grep abrtd
  713 ?        Ssl    0:00 /usr/sbin/abrtd -d -s

What am I missing?

tia,

steved.

[Jan Kratochvil]

On Sun, 18 Dec 2016 17:26:40 +0100, Steve Dickson wrote:
>  How do I get f25 to create cores, these days? 
echo >/etc/sysctl.d/foo.conf "kernel.core_pattern=core"; reboot

It gets broken by:
	/usr/lib/sysctl.d/50-coredump.conf

$ rpm -qf /usr/lib/sysctl.d/50-coredump.conf
systemd-231-10.fc25.x86_64


Jan

[Matthew Miller]

On Sun, Dec 18, 2016 at 05:56:04PM +0100, Jan Kratochvil wrote:
>  On Sun, 18 Dec 2016 17:26:40 +0100, Steve Dickson wrote:
>  > How do I get f25 to create cores, these days?
>  echo >/etc/sysctl.d/foo.conf "kernel.core_pattern=core"; reboot
>  It gets broken by:
>  	/usr/lib/sysctl.d/50-coredump.conf
>  $ rpm -qf /usr/lib/sysctl.d/50-coredump.conf
>  systemd-231-10.fc25.x86_64 
On my system, it's actually abrt-ccpp overriding _that_. See
proposed change https://fedoraproject.org/wiki/Changes/coredumpctl


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Zbigniew Jędrzejewski-Szmek]

On Sun, Dec 18, 2016 at 12:24:09PM -0500, Matthew Miller wrote:
>  On Sun, Dec 18, 2016 at 05:56:04PM +0100, Jan Kratochvil wrote:
>  > On Sun, 18 Dec 2016 17:26:40 +0100, Steve Dickson wrote:
>  > > How do I get f25 to create cores, these days?
>  > echo >/etc/sysctl.d/foo.conf "kernel.core_pattern=core"; reboot
>  > It gets broken by:
>  > 	/usr/lib/sysctl.d/50-coredump.conf
>  > $ rpm -qf /usr/lib/sysctl.d/50-coredump.conf
>  > systemd-231-10.fc25.x86_64
>  
>  On my system, it's actually abrt-ccpp overriding _that_. See
>  proposed change https://fedoraproject.org/wiki/Changes/coredumpctl 
Yes. "foo.conf" sorts after "50-coredump.conf", so you setting has
higher
priority, so systemd-sysctl should set kernel.core_pattern=core.
Most likely it's abrt overriding that pattern.

BTW, no need to reboot to reload sysctl settings,
'sudo /usr/lib/systemd/systemd-sysctl' does the same thing that happens
during boot.

Zbyszek

[Steve Dickson]

On 12/18/2016 11:56 AM, Jan Kratochvil wrote:
>  On Sun, 18 Dec 2016 17:26:40 +0100, Steve Dickson wrote:
> > How do I get f25 to create cores, these days?
>  
>  echo >/etc/sysctl.d/foo.conf "kernel.core_pattern=core"; reboot
>  
>  It gets broken by:
>  	/usr/lib/sysctl.d/50-coredump.conf
>  
>  $ rpm -qf /usr/lib/sysctl.d/50-coredump.conf
>  systemd-231-10.fc25.x86_64 
Unfortunately this did not work... :-( Thanks though...


steved.

[Michael Catanzaro]

On Sun, 2016-12-18 at 11:26 -0500, Steve Dickson wrote:
>  Hello,
>  
>  How do I get f25 to create cores, these days? 
Some more details here:

https://fedoraproject.org/wiki/Changes/coredumpctl

Michael

[Matthew Miller]

On Sun, Dec 18, 2016 at 01:00:37PM -0600, Michael Catanzaro wrote:
>  > How do I get f25 to create cores, these days?
>  Some more details here:
>  https://fedoraproject.org/wiki/Changes/coredumpctl 
I have ulimit -c returning 0 in shells on my system — have I done some
configuration I don't remember? That's the default, isn't it? Should it
stay that way with this change?

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Tom Hughes]

On 18/12/16 23:08, Matthew Miller wrote:
>  On Sun, Dec 18, 2016 at 01:00:37PM -0600, Michael Catanzaro wrote:
> >> How do I get f25 to create cores, these days?
> > Some more details here:
> > https://fedoraproject.org/wiki/Changes/coredumpctl
>
>  I have ulimit -c returning 0 in shells on my system — have I done some
>  configuration I don't remember? That's the default, isn't it? Should it
>  stay that way with this change? 
The ulimit on core dump size is (mostly) ignored if sysctl has been used 
to tell the kernel to send core dumps to a pipe.

The only exception is that a limit of 1 byte (which is impossible to set 
in most shells) effectively acts as a limit of 0 and disables core dumps 
but that's only because the kernel uses it internally to ignore core 
dumps from the process running the pipe to receive a core dump.

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

[Matthew Miller]

On Mon, Dec 19, 2016 at 12:07:06AM +0000, Tom Hughes wrote:
>  >I have ulimit -c returning 0 in shells on my system — have I done
> some
>  >configuration I don't remember? That's the default, isn't it? Should it
>  >stay that way with this change?
>  The ulimit on core dump size is (mostly) ignored if sysctl has been
>  used to tell the kernel to send core dumps to a pipe. 
Not in my testing just now — I made a small segfault program and tried
it out; with the default (ulimit -c 0) and `coredumpctl gdb` I get:

  Cannot retrieve coredump from journal or disk.
  Failed to retrieve core: No such file or directory


but when I set ulimit -c unlimited and repeat, it works.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Tom Hughes]

On 19/12/16 00:17, Matthew Miller wrote:
>  On Mon, Dec 19, 2016 at 12:07:06AM +0000, Tom Hughes wrote:
> >> I have ulimit -c returning 0 in shells on my system — have I done some
> >> configuration I don't remember? That's the default, isn't it? Should
> it
> >> stay that way with this change?
> > The ulimit on core dump size is (mostly) ignored if sysctl has been
> > used to tell the kernel to send core dumps to a pipe.
>
>  Not in my testing just now — I made a small segfault program and tried
>  it out; with the default (ulimit -c 0) and `coredumpctl gdb` I get:
>
>    Cannot retrieve coredump from journal or disk.
>    Failed to retrieve core: No such file or directory
>
>  but when I set ulimit -c unlimited and repeat, it works. 
Well all I know is what the kernel code says and what I had to do (ie 
set the limit to one byte) to stop abrt chewing CPU for hours trying to 
ingest large core dumps. The code in question is here:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/c...

As you can see, if it the limit (cprm.limit is initialised from the 
ulimit at the top of the function) is one it aborts otherwise it raises 
it to infinity for the duration of the current dump.

Maybe systemd is reading the limit itself and applying it to the data it 
receives through the pipe?

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

[Michael Catanzaro]

On Mon, 2016-12-19 at 00:25 +0000, Tom Hughes wrote:
>  Maybe systemd is reading the limit itself and applying it to the
> data
>  it 
>  receives through the pipe? 
Yes.

I'm not sure that systemd actually uses the ulimit for more than the
binary decision of whether or not to generate the core, though.

Michael

[Lennart Poettering]

On Mon, 19.12.16 00:07, Tom Hughes (tom(a)compton.nu) wrote:

>  On 18/12/16 23:08, Matthew Miller wrote:
>  > On Sun, Dec 18, 2016 at 01:00:37PM -0600, Michael Catanzaro wrote:
>  > > > How do I get f25 to create cores, these days?
>  > > Some more details here:
>  > > https://fedoraproject.org/wiki/Changes/coredumpctl
>  > 
>  > I have ulimit -c returning 0 in shells on my system — have I done some
>  > configuration I don't remember? That's the default, isn't it? Should it
>  > stay that way with this change?
>  
>  The ulimit on core dump size is (mostly) ignored if sysctl has been used to
>  tell the kernel to send core dumps to a pipe.
>  
>  The only exception is that a limit of 1 byte (which is impossible to set in
>  most shells) effectively acts as a limit of 0 and disables core dumps but
>  that's only because the kernel uses it internally to ignore core dumps from
>  the process running the pipe to receive a core dump. 
Note that "systemd-coredump" will honour the resource limit anyway,
even if the kernel ignores it as soon as a core_pattern is
installed. I am pretty sure we should honour resource limits if they
are set, as they are pretty OK and well-documented way how to enable
and disable coredump collection, and well, putting limits on them.

Note that abrtd (to my knowledge) ignores the resource limit, and will
collect coredumps in anyway if if disabled with it.

Lennart

-- 
Lennart Poettering, Red Hat

[Jakub Filak]

---------- Původní zpráva ----------
Od: Lennart Poettering <mzerqung(a)0pointer.de&gt;
Komu: Development discussions related to Fedora <devel(a)lists.fedoraproject.
org>
Datum: 19. 12. 2016 13:51:09
Předmět: Re: Creating cores in f25

"On Mon, 19.12.16 00:07, Tom Hughes (tom(a)compton.nu) wrote:

>  On 18/12/16 23:08, Matthew Miller wrote:
>  > On Sun, Dec 18, 2016 at 01:00:37PM -0600, Michael Catanzaro wrote:
>  > > > How do I get f25 to create cores, these days?
>  > > Some more details here:
>  > > https://fedoraproject.org/wiki/Changes/coredumpctl
>  > 
>  > I have ulimit -c returning 0 in shells on my system — have I done some
>  > configuration I don't remember? That's the default, isn't it? Should it
>  > stay that way with this change?
>  
>  The ulimit on core dump size is (mostly) ignored if sysctl has been used  
to
>  tell the kernel to send core dumps to a pipe.
>  
>  The only exception is that a limit of 1 byte (which is impossible to set  
in
>  most shells) effectively acts as a limit of 0 and disables core dumps
> but
>  that's only because the kernel uses it internally to ignore core dumps  
from
>  the process running the pipe to receive a core dump. 
Note that "systemd-coredump" will honour the resource limit anyway,
even if the kernel ignores it as soon as a core_pattern is
installed. I am pretty sure we should honour resource limits if they
are set, as they are pretty OK and well-documented way how to enable
and disable coredump collection, and well, putting limits on them.

Note that abrtd (to my knowledge) ignores the resource limit, and will
collect coredumps in anyway if if disabled with it.

"



ABRT does not ignore the resource limit. If the limit is not 0, ABRT creates
a core file according the value of 'sysctl kernel.core_pattern' before it 
was changed to ABRT pattern. This basically means that ABRT creates core 
file in prorcess' CWD if the limit (ulimit -c) is not 0. This is the 
behavior we are used to.




However, it is true that ABRT creates a core dump file in /var/spool/abrt/ 
regardless of the limit.




It is possible to tell kernel to skip core dumping at all by setting "RLIMIT
_CORE" to 1.










Regards,

Jakub

[Michael Catanzaro]

On Sun, 2016-12-18 at 18:08 -0500, Matthew Miller wrote:
>  I have ulimit -c returning 0 in shells on my system — have I done
>  some
>  configuration I don't remember? That's the default, isn't it? Should
>  it
>  stay that way with this change? 
No, the default is supposed to be unlimited. systemd sets it to
unlimited but is blocked by SELinux. See:

https://bugzilla.redhat.com/show_bug.cgi?id=1341829
https://bugzilla.redhat.com/show_bug.cgi?id=1365435

Also see changes in systemd 229:

https://github.com/systemd/systemd/blob/master/NEWS

Michael

[Jakub Filak]

Hi Steve,




Please have a look at this email:

https://lists.fedoraproject.org/archives/list/[e-mail]/
message/HQ4JFTYLPT5GRW6AD4M2MWGMRAPE7ITN/





systemd developers has decided to change the default RLIMIT_CORE (ulimit -c)
from "0" to unlimited, therefore ABRT must stop creating the core dump files
in CWD. Otherwise, you will get core dumps spread all over your file system 
- that is because of crashes of daemon that used to not crash with core dump
file (their RLIMIT_CORE were 0 by default; starting with systemd-229 the 
default RLIMIT_CORE is unlimited).







Regards,

Jakub


---------- Původní zpráva ----------
Od: Steve Dickson <SteveD(a)redhat.com&gt;
Komu: Development discussions related to Fedora <devel(a)lists.fedoraproject.
org>
Datum: 18. 12. 2016 17:35:43
Předmět: Creating cores in f25

"Hello,

How do I get f25 to create cores, these days?

I'm getting the segfault 
[55108.290610] rpc.gssd[13264]: segfault at 0 ip 000055dc90af9dde sp 00007f9
fb73cb7c0 error 4 in rpc.gssd[55dc90af3000+14000]

but no core so those address are meaningless. 

Everything in the kernel is set:
f25# sysctl -a | grep kernel.core 
kernel.core_pattern = |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t %P %I
kernel.core_pipe_limit = 4
kernel.core_uses_pid = 1

The abrtd daemon is up and running
f25# ps ax | grep abrtd
713 ? Ssl 0:00 /usr/sbin/abrtd -d -s

What am I missing?

tia,

steved.
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
"

[Steve Dickson]

On 12/19/2016 12:38 PM, jfilak(a)fedoraproject.org wrote:
>  Hi Steve,
>  
>  Please have a look at this email:
>
> https://lists.fedoraproject.org/archives/list/[e-mail].o...
Thanks for the pointer... 
>  
>  systemd developers has decided to change the default RLIMIT_CORE (ulimit -c) 
>  from "0" to unlimited, therefore ABRT must stop creating the core dump files in
> CWD. 
>  Otherwise, you will get core dumps spread all over your file system - 
>  that is because of crashes of daemon that used to not crash with core 
>  dump file (their RLIMIT_CORE were 0 by default; starting with systemd-229 
>  the default RLIMIT_CORE is unlimited). 
I guess I really don't care where the
core dump is created... 
I just need one to be created!! ;-) 

steved.

> 
> 
> Regards,
> Jakub
> 
> ---------- Původní zpráva ----------
> Od: Steve Dickson <SteveD(a)redhat.com&gt;
> Komu: Development discussions related to Fedora
<devel(a)lists.fedoraproject.org&gt;
> Datum: 18. 12. 2016 17:35:43
> Předmět: Creating cores in f25
> 
> 
>     Hello,
> 
>     How do I get f25 to create cores, these days?
> 
>     I'm getting the segfault
>     [55108.290610] rpc.gssd[13264]: segfault at 0 ip 000055dc90af9dde sp
00007f9fb73cb7c0 error 4 in rpc.gssd[55dc90af3000+14000]
> 
>     but no core so those address are meaningless.
> 
>     Everything in the kernel is set:
>     f25# sysctl -a | grep kernel.core
>     kernel.core_pattern = |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t %P %I
>     kernel.core_pipe_limit = 4
>     kernel.core_uses_pid = 1
> 
>     The abrtd daemon is up and running
>     f25# ps ax | grep abrtd
>     713 ? Ssl 0:00 /usr/sbin/abrtd -d -s
> 
>     What am I missing?
> 
>     tia,
> 
>     steved.
>     _______________________________________________
>     devel mailing list -- devel(a)lists.fedoraproject.org
>     To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> 
> 
> 
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>