12:12 a.m.
Hi,
We have SSSD authenticating against Active Directory on a large cluster of
hadoop machines. Intermittently we're seeing JVM processes (Apache Spark
jobs) core dumping when they attempt to lookup the group owner of a file.
The group comes from Active Directory. The group contains roughly 30 users.
Is anyone able to help identify what might be causing this?
############################################################
(gdb) bt
#0 0x00007f789005acc9 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f789005e0d8 in __GI_abort () at abort.c:89
#2 0x00007f788f3abd69 in os::abort(bool) () from
/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#3 0x00007f788f53133f in VMError::report_and_die() () from
/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#4 0x00007f788f3b4b4f in JVM_handle_linux_signal () from
/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#5 <signal handler called>
#6 sss_nss_check_header (ctx=ctx@entry=0x7f788d541280 <gr_mc_ctx>) at
../src/sss_client/nss_mc_common.c:65
#7 0x00007f788d33ed1b in sss_nss_mc_get_ctx (name=name@entry=0x7f788d33fae1
"group", ctx=ctx@entry=0x7f788d541280 <gr_mc_ctx>) at
../src/sss_client/nss_mc_common.c:151
#8 0x00007f788d33f7d9 in sss_nss_mc_getgrgid (gid=gid@entry=10002,
result=result@entry=0x7f783d325800, buffer=0x14f2bb0 "postdrop",
buflen=buflen@entry=1024) at ../src/sss_client/nss_mc_group.c:182
#9 0x00007f788d33da56 in _nss_sss_getgrgid_r (gid=10002,
result=0x7f783d325800, buffer=0x14f2bb0 "postdrop", buflen=1024,
errnop=0x7f783d329660) at ../src/sss_client/nss_group.c:454
#10 0x00007f78900e2b0c in __getgrgid_r (gid=10002, resbuf=0x7f783d325800,
buffer=0x14f2bb0 "postdrop", buflen=1024, result=0x7f783d325828) at
../nss/getXXbyYY_r.c:266
#11 0x00007f7841cabfe6 in ?? ()
#12 0x00000000014f2bb0 in ?? ()
############################################################
Here's our sssd config:
/etc/sssd/sssd.conf
[sssd]
config_file_version = 2
services = nss, pam
domains = LDAP
#debug_level = 0x4000
[nss]
[pam]
[domain/LDAP]
id_provider = ldap
auth_provider = ldap
chpass_provider = ldap
ldap_schema = rfc2307bis
ldap_uri = ldaps://192.168.16.2,ldaps://192.168.16.5
ldap_search_base = <hidden>
ldap_id_mapping = False
ldap_user_search_base = <hidden>
ldap_group_search_base = <hidden>
ldap_user_object_class = user
ldap_user_name = msSFU30Name
ldap_user_fullname = displayName
ldap_user_home_directory = unixHomeDirectory
ldap_user_principal = userPrincipalName
ldap_group_object_class = group
ldap_group_name = sAMAccountName
ldap_user_uid_number = uidNumber
ldap_user_gid_number = gidNumber
#Bind credentials
ldap_default_bind_dn = <CN>
ldap_default_authtok = secret
ldap_tls_reqcert = allow
cache_credentials = true
enumerate = false
Our nsswitch.conf:
passwd: compat sss
group: compat sss
shadow: compat
hosts: files dns
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis sss
sudoers: files sss
$ grep sss /etc/pam.d/
common-account:account [default=bad success=ok user_unknown=ignore]
pam_sss.so
common-auth:auth [success=2 default=ignore] pam_sss.so use_first_pass
common-password:password sufficient pam_sss.so use_authtok
common-session:session optional pam_sss.so
Versions:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"
Linux 3.13.0-49-generic #83-Ubuntu SMP Fri Apr 10 20:11:33 UTC 2015 x86_64
x86_64 x86_64 GNU/Linux
$ dpkg -l | grep sssd
ii sssd 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- metapackage
ii sssd-ad 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- Active
Directory back end
ii sssd-ad-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
PAC responder
ii sssd-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
common files
ii sssd-ipa 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- IPA
back end
ii sssd-krb5 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
Kerberos back end
ii sssd-krb5-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon
-- Kerberos helpers
ii sssd-ldap 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- LDAP
back end
ii sssd-proxy 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
proxy back end
ii sssd-tools 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- tools
1:30 a.m.
On (14/07/15 15:12), Gerard . wrote:
Hi,
We have SSSD authenticating against Active Directory on a large cluster of
hadoop machines. Intermittently we're seeing JVM processes (Apache Spark
jobs) core dumping when they attempt to lookup the group owner of a file.
The group comes from Active Directory. The group contains roughly 30 users.
Is anyone able to help identify what might be causing this?
############################################################
(gdb) bt
#0 0x00007f789005acc9 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f789005e0d8 in __GI_abort () at abort.c:89
#2 0x00007f788f3abd69 in os::abort(bool) () from
/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#3 0x00007f788f53133f in VMError::report_and_die() () from
/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#4 0x00007f788f3b4b4f in JVM_handle_linux_signal () from
/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#5 <signal handler called>
#6 sss_nss_check_header (ctx=ctx@entry=0x7f788d541280 <gr_mc_ctx>) at
../src/sss_client/nss_mc_common.c:65
#7 0x00007f788d33ed1b in sss_nss_mc_get_ctx (name=name@entry=0x7f788d33fae1
"group", ctx=ctx@entry=0x7f788d541280 <gr_mc_ctx>) at
../src/sss_client/nss_mc_common.c:151
#8 0x00007f788d33f7d9 in sss_nss_mc_getgrgid (gid=gid@entry=10002,
result=result@entry=0x7f783d325800, buffer=0x14f2bb0 "postdrop",
buflen=buflen@entry=1024) at ../src/sss_client/nss_mc_group.c:182
#9 0x00007f788d33da56 in _nss_sss_getgrgid_r (gid=10002,
result=0x7f783d325800, buffer=0x14f2bb0 "postdrop", buflen=1024,
errnop=0x7f783d329660) at ../src/sss_client/nss_group.c:454
#10 0x00007f78900e2b0c in __getgrgid_r (gid=10002, resbuf=0x7f783d325800,
buffer=0x14f2bb0 "postdrop", buflen=1024, result=0x7f783d325828) at
../nss/getXXbyYY_r.c:266
#11 0x00007f7841cabfe6 in ?? ()
#12 0x00000000014f2bb0 in ?? ()
############################################################
Here's our sssd config:
/etc/sssd/sssd.conf
[sssd]
config_file_version = 2
services = nss, pam
domains = LDAP
#debug_level = 0x4000
[nss]
[pam]
[domain/LDAP]
id_provider = ldap
auth_provider = ldap
chpass_provider = ldap
ldap_schema = rfc2307bis
ldap_uri = ldaps://192.168.16.2,ldaps://192.168.16.5
ldap_search_base = <hidden>
ldap_id_mapping = False
ldap_user_search_base = <hidden>
ldap_group_search_base = <hidden>
ldap_user_object_class = user
ldap_user_name = msSFU30Name
ldap_user_fullname = displayName
ldap_user_home_directory = unixHomeDirectory
ldap_user_principal = userPrincipalName
ldap_group_object_class = group
ldap_group_name = sAMAccountName
ldap_user_uid_number = uidNumber
ldap_user_gid_number = gidNumber
#Bind credentials
ldap_default_bind_dn = <CN>
ldap_default_authtok = secret
ldap_tls_reqcert = allow
cache_credentials = true
enumerate = false
Our nsswitch.conf:
passwd: compat sss
group: compat sss
shadow: compat
hosts: files dns
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis sss
sudoers: files sss
$ grep sss /etc/pam.d/
common-account:account [default=bad success=ok user_unknown=ignore]
pam_sss.so
common-auth:auth [success=2 default=ignore] pam_sss.so use_first_pass
common-password:password sufficient pam_sss.so use_authtok
common-session:session optional pam_sss.so
Versions:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"
Linux 3.13.0-49-generic #83-Ubuntu SMP Fri Apr 10 20:11:33 UTC 2015 x86_64
x86_64 x86_64 GNU/Linux
$ dpkg -l | grep sssd
ii sssd 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- metapackage
ii sssd-ad 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- Active
Directory back end
ii sssd-ad-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
PAC responder
ii sssd-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
common files
ii sssd-ipa 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- IPA
back end
ii sssd-krb5 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
Kerberos back end
ii sssd-krb5-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon
-- Kerberos helpers
ii sssd-ldap 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- LDAP
back end
ii sssd-proxy 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
proxy back end
ii sssd-tools 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- tools
It looks like a bug(s)
https://fedorahosted.org/sssd/ticket/2409
https://fedorahosted.org/sssd/ticket/2445
They are fixed in sssd >= 1.12.3
You can try to update from Timo's repository
https://launchpad.net/~sssd/+archive/ubuntu/updates
But it reminds me I wanted to prepare latest patches
for the las 1.11 release.
LS
11:46 p.m.
This fixed the bug. Thank you!
On Tue, Jul 14, 2015, 16:30 Lukas Slebodnik <lslebodn(a)redhat.com> wrote:
On (14/07/15 15:12), Gerard . wrote:
>Hi,
>
>We have SSSD authenticating against Active Directory on a large cluster of
>hadoop machines. Intermittently we're seeing JVM processes (Apache Spark
>jobs) core dumping when they attempt to lookup the group owner of a file.
>The group comes from Active Directory. The group contains roughly 30
users.
>
>Is anyone able to help identify what might be causing this?
>
>############################################################
>(gdb) bt
>#0 0x00007f789005acc9 in __GI_raise (sig=sig@entry=6) at
>../nptl/sysdeps/unix/sysv/linux/raise.c:56
>#1 0x00007f789005e0d8 in __GI_abort () at abort.c:89
>#2 0x00007f788f3abd69 in os::abort(bool) () from
>/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
>#3 0x00007f788f53133f in VMError::report_and_die() () from
>/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
>#4 0x00007f788f3b4b4f in JVM_handle_linux_signal () from
>/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so
>#5 <signal handler called>
>#6 sss_nss_check_header (ctx=ctx@entry=0x7f788d541280 <gr_mc_ctx>) at
>../src/sss_client/nss_mc_common.c:65
>#7 0x00007f788d33ed1b in sss_nss_mc_get_ctx (name=name@entry
=0x7f788d33fae1
>"group", ctx=ctx@entry=0x7f788d541280 <gr_mc_ctx>) at
>../src/sss_client/nss_mc_common.c:151
>#8 0x00007f788d33f7d9 in sss_nss_mc_getgrgid (gid=gid@entry=10002,
>result=result@entry=0x7f783d325800, buffer=0x14f2bb0 "postdrop",
>buflen=buflen@entry=1024) at ../src/sss_client/nss_mc_group.c:182
>#9 0x00007f788d33da56 in _nss_sss_getgrgid_r (gid=10002,
>result=0x7f783d325800, buffer=0x14f2bb0 "postdrop", buflen=1024,
>errnop=0x7f783d329660) at ../src/sss_client/nss_group.c:454
>#10 0x00007f78900e2b0c in __getgrgid_r (gid=10002, resbuf=0x7f783d325800,
>buffer=0x14f2bb0 "postdrop", buflen=1024, result=0x7f783d325828) at
>../nss/getXXbyYY_r.c:266
>#11 0x00007f7841cabfe6 in ?? ()
>#12 0x00000000014f2bb0 in ?? ()
>
>############################################################
>
>Here's our sssd config:
>
>/etc/sssd/sssd.conf
>
>[sssd]
>config_file_version = 2
>services = nss, pam
>domains = LDAP
>#debug_level = 0x4000
>
>[nss]
>
>[pam]
>
>[domain/LDAP]
>id_provider = ldap
>auth_provider = ldap
>chpass_provider = ldap
>
>ldap_schema = rfc2307bis
>ldap_uri = ldaps://192.168.16.2,ldaps://192.168.16.5
>ldap_search_base = <hidden>
>
>ldap_id_mapping = False
>
>ldap_user_search_base = <hidden>
>ldap_group_search_base = <hidden>
>ldap_user_object_class = user
>ldap_user_name = msSFU30Name
>ldap_user_fullname = displayName
>ldap_user_home_directory = unixHomeDirectory
>ldap_user_principal = userPrincipalName
>ldap_group_object_class = group
>ldap_group_name = sAMAccountName
>ldap_user_uid_number = uidNumber
>ldap_user_gid_number = gidNumber
>
>#Bind credentials
>ldap_default_bind_dn = <CN>
>ldap_default_authtok = secret
>
>ldap_tls_reqcert = allow
>
>cache_credentials = true
>enumerate = false
>
>Our nsswitch.conf:
>
>passwd: compat sss
>group: compat sss
>shadow: compat
>
>hosts: files dns
>networks: files
>
>protocols: db files
>services: db files
>ethers: db files
>rpc: db files
>
>netgroup: nis sss
>sudoers: files sss
>
>$ grep sss /etc/pam.d/
>common-account:account [default=bad success=ok user_unknown=ignore]
>pam_sss.so
>common-auth:auth [success=2 default=ignore] pam_sss.so use_first_pass
>common-password:password sufficient pam_sss.so use_authtok
>common-session:session optional pam_sss.so
>
>Versions:
>
>DISTRIB_ID=Ubuntu
>DISTRIB_RELEASE=14.04
>DISTRIB_CODENAME=trusty
>DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"
>
>Linux 3.13.0-49-generic #83-Ubuntu SMP Fri Apr 10 20:11:33 UTC 2015 x86_64
>x86_64 x86_64 GNU/Linux
>
>$ dpkg -l | grep sssd
>ii sssd 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
metapackage
>ii sssd-ad 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- Active
>Directory back end
>ii sssd-ad-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
>PAC responder
>ii sssd-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
>common files
>ii sssd-ipa 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- IPA
>back end
>ii sssd-krb5 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
>Kerberos back end
>ii sssd-krb5-common 1.11.5-1ubuntu3 amd64 System Security Services Daemon
>-- Kerberos helpers
>ii sssd-ldap 1.11.5-1ubuntu3 amd64 System Security Services Daemon -- LDAP
>back end
>ii sssd-proxy 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
>proxy back end
>ii sssd-tools 1.11.5-1ubuntu3 amd64 System Security Services Daemon --
tools
It looks like a bug(s)
https://fedorahosted.org/sssd/ticket/2409
https://fedorahosted.org/sssd/ticket/2445
They are fixed in sssd >= 1.12.3
You can try to update from Timo's repository
https://launchpad.net/~sssd/+archive/ubuntu/updates
But it reminds me I wanted to prepare latest patches
for the las 1.11 release.
LS
_______________________________________________
sssd-users mailing list
sssd-users(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/sssd-users
3214
days inactive
3216
days old
sssd-users@lists.fedorahosted.org
2 comments
2 participants
participants (2)
-
Gerard . -
Lukas Slebodnik