sssd-ad on centos 7

[William Edsall]

Hello list,
 I've configured sssd on Centos 7 with the very basics. I'm able to id my
own user account, which was used to join the domain (via realm), but unable
to id any other account.
 Does anything make sense about this? I should mention this is a very large
 (50,000+) corporate AD.

Thanks
William

[Michal Židek]

On 08/30/2017 09:49 PM, William Edsall wrote:
>  Hello list,
>    I've configured sssd on Centos 7 with the very basics. I'm able to id 
>  my own user account, which was used to join the domain (via realm), but 
>  unable to id any other account.
>    Does anything make sense about this? I should mention this is a very 
>  large  (50,000+) corporate AD.
>  
>  Thanks
>  William
>  
>   
Hello,

please provide sssd domain and sssd_nss logs with debug_level = 9
as well as your SSSD configuration file.

For more details see:
https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html

Michal

[William Edsall]

Steps:
 left realm, joined realm as user1, added debug_level and ad_server to
sssd.conf (it seems to hang when it runs into a dead ad_server), restarted
nssd.

I ran an id on user1, it returned data. No data for user2.

I then cleared cache using: sss_cache -E, id'ed user1 again and data was
returned. Still no data for user2.


[sssd]
domains = example.com
default_domain_suffix = example.com
config_file_version = 2
services = nss, pam


[domain/example.com]
debug_level = 9
ad_domain = example.com
ad_server = EXAMPLE.EXAMPLE.COM
krb5_realm = EXAMPLE.COM
realmd_tags = manages-system joined-with-samba
cache_credentials = True
id_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = True
use_fully_qualified_names = True
fallback_homedir = /home/%u@%d
access_provider = ad


Logs:
sssd_nss is ~700 of the following lines:
(Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010): The
Data Provider returned an error
[org.freedesktop.sssd.Error.DataProvider.Offline]

sssd_example.com.log (attached).

On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com&gt; wrote:

>  On 08/30/2017 09:49 PM, William Edsall wrote:
>
> > Hello list,
> >   I've configured sssd on Centos 7 with the very basics. I'm able to id
> > my own user account, which was used to join the domain (via realm), but
> > unable to id any other account.
> >   Does anything make sense about this? I should mention this is a very
> > large  (50,000+) corporate AD.
> >
> > Thanks
> > William
> >
> >
> >
>  Hello,
>
>  please provide sssd domain and sssd_nss logs with debug_level = 9
>  as well as your SSSD configuration file.
>
>  For more details see:
>  https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
>
>  Michal
>  _______________________________________________
>  sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
>

[William Edsall]

Further testing I think user1 may have been cached all along. I was not
clearing cache while sssd was stopped.

After stopping, clearing, starting - id user1 hangs. I then add ad_server
and debug_level to sssd.conf and restart it. I can now id user1 but I
believe it's coming from cache.

On Thu, Aug 31, 2017 at 9:28 AM, William Edsall <wedsall(a)gmail.com&gt; wrote:

>  Steps:
>   left realm, joined realm as user1, added debug_level and ad_server to
>  sssd.conf (it seems to hang when it runs into a dead ad_server), restarted
>  nssd.
>
>  I ran an id on user1, it returned data. No data for user2.
>
>  I then cleared cache using: sss_cache -E, id'ed user1 again and data was
>  returned. Still no data for user2.
>
>
>  [sssd]
>  domains = example.com
>  default_domain_suffix = example.com
>  config_file_version = 2
>  services = nss, pam
>
>
>  [domain/example.com]
>  debug_level = 9
>  ad_domain = example.com
>  ad_server = EXAMPLE.EXAMPLE.COM
>  krb5_realm = EXAMPLE.COM
>  realmd_tags = manages-system joined-with-samba
>  cache_credentials = True
>  id_provider = ad
>  krb5_store_password_if_offline = True
>  default_shell = /bin/bash
>  ldap_id_mapping = True
>  use_fully_qualified_names = True
>  fallback_homedir = /home/%u@%d
>  access_provider = ad
>
>
>  Logs:
>  sssd_nss is ~700 of the following lines:
>  (Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010): The
>  Data Provider returned an error [org.freedesktop.sssd.Error.
>  DataProvider.Offline]
>
>  sssd_example.com.log (attached).
>
>  On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com&gt; wrote:
>
> > On 08/30/2017 09:49 PM, William Edsall wrote:
> >
> >> Hello list,
> >>   I've configured sssd on Centos 7 with the very basics. I'm able to id
> >> my own user account, which was used to join the domain (via realm), but
> >> unable to id any other account.
> >>   Does anything make sense about this? I should mention this is a very
> >> large  (50,000+) corporate AD.
> >>
> >> Thanks
> >> William
> >>
> >>
> >>
> > Hello,
> >
> > please provide sssd domain and sssd_nss logs with debug_level = 9
> > as well as your SSSD configuration file.
> >
> > For more details see:
> > https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
> >
> > Michal
> > _______________________________________________
> > sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >
>
>

[Michal Židek]

Hi,

I do not see the email where you sent the logs as attachment.
Maybe it is stuck in moderation (maybe the attachment was too big
or something). I only noticed you sent something thanks to your
last message. Can you please sent the logs directly to me?

If the logs are too big, It will help if you stop sssd, delete the logs, 
start sssd again and redo the test. This will keep the logs shorter.

Thanks,

Michal

On 08/31/2017 03:45 PM, William Edsall wrote:
>  Further testing I think user1 may have been cached all along. I was
> not 
>  clearing cache while sssd was stopped.
>  
>  After stopping, clearing, starting - id user1 hangs. I then add 
>  ad_server and debug_level to sssd.conf and restart it. I can now id 
>  user1 but I believe it's coming from cache.
>  
>  On Thu, Aug 31, 2017 at 9:28 AM, William Edsall <wedsall(a)gmail.com 
>  <mailto:[e-mail]>> wrote:
>  
>  
>      Steps:
>        left realm, joined realm as user1, added debug_level and ad_server
>      to sssd.conf (it seems to hang when it runs into a dead ad_server),
>      restarted nssd.
>  
>      I ran an id on user1, it returned data. No data for user2.
>  
>      I then cleared cache using: sss_cache -E, id'ed user1 again and data
>      was returned. Still no data for user2.
>  
>  
>      [sssd]
>      domains = example.com <http://example.com>
>      default_domain_suffix = example.com <http://example.com>
>      config_file_version = 2
>      services = nss, pam
>  
>  
>      [domain/example.com <http://example.com>]
>      debug_level = 9
>      ad_domain = example.com <http://example.com>
>      ad_server = EXAMPLE.EXAMPLE.COM <http://EXAMPLE.EXAMPLE.COM>
>      krb5_realm = EXAMPLE.COM <http://EXAMPLE.COM>
>      realmd_tags = manages-system joined-with-samba
>      cache_credentials = True
>      id_provider = ad
>      krb5_store_password_if_offline = True
>      default_shell = /bin/bash
>      ldap_id_mapping = True
>      use_fully_qualified_names = True
>      fallback_homedir = /home/%u@%d
>      access_provider = ad
>  
>  
>      Logs:
>      sssd_nss is ~700 of the following lines:
>      (Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010):
>      The Data Provider returned an error
>      [org.freedesktop.sssd.Error.DataProvider.Offline]
>  
>      sssd_example.com.log (attached).
>  
>      On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com
>      <mailto:[e-mail]>> wrote:
>  
>          On 08/30/2017 09:49 PM, William Edsall wrote:
>  
>              Hello list,
>                 I've configured sssd on Centos 7 with the very basics.
>              I'm able to id my own user account, which was used to join
>              the domain (via realm), but unable to id any other account.
>                 Does anything make sense about this? I should mention
>              this is a very large  (50,000+) corporate AD.
>  
>              Thanks
>              William
>  
>  
>  
>          Hello,
>  
>          please provide sssd domain and sssd_nss logs with debug_level = 9
>          as well as your SSSD configuration file.
>  
>          For more details see:
>          https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
>          <https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html>
>  
>          Michal
>          _______________________________________________
>          sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>          <mailto:[e-mail]>
>          To unsubscribe send an email to
>          sssd-users-leave(a)lists.fedorahosted.org
>          <mailto:[e-mail]>
>  
>  
>  
>  
>  
>  _______________________________________________
>  sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
>

[Michal Židek]

I forgot to say one think. You can sanitize the logs, but please
sent the whole logs (not just parts) for the sssd_domain and
sssd_nss logs (maybe you actually did this, but as I said,
I do not see the mail with the attachment).

Michal

On 08/31/2017 04:31 PM, Michal Židek wrote:
>  Hi,
>  
>  I do not see the email where you sent the logs as attachment.
>  Maybe it is stuck in moderation (maybe the attachment was too big
>  or something). I only noticed you sent something thanks to your
>  last message. Can you please sent the logs directly to me?
>  
>  If the logs are too big, It will help if you stop sssd, delete the logs, 
>  start sssd again and redo the test. This will keep the logs shorter.
>  
>  Thanks,
>  
>  Michal
>  
>  On 08/31/2017 03:45 PM, William Edsall wrote:
> > Further testing I think user1 may have been cached all along. I was 
> > not clearing cache while sssd was stopped.
> >
> > After stopping, clearing, starting - id user1 hangs. I then add 
> > ad_server and debug_level to sssd.conf and restart it. I can now id 
> > user1 but I believe it's coming from cache.
> >
> > On Thu, Aug 31, 2017 at 9:28 AM, William Edsall <wedsall(a)gmail.com 
> > <mailto:[e-mail]>> wrote:
> >
> >
> >     Steps:
> >       left realm, joined realm as user1, added debug_level and ad_server
> >     to sssd.conf (it seems to hang when it runs into a dead ad_server),
> >     restarted nssd.
> >
> >     I ran an id on user1, it returned data. No data for user2.
> >
> >     I then cleared cache using: sss_cache -E, id'ed user1 again and data
> >     was returned. Still no data for user2.
> >
> >
> >     [sssd]
> >     domains = example.com <http://example.com>
> >     default_domain_suffix = example.com <http://example.com>
> >     config_file_version = 2
> >     services = nss, pam
> >
> >
> >     [domain/example.com <http://example.com>]
> >     debug_level = 9
> >     ad_domain = example.com <http://example.com>
> >     ad_server = EXAMPLE.EXAMPLE.COM <http://EXAMPLE.EXAMPLE.COM>
> >     krb5_realm = EXAMPLE.COM <http://EXAMPLE.COM>
> >     realmd_tags = manages-system joined-with-samba
> >     cache_credentials = True
> >     id_provider = ad
> >     krb5_store_password_if_offline = True
> >     default_shell = /bin/bash
> >     ldap_id_mapping = True
> >     use_fully_qualified_names = True
> >     fallback_homedir = /home/%u@%d
> >     access_provider = ad
> >
> >
> >     Logs:
> >     sssd_nss is ~700 of the following lines:
> >     (Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010):
> >     The Data Provider returned an error
> >     [org.freedesktop.sssd.Error.DataProvider.Offline]
> >
> >     sssd_example.com.log (attached).
> >
> >     On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com
> >     <mailto:[e-mail]>> wrote:
> >
> >         On 08/30/2017 09:49 PM, William Edsall wrote:
> >
> >             Hello list,
> >                I've configured sssd on Centos 7 with the very basics.
> >             I'm able to id my own user account, which was used to join
> >             the domain (via realm), but unable to id any other account.
> >                Does anything make sense about this? I should mention
> >             this is a very large  (50,000+) corporate AD.
> >
> >             Thanks
> >             William
> >
> >
> >
> >         Hello,
> >
> >         please provide sssd domain and sssd_nss logs with debug_level = 9
> >         as well as your SSSD configuration file.
> >
> >         For more details see:
> >         https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
> >         <https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html>
> >
> >         Michal
> >         _______________________________________________
> >         sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >         <mailto:[e-mail]>
> >         To unsubscribe send an email to
> >         sssd-users-leave(a)lists.fedorahosted.org
> >         <mailto:[e-mail]>
> >
> >
> >
> >
> >
> > _______________________________________________
> > sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >
>  _______________________________________________
>  sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org

[Michal Židek]

I have the important part of logs now :)

Looking at the domain logs, this looks like a DNS or networking issue.
Are you sure you can resolve the EXAMPLE.EXAMPLE.COM from the machine
with SSSD?

(Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]] 
[fo_resolve_service_done] (0x0020): Failed to resolve server 
'EXAMPLE.EXAMPLE.COM': Domain name not found

(Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]] 
[set_server_common_status] (0x0100): Marking server 
'EXAMPLE.EXAMPLE.COM' as 'not working'

(Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]] 
[be_resolve_server_process] (0x0080): Couldn't resolve server 
(EXAMPLE.EXAMPLE.COM), resolver returned [11]: Resource temporarily 
unavailable

On 08/31/2017 04:38 PM, Michal Židek wrote:
>  I forgot to say one think. You can sanitize the logs, but please
>  sent the whole logs (not just parts) for the sssd_domain and
>  sssd_nss logs (maybe you actually did this, but as I said,
>  I do not see the mail with the attachment).
>  
>  Michal
>  
>  On 08/31/2017 04:31 PM, Michal Židek wrote:
> > Hi,
> >
> > I do not see the email where you sent the logs as attachment.
> > Maybe it is stuck in moderation (maybe the attachment was too big
> > or something). I only noticed you sent something thanks to your
> > last message. Can you please sent the logs directly to me?
> >
> > If the logs are too big, It will help if you stop sssd, delete the 
> > logs, start sssd again and redo the test. This will keep the logs 
> > shorter.
> >
> > Thanks,
> >
> > Michal
> >
> > On 08/31/2017 03:45 PM, William Edsall wrote:
> >> Further testing I think user1 may have been cached all along. I was 
> >> not clearing cache while sssd was stopped.
> >>
> >> After stopping, clearing, starting - id user1 hangs. I then add 
> >> ad_server and debug_level to sssd.conf and restart it. I can now id 
> >> user1 but I believe it's coming from cache.
> >>
> >> On Thu, Aug 31, 2017 at 9:28 AM, William Edsall <wedsall(a)gmail.com 
> >> <mailto:[e-mail]>> wrote:
> >>
> >>
> >>     Steps:
> >>       left realm, joined realm as user1, added debug_level and ad_server
> >>     to sssd.conf (it seems to hang when it runs into a dead ad_server),
> >>     restarted nssd.
> >>
> >>     I ran an id on user1, it returned data. No data for user2.
> >>
> >>     I then cleared cache using: sss_cache -E, id'ed user1 again and data
> >>     was returned. Still no data for user2.
> >>
> >>
> >>     [sssd]
> >>     domains = example.com <http://example.com>
> >>     default_domain_suffix = example.com <http://example.com>
> >>     config_file_version = 2
> >>     services = nss, pam
> >>
> >>
> >>     [domain/example.com <http://example.com>]
> >>     debug_level = 9
> >>     ad_domain = example.com <http://example.com>
> >>     ad_server = EXAMPLE.EXAMPLE.COM <http://EXAMPLE.EXAMPLE.COM>
> >>     krb5_realm = EXAMPLE.COM <http://EXAMPLE.COM>
> >>     realmd_tags = manages-system joined-with-samba
> >>     cache_credentials = True
> >>     id_provider = ad
> >>     krb5_store_password_if_offline = True
> >>     default_shell = /bin/bash
> >>     ldap_id_mapping = True
> >>     use_fully_qualified_names = True
> >>     fallback_homedir = /home/%u@%d
> >>     access_provider = ad
> >>
> >>
> >>     Logs:
> >>     sssd_nss is ~700 of the following lines:
> >>     (Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010):
> >>     The Data Provider returned an error
> >>     [org.freedesktop.sssd.Error.DataProvider.Offline]
> >>
> >>     sssd_example.com.log (attached).
> >>
> >>     On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com
> >>     <mailto:[e-mail]>> wrote:
> >>
> >>         On 08/30/2017 09:49 PM, William Edsall wrote:
> >>
> >>             Hello list,
> >>                I've configured sssd on Centos 7 with the very basics.
> >>             I'm able to id my own user account, which was used to join
> >>             the domain (via realm), but unable to id any other account.
> >>                Does anything make sense about this? I should mention
> >>             this is a very large  (50,000+) corporate AD.
> >>
> >>             Thanks
> >>             William
> >>
> >>
> >>
> >>         Hello,
> >>
> >>         please provide sssd domain and sssd_nss logs with debug_level 
> >> = 9
> >>         as well as your SSSD configuration file.
> >>
> >>         For more details see:
> >>         https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
> >>         <https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html>
> >>
> >>         Michal
> >>         _______________________________________________
> >>         sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >>         <mailto:[e-mail]>
> >>         To unsubscribe send an email to
> >>         sssd-users-leave(a)lists.fedorahosted.org
> >>         <mailto:[e-mail]>
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >> To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >>
> > _______________________________________________
> > sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
>  _______________________________________________
>  sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org

[William Edsall]

Replied directly with updated logs. There was a DNS issue caused by myself
during a frenzy of testing.

On Thu, Aug 31, 2017 at 10:47 AM, Michal Židek <mzidek(a)redhat.com&gt; wrote:

>  I have the important part of logs now :)
>
>  Looking at the domain logs, this looks like a DNS or networking issue.
>  Are you sure you can resolve the EXAMPLE.EXAMPLE.COM from the machine
>  with SSSD?
>
>  (Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]]
>  [fo_resolve_service_done] (0x0020): Failed to resolve server '
>  EXAMPLE.EXAMPLE.COM': Domain name not found
>
>  (Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]]
>  [set_server_common_status] (0x0100): Marking server 'EXAMPLE.EXAMPLE.COM'
>  as 'not working'
>
>  (Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]]
>  [be_resolve_server_process] (0x0080): Couldn't resolve server (
>  EXAMPLE.EXAMPLE.COM), resolver returned [11]: Resource temporarily
>  unavailable
>
>
>  On 08/31/2017 04:38 PM, Michal Židek wrote:
>
> > I forgot to say one think. You can sanitize the logs, but please
> > sent the whole logs (not just parts) for the sssd_domain and
> > sssd_nss logs (maybe you actually did this, but as I said,
> > I do not see the mail with the attachment).
> >
> > Michal
> >
> > On 08/31/2017 04:31 PM, Michal Židek wrote:
> >
> >> Hi,
> >>
> >> I do not see the email where you sent the logs as attachment.
> >> Maybe it is stuck in moderation (maybe the attachment was too big
> >> or something). I only noticed you sent something thanks to your
> >> last message. Can you please sent the logs directly to me?
> >>
> >> If the logs are too big, It will help if you stop sssd, delete the logs,
> >> start sssd again and redo the test. This will keep the logs shorter.
> >>
> >> Thanks,
> >>
> >> Michal
> >>
> >> On 08/31/2017 03:45 PM, William Edsall wrote:
> >>
> >>> Further testing I think user1 may have been cached all along. I was not
> >>> clearing cache while sssd was stopped.
> >>>
> >>> After stopping, clearing, starting - id user1 hangs. I then add
> >>> ad_server and debug_level to sssd.conf and restart it. I can now id user1
> >>> but I believe it's coming from cache.
> >>>
> >>> On Thu, Aug 31, 2017 at 9:28 AM, William Edsall <wedsall(a)gmail.com
> >>> <mailto:[e-mail]>> wrote:
> >>>
> >>>
> >>>     Steps:
> >>>       left realm, joined realm as user1, added debug_level and ad_server
> >>>     to sssd.conf (it seems to hang when it runs into a dead ad_server),
> >>>     restarted nssd.
> >>>
> >>>     I ran an id on user1, it returned data. No data for user2.
> >>>
> >>>     I then cleared cache using: sss_cache -E, id'ed user1 again and data
> >>>     was returned. Still no data for user2.
> >>>
> >>>
> >>>     [sssd]
> >>>     domains = example.com <http://example.com>
> >>>     default_domain_suffix = example.com <http://example.com>
> >>>     config_file_version = 2
> >>>     services = nss, pam
> >>>
> >>>
> >>>     [domain/example.com <http://example.com>]
> >>>     debug_level = 9
> >>>     ad_domain = example.com <http://example.com>
> >>>     ad_server = EXAMPLE.EXAMPLE.COM <http://EXAMPLE.EXAMPLE.COM>
> >>>     krb5_realm = EXAMPLE.COM <http://EXAMPLE.COM>
> >>>     realmd_tags = manages-system joined-with-samba
> >>>     cache_credentials = True
> >>>     id_provider = ad
> >>>     krb5_store_password_if_offline = True
> >>>     default_shell = /bin/bash
> >>>     ldap_id_mapping = True
> >>>     use_fully_qualified_names = True
> >>>     fallback_homedir = /home/%u@%d
> >>>     access_provider = ad
> >>>
> >>>
> >>>     Logs:
> >>>     sssd_nss is ~700 of the following lines:
> >>>     (Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010):
> >>>     The Data Provider returned an error
> >>>     [org.freedesktop.sssd.Error.DataProvider.Offline]
> >>>
> >>>     sssd_example.com.log (attached).
> >>>
> >>>     On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com
> >>>     <mailto:[e-mail]>> wrote:
> >>>
> >>>         On 08/30/2017 09:49 PM, William Edsall wrote:
> >>>
> >>>             Hello list,
> >>>                I've configured sssd on Centos 7 with the very basics.
> >>>             I'm able to id my own user account, which was used to join
> >>>             the domain (via realm), but unable to id any other account.
> >>>                Does anything make sense about this? I should mention
> >>>             this is a very large  (50,000+) corporate AD.
> >>>
> >>>             Thanks
> >>>             William
> >>>
> >>>
> >>>
> >>>         Hello,
> >>>
> >>>         please provide sssd domain and sssd_nss logs with debug_level =
> >>> 9
> >>>         as well as your SSSD configuration file.
> >>>
> >>>         For more details see:
> >>>         https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
> >>>         <https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html>
> >>>
> >>>         Michal
> >>>         _______________________________________________
> >>>         sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >>>         <mailto:[e-mail]>
> >>>         To unsubscribe send an email to
> >>>         sssd-users-leave(a)lists.fedorahosted.org
> >>>         <mailto:[e-mail]>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >>> To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >>>
> >>> _______________________________________________
> >> sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >> To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >>
> > _______________________________________________
> > sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >
>  _______________________________________________
>  sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
>

[William Edsall]

Had a few communications with Michal but we're still stuck.

One issue is that we have dozens of domain controllers globally. A standard
dns lookup could give me a domain controller overseas which will be slow,
or maybe even a domain controller that isn't responding. As such, I have
been inserting ad_server = x into the sssd.conf to improve performance.

I noticed that if I do not insert ad_server = x, I'm getting different
results. My initial id request is very slow but seems to produce results.
While searching, it seems to also be 'inserting' users into the users hash
table - almost as if it's searching and inserting our entire user database?
For example there are countless lines of the following:
(Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
[sdap_nested_group_hash_insert] (0x4000): Inserting
[CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table [users]

As my initial id request returns, it seems to return several chunks of my
group ids at once as if it's processing them individually and searching all
users in that group (thus the above log entries).

Not sure if this helps or just muds up the issue but it's strange indeed.

On Thu, Aug 31, 2017 at 10:47 AM, Michal Židek <mzidek(a)redhat.com&gt; wrote:

>  I have the important part of logs now :)
>
>  Looking at the domain logs, this looks like a DNS or networking issue.
>  Are you sure you can resolve the EXAMPLE.EXAMPLE.COM from the machine
>  with SSSD?
>
>  (Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]]
>  [fo_resolve_service_done] (0x0020): Failed to resolve server '
>  EXAMPLE.EXAMPLE.COM': Domain name not found
>
>  (Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]]
>  [set_server_common_status] (0x0100): Marking server 'EXAMPLE.EXAMPLE.COM'
>  as 'not working'
>
>  (Thu Aug 31 09:16:17 2017) [sssd[be[example.com]]]
>  [be_resolve_server_process] (0x0080): Couldn't resolve server (
>  EXAMPLE.EXAMPLE.COM), resolver returned [11]: Resource temporarily
>  unavailable
>
>
>  On 08/31/2017 04:38 PM, Michal Židek wrote:
>
> > I forgot to say one think. You can sanitize the logs, but please
> > sent the whole logs (not just parts) for the sssd_domain and
> > sssd_nss logs (maybe you actually did this, but as I said,
> > I do not see the mail with the attachment).
> >
> > Michal
> >
> > On 08/31/2017 04:31 PM, Michal Židek wrote:
> >
> >> Hi,
> >>
> >> I do not see the email where you sent the logs as attachment.
> >> Maybe it is stuck in moderation (maybe the attachment was too big
> >> or something). I only noticed you sent something thanks to your
> >> last message. Can you please sent the logs directly to me?
> >>
> >> If the logs are too big, It will help if you stop sssd, delete the logs,
> >> start sssd again and redo the test. This will keep the logs shorter.
> >>
> >> Thanks,
> >>
> >> Michal
> >>
> >> On 08/31/2017 03:45 PM, William Edsall wrote:
> >>
> >>> Further testing I think user1 may have been cached all along. I was not
> >>> clearing cache while sssd was stopped.
> >>>
> >>> After stopping, clearing, starting - id user1 hangs. I then add
> >>> ad_server and debug_level to sssd.conf and restart it. I can now id user1
> >>> but I believe it's coming from cache.
> >>>
> >>> On Thu, Aug 31, 2017 at 9:28 AM, William Edsall <wedsall(a)gmail.com
> >>> <mailto:[e-mail]>> wrote:
> >>>
> >>>
> >>>     Steps:
> >>>       left realm, joined realm as user1, added debug_level and ad_server
> >>>     to sssd.conf (it seems to hang when it runs into a dead ad_server),
> >>>     restarted nssd.
> >>>
> >>>     I ran an id on user1, it returned data. No data for user2.
> >>>
> >>>     I then cleared cache using: sss_cache -E, id'ed user1 again and data
> >>>     was returned. Still no data for user2.
> >>>
> >>>
> >>>     [sssd]
> >>>     domains = example.com <http://example.com>
> >>>     default_domain_suffix = example.com <http://example.com>
> >>>     config_file_version = 2
> >>>     services = nss, pam
> >>>
> >>>
> >>>     [domain/example.com <http://example.com>]
> >>>     debug_level = 9
> >>>     ad_domain = example.com <http://example.com>
> >>>     ad_server = EXAMPLE.EXAMPLE.COM <http://EXAMPLE.EXAMPLE.COM>
> >>>     krb5_realm = EXAMPLE.COM <http://EXAMPLE.COM>
> >>>     realmd_tags = manages-system joined-with-samba
> >>>     cache_credentials = True
> >>>     id_provider = ad
> >>>     krb5_store_password_if_offline = True
> >>>     default_shell = /bin/bash
> >>>     ldap_id_mapping = True
> >>>     use_fully_qualified_names = True
> >>>     fallback_homedir = /home/%u@%d
> >>>     access_provider = ad
> >>>
> >>>
> >>>     Logs:
> >>>     sssd_nss is ~700 of the following lines:
> >>>     (Thu Aug 31 09:21:05 2017) [sssd[nss]] [sss_dp_get_reply] (0x0010):
> >>>     The Data Provider returned an error
> >>>     [org.freedesktop.sssd.Error.DataProvider.Offline]
> >>>
> >>>     sssd_example.com.log (attached).
> >>>
> >>>     On Thu, Aug 31, 2017 at 7:44 AM, Michal Židek <mzidek(a)redhat.com
> >>>     <mailto:[e-mail]>> wrote:
> >>>
> >>>         On 08/30/2017 09:49 PM, William Edsall wrote:
> >>>
> >>>             Hello list,
> >>>                I've configured sssd on Centos 7 with the very basics.
> >>>             I'm able to id my own user account, which was used to join
> >>>             the domain (via realm), but unable to id any other account.
> >>>                Does anything make sense about this? I should mention
> >>>             this is a very large  (50,000+) corporate AD.
> >>>
> >>>             Thanks
> >>>             William
> >>>
> >>>
> >>>
> >>>         Hello,
> >>>
> >>>         please provide sssd domain and sssd_nss logs with debug_level =
> >>> 9
> >>>         as well as your SSSD configuration file.
> >>>
> >>>         For more details see:
> >>>         https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
> >>>         <https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html>
> >>>
> >>>         Michal
> >>>         _______________________________________________
> >>>         sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >>>         <mailto:[e-mail]>
> >>>         To unsubscribe send an email to
> >>>         sssd-users-leave(a)lists.fedorahosted.org
> >>>         <mailto:[e-mail]>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >>> To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >>>
> >>> _______________________________________________
> >> sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> >> To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >>
> > _______________________________________________
> > sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
> >
>  _______________________________________________
>  sssd-users mailing list -- sssd-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to sssd-users-leave(a)lists.fedorahosted.org
>

[Lukas Slebodnik]

On (01/09/17 09:33), William Edsall wrote:
> Had a few communications with Michal but we're still stuck.
>
> One issue is that we have dozens of domain controllers globally. A standard
> dns lookup could give me a domain controller overseas which will be slow,
> or maybe even a domain controller that isn't responding. As such, I have
> been inserting ad_server = x into the sssd.conf to improve performance.
>
> I noticed that if I do not insert ad_server = x, I'm getting different
> results. My initial id request is very slow but seems to produce results.
> While searching, it seems to also be 'inserting' users into the users hash
> table - almost as if it's searching and inserting our entire user database?
> For example there are countless lines of the following:
> (Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
> [sdap_nested_group_hash_insert] (0x4000): Inserting
> [CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table [users]
>
> As my initial id request returns, it seems to return several chunks of my
> group ids at once as if it's processing them individually and searching all
> users in that group (thus the above log entries).
>
> Not sure if this helps or just muds up the issue but it's strange indeed.
>  
You needn't hardcode ad_server. You can still rely on dns discovery.
I assume you use sites in AD. So you can "pin" sssd to your local/nearest site
with option ad_site.

More details in man sssd-ad -> ad_site

LS

[John Beranek]

On 1 September 2017 at 15:54, Lukas Slebodnik <lslebodn(a)redhat.com&gt; wrote:
>  On (01/09/17 09:33), William Edsall wrote:
>  >Had a few communications with Michal but we're still stuck.
>  >
>  >One issue is that we have dozens of domain controllers globally. A standard
>  >dns lookup could give me a domain controller overseas which will be slow,
>  >or maybe even a domain controller that isn't responding. As such, I have
>  >been inserting ad_server = x into the sssd.conf to improve performance.
>  >
>  >I noticed that if I do not insert ad_server = x, I'm getting different
>  >results. My initial id request is very slow but seems to produce results.
>  >While searching, it seems to also be 'inserting' users into the users hash
>  >table - almost as if it's searching and inserting our entire user database?
>  >For example there are countless lines of the following:
>  >(Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
>  >[sdap_nested_group_hash_insert] (0x4000): Inserting
>  >[CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table [users]
>  >
>  >As my initial id request returns, it seems to return several chunks of my
>  >group ids at once as if it's processing them individually and searching all
>  >users in that group (thus the above log entries).
>  >
>  >Not sure if this helps or just muds up the issue but it's strange indeed.
>  >
>  You needn't hardcode ad_server. You can still rely on dns discovery.
>  I assume you use sites in AD. So you can "pin" sssd to your local/nearest site
>  with option ad_site. 
I've got something to add to this, some behaviour we're seeing with
CentOS 7 servers using sssd-ad.

sssd-1.14.0-43.el7_3.18.x86_64
sssd-ad-1.14.0-43.el7_3.18.x86_64
sssd-client-1.14.0-43.el7_3.18.x86_64
sssd-common-1.14.0-43.el7_3.18.x86_64
sssd-common-pac-1.14.0-43.el7_3.18.x86_64
sssd-ipa-1.14.0-43.el7_3.18.x86_64
sssd-krb5-1.14.0-43.el7_3.18.x86_64
sssd-krb5-common-1.14.0-43.el7_3.18.x86_64
sssd-ldap-1.14.0-43.el7_3.18.x86_64
sssd-proxy-1.14.0-43.el7_3.18.x86_64

In our case we have some DCs which are located at a partner site, and
are therefore inaccessible to clients on our standard LANs.

When SSSD starts it will correctly determine there are 2 primary DCs
(these are the ones for the site) and 7 backup DCs.

However, what is happening from time to time is that for some reason
I've not yet determined the connection(s) to the primary DC(s) are
dropping, and then sssd attempts to connect to one of the DCs that are
inaccessible.

In what circumstances would sssd prefer a backup server to a primary server?

I've got a chunk of log which I've anonymised:

https://paste.fedoraproject.org/paste/G69lC9COQfbnWFI~qtFLZw

Here the server is PAL221 in site "London" and the local DCs are dc03
and dc04. dc06 is in the site which cannot be accessed.

I have now turned debug level up to 9 for this domain config, and
could provide the unmodified log privately if required.

Cheers,

John

-- 
John Beranek                         To generalise is to be an idiot.
http://redux.org.uk/                                 -- William Blake

[John Beranek]

On 11 September 2017 at 12:23, John Beranek <john(a)redux.org.uk&gt; wrote:
>  On 1 September 2017 at 15:54, Lukas Slebodnik
> <lslebodn(a)redhat.com&gt; wrote:
> >
> > On (01/09/17 09:33), William Edsall wrote:
> > >Had a few communications with Michal but we're still stuck.
> > >
> > >One issue is that we have dozens of domain controllers globally. A standard
> > >dns lookup could give me a domain controller overseas which will be slow,
> > >or maybe even a domain controller that isn't responding. As such, I have
> > >been inserting ad_server = x into the sssd.conf to improve performance.
> > >
> > >I noticed that if I do not insert ad_server = x, I'm getting different
> > >results. My initial id request is very slow but seems to produce results.
> > >While searching, it seems to also be 'inserting' users into the users
> hash
> > >table - almost as if it's searching and inserting our entire user database?
> > >For example there are countless lines of the following:
> > >(Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
> > >[sdap_nested_group_hash_insert] (0x4000): Inserting
> > >[CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table [users]
> > >
> > >As my initial id request returns, it seems to return several chunks of my
> > >group ids at once as if it's processing them individually and searching all
> > >users in that group (thus the above log entries).
> > >
> > >Not sure if this helps or just muds up the issue but it's strange indeed.
> > >
> > You needn't hardcode ad_server. You can still rely on dns discovery.
> > I assume you use sites in AD. So you can "pin" sssd to your local/nearest
> site
> > with option ad_site.
>
>  I've got something to add to this, some behaviour we're seeing with
>  CentOS 7 servers using sssd-ad. 
Looking in logs for where it decided to connect to a backup DC, the
best I can find is the following sort of errors (or at least things
that look like errors) from sysdb lookups, followed by the new LDAP
connection to the backup DC:

(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sdap_save_groups]
(0x0040): Failed to store group 1 members.
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]]
[sysdb_set_cache_entry_attr] (0x0080): ldb_modify failed: [No such
object](32)[ldb_wait: No such object (32)]
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sysdb_set_entry_attr]
(0x0080): Cannot set ts attrs for
name=DL_RBA_SMBUsersFolders-ReadPerms@ad,cn=groups,cn=EXAMPLE,cn=sysdb
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]]
[sysdb_set_cache_entry_attr] (0x0080): ldb_modify failed: [No such
object](32)[ldb_wait: No such object (32)]
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sysdb_set_entry_attr]
(0x0080): Cannot set ts attrs for
name=GG_UserAccountProvisioningAdmins@ad,cn=groups,cn=EXAMPLE,cn=sysdb
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]]
[sysdb_set_cache_entry_attr] (0x0080): ldb_modify failed: [No such
object](32)[ldb_wait: No such object (32)]
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sysdb_set_entry_attr]
(0x0080): Cannot set ts attrs for
name=ITSupport@ad,cn=groups,cn=EXAMPLE,cn=sysdb
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]]
[sysdb_set_cache_entry_attr] (0x0080): ldb_modify failed: [No such
object](32)[ldb_wait: No such object (32)]
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sysdb_set_entry_attr]
(0x0080): Cannot set ts attrs for
name=DL_RBA_SMBUsersFolders-ReadPerms@ad,cn=groups,cn=EXAMPLE,cn=sysdb
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]]
[sysdb_set_cache_entry_attr] (0x0080): ldb_modify failed: [No such
object](32)[ldb_wait: No such object (32)]
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sysdb_set_entry_attr]
(0x0080): Cannot set ts attrs for
name=GG_UserAccountProvisioningAdmins@ad,cn=groups,cn=EXAMPLE,cn=sysdb
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]]
[sysdb_set_cache_entry_attr] (0x0080): ldb_modify failed: [No such
object](32)[ldb_wait: No such object (32)]
(Mon Sep 11 04:24:52 2017) [sssd[be[EXAMPLE]]] [sysdb_set_entry_attr]
(0x0080): Cannot set ts attrs for
name=ITSupport@ad,cn=groups,cn=EXAMPLE,cn=sysdb
(Mon Sep 11 04:37:09 2017) [sssd[be[EXAMPLE]]]
[check_if_pac_is_available] (0x0040): find_user_entry failed.
(Mon Sep 11 04:37:09 2017) [sssd[be[EXAMPLE]]]
[fo_resolve_service_send] (0x0100): Trying to resolve service
'EXAMPLE_GC'
(Mon Sep 11 04:37:09 2017) [sssd[be[EXAMPLE]]] [resolv_getsrv_send]
(0x0100): Trying to resolve SRV record of '_ldap._tcp.example.com'
(Mon Sep 11 04:37:09 2017) [sssd[be[EXAMPLE]]]
[resolv_gethostbyname_files_send] (0x0100): Trying to resolve A record
of 'dc07.example.com' in files
(Mon Sep 11 04:37:09 2017) [sssd[be[EXAMPLE]]]
[resolv_gethostbyname_files_send] (0x0100): Trying to resolve AAAA
record of 'dc07.example.com' in files
(Mon Sep 11 04:37:09 2017) [sssd[be[EXAMPLE]]]
[resolv_gethostbyname_dns_query] (0x0100): Trying to resolve A record
of 'dc07.example.com' in DNS
(Mon Sep 11 04:37:15 2017) [sssd[be[EXAMPLE]]]
[fo_resolve_service_timeout] (0x0080): Service resolving timeout
reached
(Mon Sep 11 04:37:26 2017) [sssd[be[EXAMPLE]]]
[sdap_id_conn_data_expire_handler] (0x0080): connection is about to
expire, releasing it
(Mon Sep 11 04:53:16 2017) [sssd[be[EXAMPLE]]]
[fo_resolve_service_send] (0x0100): Trying to resolve service
'EXAMPLE'
(Mon Sep 11 04:53:16 2017) [sssd[be[EXAMPLE]]] [get_server_status]
(0x0100): Hostname resolution expired, resetting the server status of
'dc01.example.com'
(Mon Sep 11 04:53:16 2017) [sssd[be[EXAMPLE]]]
[set_server_common_status] (0x0100): Marking server 'dc01.example.com'
as 'name not resolved'
(Mon Sep 11 04:53:16 2017) [sssd[be[EXAMPLE]]] [collapse_srv_lookup]
(0x0100): Need to refresh SRV lookup for domain
Howden._sites.example.com

John

-- 
John Beranek                         To generalise is to be an idiot.
http://redux.org.uk/                                 -- William Blake

[Jakub Hrozek]

On Mon, Sep 11, 2017 at 12:23:26PM +0100, John Beranek wrote:
>  On 1 September 2017 at 15:54, Lukas Slebodnik
> <lslebodn(a)redhat.com&gt; wrote:
>  >
>  > On (01/09/17 09:33), William Edsall wrote:
>  > >Had a few communications with Michal but we're still stuck.
>  > >
>  > >One issue is that we have dozens of domain controllers globally. A standard
>  > >dns lookup could give me a domain controller overseas which will be slow,
>  > >or maybe even a domain controller that isn't responding. As such, I have
>  > >been inserting ad_server = x into the sssd.conf to improve performance.
>  > >
>  > >I noticed that if I do not insert ad_server = x, I'm getting different
>  > >results. My initial id request is very slow but seems to produce results.
>  > >While searching, it seems to also be 'inserting' users into the users
> hash
>  > >table - almost as if it's searching and inserting our entire user database?
>  > >For example there are countless lines of the following:
>  > >(Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
>  > >[sdap_nested_group_hash_insert] (0x4000): Inserting
>  > >[CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table [users]
>  > >
>  > >As my initial id request returns, it seems to return several chunks of my
>  > >group ids at once as if it's processing them individually and searching all
>  > >users in that group (thus the above log entries).
>  > >
>  > >Not sure if this helps or just muds up the issue but it's strange indeed.
>  > >
>  > You needn't hardcode ad_server. You can still rely on dns discovery.
>  > I assume you use sites in AD. So you can "pin" sssd to your local/nearest
> site
>  > with option ad_site.
>  
>  I've got something to add to this, some behaviour we're seeing with
>  CentOS 7 servers using sssd-ad.
>  
>  sssd-1.14.0-43.el7_3.18.x86_64
>  sssd-ad-1.14.0-43.el7_3.18.x86_64
>  sssd-client-1.14.0-43.el7_3.18.x86_64
>  sssd-common-1.14.0-43.el7_3.18.x86_64
>  sssd-common-pac-1.14.0-43.el7_3.18.x86_64
>  sssd-ipa-1.14.0-43.el7_3.18.x86_64
>  sssd-krb5-1.14.0-43.el7_3.18.x86_64
>  sssd-krb5-common-1.14.0-43.el7_3.18.x86_64
>  sssd-ldap-1.14.0-43.el7_3.18.x86_64
>  sssd-proxy-1.14.0-43.el7_3.18.x86_64
>  
>  In our case we have some DCs which are located at a partner site, and
>  are therefore inaccessible to clients on our standard LANs.
>  
>  When SSSD starts it will correctly determine there are 2 primary DCs
>  (these are the ones for the site) and 7 backup DCs.
>  
>  However, what is happening from time to time is that for some reason
>  I've not yet determined the connection(s) to the primary DC(s) are
>  dropping, and then sssd attempts to connect to one of the DCs that are
>  inaccessible.
>  
>  In what circumstances would sssd prefer a backup server to a primary server?
>  
>  I've got a chunk of log which I've anonymised:
>  
>  https://paste.fedoraproject.org/paste/G69lC9COQfbnWFI~qtFLZw 
The issue is here:
(Mon Sep 11 11:16:15 2017) [sssd[be[EXAMPLE]]]
[sdap_id_conn_data_expire_handler] (0x0080): connection is about to
expire, releasing it
(Mon Sep 11 11:16:15 2017) [sssd[be[EXAMPLE]]]
[sdap_id_conn_data_expire_handler] (0x0080): connection is about to
expire, releasing it
(Mon Sep 11 11:17:50 2017) [sssd[be[EXAMPLE]]] [fo_resolve_service_send]
(0x0100): Trying to resolve service 'EXAMPLE'
(Mon Sep 11 11:17:50 2017) [sssd[be[EXAMPLE]]] [collapse_srv_lookup]
(0x0100): Need to refresh SRV lookup for domain
London._sites.example.com
(Mon Sep 11 11:17:50 2017) [sssd[be[EXAMPLE]]] [resolv_getsrv_send]
(0x0100): Trying to resolve SRV record of '_ldap._tcp.example.com'
(Mon Sep 11 11:17:50 2017) [sssd[be[EXAMPLE]]]
[resolv_gethostbyname_files_send] (0x0100): Trying to resolve A record
of 'dc07.example.com' in files
(Mon Sep 11 11:17:50 2017) [sssd[be[EXAMPLE]]]
[resolv_gethostbyname_files_send] (0x0100): Trying to resolve AAAA
record of 'dc07.example.com' in files
(Mon Sep 11 11:17:50 2017) [sssd[be[EXAMPLE]]]
[resolv_gethostbyname_dns_query] (0x0100): Trying to resolve A record of
'dc07.example.com' in DNS
(Mon Sep 11 11:17:56 2017) [sssd[be[EXAMPLE]]]
[fo_resolve_service_timeout] (0x0080): Service resolving timeout reached
(Mon Sep 11 11:17:56 2017) [sssd[be[EXAMPLE]]] [sdap_id_op_connect_done]
(0x0020): Failed to connect, going offline (5 [Input/output error])
(Mon Sep 11 11:17:56 2017) [sssd[be[EXAMPLE]]] [be_run_offline_cb]
(0x0080): Going offline. Running callbacks.

So this is a known bug where locating the site (even though we know it
already) can contact the DCs outside that site.

In the meantime, until the fix is released, you can hardcode the site
using the 'ad_site' option.

[John Beranek]

On 11 September 2017 at 14:28, Jakub Hrozek wrote:
>  On Mon, Sep 11, 2017 at 12:23:26PM +0100, John Beranek wrote:
> > On 1 September 2017 at 15:54, Lukas Slebodnik wrote:
> > >
> > > On (01/09/17 09:33), William Edsall wrote:
> > > >Had a few communications with Michal but we're still stuck.
> > > >
> > > >One issue is that we have dozens of domain controllers globally. A standard
> > > >dns lookup could give me a domain controller overseas which will be slow,
> > > >or maybe even a domain controller that isn't responding. As such, I
> have
> > > >been inserting ad_server = x into the sssd.conf to improve performance.
> > > >
> > > >I noticed that if I do not insert ad_server = x, I'm getting different
> > > >results. My initial id request is very slow but seems to produce results.
> > > >While searching, it seems to also be 'inserting' users into the
> users hash
> > > >table - almost as if it's searching and inserting our entire user
> database?
> > > >For example there are countless lines of the following:
> > > >(Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
> > > >[sdap_nested_group_hash_insert] (0x4000): Inserting
> > > >[CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table [users]
> > > >
> > > >As my initial id request returns, it seems to return several chunks of my
> > > >group ids at once as if it's processing them individually and searching
> all
> > > >users in that group (thus the above log entries).
> > > >
> > > >Not sure if this helps or just muds up the issue but it's strange
> indeed.
> > > >
> > > You needn't hardcode ad_server. You can still rely on dns discovery.
> > > I assume you use sites in AD. So you can "pin" sssd to your
> local/nearest site
> > > with option ad_site.
> >
> > I've got something to add to this, some behaviour we're seeing with
> > CentOS 7 servers using sssd-ad.
> >
> > sssd-1.14.0-43.el7_3.18.x86_64
> > sssd-ad-1.14.0-43.el7_3.18.x86_64
> > sssd-client-1.14.0-43.el7_3.18.x86_64
> > sssd-common-1.14.0-43.el7_3.18.x86_64
> > sssd-common-pac-1.14.0-43.el7_3.18.x86_64
> > sssd-ipa-1.14.0-43.el7_3.18.x86_64
> > sssd-krb5-1.14.0-43.el7_3.18.x86_64
> > sssd-krb5-common-1.14.0-43.el7_3.18.x86_64
> > sssd-ldap-1.14.0-43.el7_3.18.x86_64
> > sssd-proxy-1.14.0-43.el7_3.18.x86_64
> >
> > In our case we have some DCs which are located at a partner site, and
> > are therefore inaccessible to clients on our standard LANs.
> >
> > When SSSD starts it will correctly determine there are 2 primary DCs
> > (these are the ones for the site) and 7 backup DCs.
> >
> > However, what is happening from time to time is that for some reason
> > I've not yet determined the connection(s) to the primary DC(s) are
> > dropping, and then sssd attempts to connect to one of the DCs that are
> > inaccessible.
> >
> > In what circumstances would sssd prefer a backup server to a primary server?
> >
> > I've got a chunk of log which I've anonymised:
> >
> > https://paste.fedoraproject.org/paste/G69lC9COQfbnWFI~qtFLZw
>
>  The issue is here: 
[snip]
>  So this is a known bug where locating the site (even though we know it
>  already) can contact the DCs outside that site.
>
>  In the meantime, until the fix is released, you can hardcode the site
>  using the 'ad_site' option. 
Thank you Jakub, is there a ticket/BZ for this? I noticed after my
post that it's evident on CentOS 6 too.

Cheers,

John

-- 
John Beranek                         To generalise is to be an idiot.
http://redux.org.uk/                                 -- William Blake

[John Beranek]

On 12 September 2017 at 17:59, John Beranek wrote:
>  On 11 September 2017 at 14:28, Jakub Hrozek wrote:
> > On Mon, Sep 11, 2017 at 12:23:26PM +0100, John Beranek wrote:
> >> On 1 September 2017 at 15:54, Lukas Slebodnik wrote:
> >> >
> >> > On (01/09/17 09:33), William Edsall wrote:
> >> > >Had a few communications with Michal but we're still stuck.
> >> > >
> >> > >One issue is that we have dozens of domain controllers globally. A
> standard
> >> > >dns lookup could give me a domain controller overseas which will be
> slow,
> >> > >or maybe even a domain controller that isn't responding. As such, I
> have
> >> > >been inserting ad_server = x into the sssd.conf to improve performance.
> >> > >
> >> > >I noticed that if I do not insert ad_server = x, I'm getting
> different
> >> > >results. My initial id request is very slow but seems to produce
> results.
> >> > >While searching, it seems to also be 'inserting' users into the
> users hash
> >> > >table - almost as if it's searching and inserting our entire user
> database?
> >> > >For example there are countless lines of the following:
> >> > >(Fri Sep  1 09:28:37 2017) [sssd[be[example.com]]]
> >> > >[sdap_nested_group_hash_insert] (0x4000): Inserting
> >> > >[CN=user_name,OU=bla,OU=bla Users,DC=dow,DC=com] into hash table
> [users]
> >> > >
> >> > >As my initial id request returns, it seems to return several chunks of
> my
> >> > >group ids at once as if it's processing them individually and
> searching all
> >> > >users in that group (thus the above log entries).
> >> > >
> >> > >Not sure if this helps or just muds up the issue but it's strange
> indeed.
> >> > >
> >> > You needn't hardcode ad_server. You can still rely on dns discovery.
> >> > I assume you use sites in AD. So you can "pin" sssd to your
> local/nearest site
> >> > with option ad_site.
> >>
> >> I've got something to add to this, some behaviour we're seeing with
> >> CentOS 7 servers using sssd-ad.
> >>
> >> sssd-1.14.0-43.el7_3.18.x86_64
> >> sssd-ad-1.14.0-43.el7_3.18.x86_64
> >> sssd-client-1.14.0-43.el7_3.18.x86_64
> >> sssd-common-1.14.0-43.el7_3.18.x86_64
> >> sssd-common-pac-1.14.0-43.el7_3.18.x86_64
> >> sssd-ipa-1.14.0-43.el7_3.18.x86_64
> >> sssd-krb5-1.14.0-43.el7_3.18.x86_64
> >> sssd-krb5-common-1.14.0-43.el7_3.18.x86_64
> >> sssd-ldap-1.14.0-43.el7_3.18.x86_64
> >> sssd-proxy-1.14.0-43.el7_3.18.x86_64
> >>
> >> In our case we have some DCs which are located at a partner site, and
> >> are therefore inaccessible to clients on our standard LANs.
> >>
> >> When SSSD starts it will correctly determine there are 2 primary DCs
> >> (these are the ones for the site) and 7 backup DCs.
> >>
> >> However, what is happening from time to time is that for some reason
> >> I've not yet determined the connection(s) to the primary DC(s) are
> >> dropping, and then sssd attempts to connect to one of the DCs that are
> >> inaccessible.
> >>
> >> In what circumstances would sssd prefer a backup server to a primary server?
> >>
> >> I've got a chunk of log which I've anonymised:
> >>
> >> https://paste.fedoraproject.org/paste/G69lC9COQfbnWFI~qtFLZw
> >
> > The issue is here:
>  [snip]
> >
> > So this is a known bug where locating the site (even though we know it
> > already) can contact the DCs outside that site.
> >
> > In the meantime, until the fix is released, you can hardcode the site
> > using the 'ad_site' option.
>
>  Thank you Jakub, is there a ticket/BZ for this? I noticed after my
>  post that it's evident on CentOS 6 too. 
Ah, https://pagure.io/SSSD/sssd/issue/3265 ?

John

-- 
John Beranek                         To generalise is to be an idiot.
http://redux.org.uk/                                 -- William Blake

[John Beranek]

On 12 September 2017 at 18:03, John Beranek wrote:
>  On 12 September 2017 at 17:59, John Beranek wrote:
> > On 11 September 2017 at 14:28, Jakub Hrozek wrote:
> >> On Mon, Sep 11, 2017 at 12:23:26PM +0100, John Beranek wrote:
> >> The issue is here:
> > [snip]
> >>
> >> So this is a known bug where locating the site (even though we know it
> >> already) can contact the DCs outside that site.
> >>
> >> In the meantime, until the fix is released, you can hardcode the site
> >> using the 'ad_site' option.
> >
> > Thank you Jakub, is there a ticket/BZ for this? I noticed after my
> > post that it's evident on CentOS 6 too.
>
>  Ah, https://pagure.io/SSSD/sssd/issue/3265 ? 
and https://pagure.io/SSSD/sssd/issue/2702 I presume.

John

-- 
John Beranek                         To generalise is to be an idiot.
http://redux.org.uk/                                 -- William Blake

[Jakub Hrozek]

On Tue, Sep 12, 2017 at 06:06:19PM +0100, John Beranek wrote:
>  On 12 September 2017 at 18:03, John Beranek wrote:
>  > On 12 September 2017 at 17:59, John Beranek wrote:
>  >> On 11 September 2017 at 14:28, Jakub Hrozek wrote:
>  >>> On Mon, Sep 11, 2017 at 12:23:26PM +0100, John Beranek wrote:
>  >>> The issue is here:
>  >> [snip]
>  >>>
>  >>> So this is a known bug where locating the site (even though we know it
>  >>> already) can contact the DCs outside that site.
>  >>>
>  >>> In the meantime, until the fix is released, you can hardcode the site
>  >>> using the 'ad_site' option.
>  >>
>  >> Thank you Jakub, is there a ticket/BZ for this? I noticed after my
>  >> post that it's evident on CentOS 6 too.
>  >
>  > Ah, https://pagure.io/SSSD/sssd/issue/3265 ?
>  
>  and https://pagure.io/SSSD/sssd/issue/2702 I presume. 
#3265 is what we want to fix in the next version.