IPA Locations: client chooses wrong/random IPA server

[Denis Iskandarov]

Hello,

I have two datacenters. One is self hosted so all the docs are working just
fine. The second one is AWS.
 I've managed to setup replica between "static" DC , "DC1", and AWS "cloud"
DC with all its problems.
There are still issues to solve within AWS.
But "static" which has Bind name servers is not working properly.

Client hosts are using my external DNSs, with proper zone delegation to IPA.
And are slaves of zone delegated to FreeIPA - so they do receive all
updates from IPA.
Default TTL lowered to 5min.

Configured IPA locations.

During install ipa-client-install in "DC1" chooses wrong IPA server, one
dedicated for AWS.
And because of our security policies clients are not allowed to communicate
with AWS IPA - so installation completely fails.
At some point randomly of course it chooses correct IPA server.

dig on every host in network shows correct priorities and weights.

Below I will paste related configs outputs:

; auth.tld zone delegation
> auth.tld.                       IN      NS      ipa.auth.tld.
> auth.tld.                       IN      NS      dns1.tld.
> ipa.auth.tld.                 IN      A       10.200.10.9
> ipa-aws.auth.tld.          IN      A       10.70.83.86

on client host, and every other dig looks like:
> # dig +short _ldap._tcp.auth.tld SRV
> _ldap._tcp.dc1._locations.auth.tld.
> 0 100 389 ipa.auth.tld.
> 50 100 389 ipa-aws.auth.tld.

or putting servers on different positions but still with correct priorities:
> dig +short _ldap._tcp.auth.tld SRV
> _ldap._tcp.dc1._locations.auth.tld.
> 50 100 389 ipa-aws.auth.tld.
> 0 100 389 ipa.auth.tld.

[root@client-001 ~]# /usr/sbin/ipa-client-install --domain='auth.tld'
> --principal='hostreg' --password='xxxxxxx' --mkhomedir --no-sshd --no-ntp
> Discovery was successful!
> Client hostname: client-001.dmz.dc1.tld
> Realm: AUTH.tld
> DNS Domain: auth.tld
> IPA Server: ipa-aws.auth.tld
> BaseDN: dc=auth,dc=tld
> Continue to configure the system with these values? [no]: no
> Installation failed. Rolling back changes.
> IPA client is not configured on this system.

I don't understand, why if dig shows correct priorities information,
ipa-client-install is choosing wrong server.
Looks like ipa-client install not treating priorities and is choosing first
ipa server from dig list.

Can't find out what I'm missing

[Martin Bašti]

On 22.06.2017 09:45, Denis Iskandarov via FreeIPA-users wrote:
> Hello,
>
> I have two datacenters. One is self hosted so all the docs are working 
> just fine. The second one is AWS.
>  I've managed to setup replica between "static" DC , "DC1", and AWS 
> "cloud" DC with all its problems.
> There are still issues to solve within AWS.
> But "static" which has Bind name servers is not working properly.
>
> Client hosts are using my external DNSs, with proper zone delegation 
> to IPA.
> And are slaves of zone delegated to FreeIPA - so they do receive all 
> updates from IPA.
> Default TTL lowered to 5min.
>
> Configured IPA locations.
>
> During install ipa-client-install in "DC1" chooses wrong IPA server, 
> one dedicated for AWS.
> And because of our security policies clients are not allowed to 
> communicate with AWS IPA - so installation completely fails.
> At some point randomly of course it chooses correct IPA server.
>
> dig on every host in network shows correct priorities and weights.
>
> Below I will paste related configs outputs:
> ; auth.tld zone delegation
> auth.tld.                       IN      NS  ipa.auth.tld.
> auth.tld.                       IN      NS      dns1.tld.
> ipa.auth.tld.                 IN      A       10.200.10.9
> ipa-aws.auth.tld.          IN      A       10.70.83.86
>
>
> on client host, and every other dig looks like:
> # dig +short _ldap._tcp.auth.tld SRV
> _ldap._tcp.dc1._locations.auth.tld.
> 0 100 389 ipa.auth.tld.
> 50 100 389 ipa-aws.auth.tld.
>
>
> or putting servers on different positions but still with correct 
> priorities:
> dig +short _ldap._tcp.auth.tld SRV
> _ldap._tcp.dc1._locations.auth.tld.
> 50 100 389 ipa-aws.auth.tld.
> 0 100 389 ipa.auth.tld.
>
>
>  [root@client-001 ~]# /usr/sbin/ipa-client-install
> --domain='auth.tld' --principal='hostreg' --password='xxxxxxx'
> --mkhomedir --no-sshd --no-ntp
> Discovery was successful!
> Client hostname: client-001.dmz.dc1.tld
> Realm: AUTH.tld
> DNS Domain: auth.tld
> IPA Server: ipa-aws.auth.tld
> BaseDN: dc=auth,dc=tld
> Continue to configure the system with these values? [no]: no
> Installation failed. Rolling back changes.
> IPA client is not configured on this system.
>
>
> I don't understand, why if dig shows correct priorities information, 
> ipa-client-install is choosing wrong server.
> Looks like ipa-client install not treating priorities and is choosing 
> first ipa server from dig list.
>
> Can't find out what I'm missing
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

Hello, ipa-client-install and IPA CLI doesn't respect priorities so 
server is really chosen randomly. ipa-client-install will be fixed in 
4.6 to support SRV priorities.

SSSD on client however support priorities, so after successful 
installation SSSD will choose correct servers.

Sorry.

[Denis Iskandarov]

Thanks for such quick response.

Nice to know that it's "broken by current design".
Now I can sleep calm.

I'm interested if such initial setup against "wrong" IPA server can have
some negative side effects?
1. Like client in "DC1" will first ask try ask IPA in "AWS"
2. If we loose connection between DC1 and AWS - client in DC1 will "lose
his mind"

In simple words:
I want to make sure that IPA server chosen during install want be evolved
in further life-cycle of client host
Of course i'm not talking about disaster failover case, usual operations
when proper IPA is healthy and reachable

Thanks

On Thu, Jun 22, 2017 at 12:00 PM, Martin Bašti  wrote:
> On 22.06.2017 09:45, Denis Iskandarov via FreeIPA-users wrote:
>
> Hello,
>
> I have two datacenters. One is self hosted so all the docs are working
> just fine. The second one is AWS.
>  I've managed to setup replica between "static" DC , "DC1", and AWS
> "cloud" DC with all its problems.
> There are still issues to solve within AWS.
> But "static" which has Bind name servers is not working properly.
>
> Client hosts are using my external DNSs, with proper zone delegation to
> IPA.
> And are slaves of zone delegated to FreeIPA - so they do receive all
> updates from IPA.
> Default TTL lowered to 5min.
>
> Configured IPA locations.
>
> During install ipa-client-install in "DC1" chooses wrong IPA server, one
> dedicated for AWS.
> And because of our security policies clients are not allowed to
> communicate with AWS IPA - so installation completely fails.
> At some point randomly of course it chooses correct IPA server.
>
> dig on every host in network shows correct priorities and weights.
>
> Below I will paste related configs outputs:
>
> ; auth.tld zone delegation
> > auth.tld.                       IN      NS      ipa.auth.tld.
> > auth.tld.                       IN      NS      dns1.tld.
> > ipa.auth.tld.                 IN      A       10.200.10.9
> > ipa-aws.auth.tld.          IN      A       10.70.83.86
>
> on client host, and every other dig looks like:
> > # dig +short _ldap._tcp.auth.tld SRV
> > _ldap._tcp.dc1._locations.auth.tld.
> > 0 100 389 ipa.auth.tld.
> > 50 100 389 ipa-aws.auth.tld.
>
> or putting servers on different positions but still with correct
> priorities:
> > dig +short _ldap._tcp.auth.tld SRV
> > _ldap._tcp.dc1._locations.auth.tld.
> > 50 100 389 ipa-aws.auth.tld.
> > 0 100 389 ipa.auth.tld.
>
> [root@client-001 ~]# /usr/sbin/ipa-client-install --domain='auth.tld'
> > --principal='hostreg' --password='xxxxxxx' --mkhomedir --no-sshd --no-ntp
> > Discovery was successful!
> > Client hostname: client-001.dmz.dc1.tld
> > Realm: AUTH.tld
> > DNS Domain: auth.tld
> > IPA Server: ipa-aws.auth.tld
> > BaseDN: dc=auth,dc=tld
> > Continue to configure the system with these values? [no]: no
> > Installation failed. Rolling back changes.
> > IPA client is not configured on this system.
>
> I don't understand, why if dig shows correct priorities information,
> ipa-client-install is choosing wrong server.
> Looks like ipa-client install not treating priorities and is choosing
> first ipa server from dig list.
>
> Can't find out what I'm missing
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>
> Hello, ipa-client-install and IPA CLI doesn't respect priorities so server
> is really chosen randomly. ipa-client-install will be fixed in 4.6 to
> support SRV priorities.
>
> SSSD on client however support priorities, so after successful
> installation SSSD will choose correct servers.
>
> Sorry.
>
> --
> Martin Bašti
> Software Engineer
> Red Hat Czech

[Martin Bašti]

After initial installation it should just work, as majority of features 
on client is done by SSSD daemon that support SRV priorities. 
ipa-client-install just set up services on client. Also please note, 
when you use --server option with ipa-client-install will turn off 
server autodiscovery in SSSD, so please don't use it.

only IPA CLI commands will ask random servers.

Martin

On 22.06.2017 10:22, Denis Iskandarov wrote:
> Thanks for such quick response.
>
> Nice to know that it's "broken by current design".
> Now I can sleep calm.
>
> I'm interested if such initial setup against "wrong" IPA server can 
> have some negative side effects?
> Like client in "DC1" will first ask try ask IPA in "AWS"
> If we loose connection between DC1 and AWS - client in DC1 will
>
> "lose his mind"
>
> In simple words:
> I want to make sure that IPA server chosen during install want be 
> evolved in further life-cycle of client host
> Of course i'm not talking about disaster failover case, usual 
> operations when proper IPA is healthy and reachable
>
> Thanks
>
> On Thu, Jun 22, 2017 at 12:00 PM, Martin Bašti <[e-mail] 
> > wrote:
> On 22.06.2017 09:45, Denis Iskandarov via FreeIPA-users wrote:
> > Hello,
> >
> > I have two datacenters. One is self hosted so all the docs are
> > working just fine. The second one is AWS.
> >  I've managed to setup replica between "static" DC , "DC1", and
> > AWS "cloud" DC with all its problems.
> > There are still issues to solve within AWS.
> > But "static" which has Bind name servers is not working properly.
> >
> > Client hosts are using my external DNSs, with proper zone
> > delegation to IPA.
> > And are slaves of zone delegated to FreeIPA - so they do receive
> > all updates from IPA.
> > Default TTL lowered to 5min.
> >
> > Configured IPA locations.
> >
> > During install ipa-client-install in "DC1" chooses wrong IPA
> > server, one dedicated for AWS.
> > And because of our security policies clients are not allowed to
> > communicate with AWS IPA - so installation completely fails.
> > At some point randomly of course it chooses correct IPA server.
> >
> > dig on every host in network shows correct priorities and weights.
> >
> > Below I will paste related configs outputs:
> >
> >     ; auth.tld zone delegation
> >     auth.tld.                       IN      NS  ipa.auth.tld.
> >     auth.tld.                       IN      NS  dns1.tld.
> >     ipa.auth.tld.                 IN      A 10.200.10.9
> >     ipa-aws.auth.tld.          IN      A 10.70.83.86
> >
> >
> >  on client host, and every other dig looks like:
> >
> >     # dig +short _ldap._tcp.auth.tld SRV
> >     _ldap._tcp.dc1._locations.auth.tld.
> >     0 100 389 ipa.auth.tld.
> >     50 100 389 ipa-aws.auth.tld.
> >
> >
> > or putting servers on different positions but still with correct
> > priorities:
> >
> >     dig +short _ldap._tcp.auth.tld SRV
> >     _ldap._tcp.dc1._locations.auth.tld.
> >     50 100 389 ipa-aws.auth.tld.
> >     0 100 389 ipa.auth.tld.
> >
> >
> >      [root@client-001 ~]# /usr/sbin/ipa-client-install
> >     --domain='auth.tld' --principal='hostreg'
> >     --password='xxxxxxx' --mkhomedir --no-sshd --no-ntp
> >     Discovery was successful!
> >     Client hostname: client-001.dmz.dc1.tld
> >     Realm: AUTH.tld
> >     DNS Domain: auth.tld
> >     IPA Server: ipa-aws.auth.tld
> >     BaseDN: dc=auth,dc=tld
> >     Continue to configure the system with these values? [no]: no
> >     Installation failed. Rolling back changes.
> >     IPA client is not configured on this system.
> >
> >
> > I don't understand, why if dig shows correct priorities
> > information, ipa-client-install is choosing wrong server.
> > Looks like ipa-client install not treating priorities and is
> > choosing first ipa server from dig list.
> >
> > Can't find out what I'm missing
> >
> >
> > _______________________________________________
> > FreeIPA-users mailing list [e-mail]
> > <mailto:[e-mail]>
> > To unsubscribe send an email [e-mail]
> > <mailto:[e-mail]>
> Hello, ipa-client-install and IPA CLI doesn't respect priorities
> so server is really chosen randomly. ipa-client-install will be
> fixed in 4.6 to support SRV priorities.
>
> SSSD on client however support priorities, so after successful
> installation SSSD will choose correct servers.
>
> Sorry.
>
>
> -- 
> Martin Bašti
> Software Engineer
> Red Hat Czech

[Denis Iskandarov]

Yea, auto-discovery is crucial in my architecture.
Thanks for your help!

On Thu, Jun 22, 2017 at 12:46 PM, Martin Bašti  wrote:
> After initial installation it should just work, as majority of features on
> client is done by SSSD daemon that support SRV priorities.
> ipa-client-install just set up services on client. Also please note, when
> you use --server option with ipa-client-install will turn off server
> autodiscovery in SSSD, so please don't use it.
>
> only IPA CLI commands will ask random servers.
>
> Martin
>
> On 22.06.2017 10:22, Denis Iskandarov wrote:
>
> Thanks for such quick response.
>
> Nice to know that it's "broken by current design".
> Now I can sleep calm.
>
> I'm interested if such initial setup against "wrong" IPA server can have
> some negative side effects?
> Like client in "DC1" will first ask try ask IPA in "AWS"
> If we loose connection between DC1 and AWS - client in DC1 will "lose
>
> his mind"
>
> In simple words:
> I want to make sure that IPA server chosen during install want be evolved
> in further life-cycle of client host
> Of course i'm not talking about disaster failover case, usual operations
> when proper IPA is healthy and reachable
>
> Thanks
>
> On Thu, Jun 22, 2017 at 12:00 PM, Martin Bašti  wrote:
> > On 22.06.2017 09:45, Denis Iskandarov via FreeIPA-users wrote:
> >
> > Hello,
> >
> > I have two datacenters. One is self hosted so all the docs are working
> > just fine. The second one is AWS.
> >  I've managed to setup replica between "static" DC , "DC1", and AWS
> > "cloud" DC with all its problems.
> > There are still issues to solve within AWS.
> > But "static" which has Bind name servers is not working properly.
> >
> > Client hosts are using my external DNSs, with proper zone delegation to
> > IPA.
> > And are slaves of zone delegated to FreeIPA - so they do receive all
> > updates from IPA.
> > Default TTL lowered to 5min.
> >
> > Configured IPA locations.
> >
> > During install ipa-client-install in "DC1" chooses wrong IPA server, one
> > dedicated for AWS.
> > And because of our security policies clients are not allowed to
> > communicate with AWS IPA - so installation completely fails.
> > At some point randomly of course it chooses correct IPA server.
> >
> > dig on every host in network shows correct priorities and weights.
> >
> > Below I will paste related configs outputs:
> >
> > ; auth.tld zone delegation
> > > auth.tld.                       IN      NS      ipa.auth.tld.
> > > auth.tld.                       IN      NS      dns1.tld.
> > > ipa.auth.tld.                 IN      A       10.200.10.9
> > > ipa-aws.auth.tld.          IN      A       10.70.83.86
> >
> > on client host, and every other dig looks like:
> > > # dig +short _ldap._tcp.auth.tld SRV
> > > _ldap._tcp.dc1._locations.auth.tld.
> > > 0 100 389 ipa.auth.tld.
> > > 50 100 389 ipa-aws.auth.tld.
> >
> > or putting servers on different positions but still with correct
> > priorities:
> > > dig +short _ldap._tcp.auth.tld SRV
> > > _ldap._tcp.dc1._locations.auth.tld.
> > > 50 100 389 ipa-aws.auth.tld.
> > > 0 100 389 ipa.auth.tld.
> >
> > [root@client-001 ~]# /usr/sbin/ipa-client-install --domain='auth.tld'
> > > --principal='hostreg' --password='xxxxxxx' --mkhomedir --no-sshd --no-ntp
> > > Discovery was successful!
> > > Client hostname: client-001.dmz.dc1.tld
> > > Realm: AUTH.tld
> > > DNS Domain: auth.tld
> > > IPA Server: ipa-aws.auth.tld
> > > BaseDN: dc=auth,dc=tld
> > > Continue to configure the system with these values? [no]: no
> > > Installation failed. Rolling back changes.
> > > IPA client is not configured on this system.
> >
> > I don't understand, why if dig shows correct priorities information,
> > ipa-client-install is choosing wrong server.
> > Looks like ipa-client install not treating priorities and is choosing
> > first ipa server from dig list.
> >
> > Can't find out what I'm missing
> >
> > FreeIPA-users mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> >
> > Hello, ipa-client-install and IPA CLI doesn't respect priorities so
> > server is really chosen randomly. ipa-client-install will be fixed in 4.6
> > to support SRV priorities.
> >
> > SSSD on client however support priorities, so after successful
> > installation SSSD will choose correct servers.
> >
> > Sorry.
> >
> > --
> > Martin Bašti
> > Software Engineer
> > Red Hat Czech
>
> --
> Martin Bašti
> Software Engineer
> Red Hat Czech