ipa-replica-install command failed, exception: NotFound: ldap service not found

Trying to create a replica server with ipa-replica-install, but it breaks during installation while restarting the directory service saying that LDAP service not found. But I can see LDAP server is running.

I have created around 3 replicas using the same procedure about 4 months ago, but now it is failing. I cannot find any obvious reason for this issue.
All the machines are on CentOS 7.x.

Master ipa package versions:
ipa-common-4.4.0-14.el7.centos.6.noarch
ipa-client-common-4.4.0-14.el7.centos.6.noarch
ipa-server-dns-4.4.0-14.el7.centos.6.noarch
ipa-admintools-4.4.0-14.el7.centos.6.noarch
ipa-server-4.4.0-14.el7.centos.6.x86_64

Also tried after updating above to el7.centos.7 packages

Replica ipa package versions:
ipa-common-4.4.0-14.el7.centos.7.noarch
ipa-server-4.4.0-14.el7.centos.7.x86_64
ipa-client-4.4.0-14.el7.centos.7.x86_64
ipa-server-common-4.4.0-14.el7.centos.7.noarch
ipa-admintools-4.4.0-14.el7.centos.7.noarch
ipa-client-common-4.4.0-14.el7.centos.7.noarch
ipa-server-dns-4.4.0-14.el7.centos.7.noarch

Actual results:
[root@auth03-esy1 ~]# ipa-replica-install --principal admin --admin-password XXXXXXXX --server=auth02-esy1.srv.symbionetworks.com --domain=auth.mnfgroup.limited --setup-ca
Configuring client side components
Client hostname: auth03-esy1.srv.symbionetworks.com
Realm: AUTH.MNFGROUP.LIMITED
DNS Domain: auth.mnfgroup.limited
IPA Server: auth02-esy1.srv.symbionetworks.com
BaseDN: dc=auth,dc=mnfgroup,dc=limited

Skipping synchronizing time with NTP server.
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=AUTH.MNFGROUP.LIMITED
    Issuer:      CN=Certificate Authority,O=AUTH.MNFGROUP.LIMITED
    Valid From:  Wed Mar 15 01:04:16 2017 UTC
    Valid Until: Sun Mar 15 01:04:16 2037 UTC

Enrolled in IPA realm AUTH.MNFGROUP.LIMITED
Created /etc/ipa/default.conf
New SSSD config will be created
Configured sudoers in /etc/nsswitch.conf
Configured /etc/sssd/sssd.conf
Configured /etc/krb5.conf for IPA realm AUTH.MNFGROUP.LIMITED
trying https://auth02-esy1.srv.symbionetworks.com/ipa/json
Forwarding 'ping' to json server 'https://auth02-esy1.srv.symbionetworks.com/ipa/json'
Forwarding 'ca_is_enabled' to json server 'https://auth02-esy1.srv.symbionetworks.com/ipa/json'
Systemwide CA database updated.
Hostname (auth03-esy1.srv.symbionetworks.com) does not have A/AAAA record.
Failed to update DNS records.
Missing A/AAAA record(s) for host auth03-esy1.srv.symbionetworks.com: 10.53.1.3.
Missing reverse record(s) for address(es): 10.53.1.3.
Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ed25519_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ecdsa_key.pub
Forwarding 'host_mod' to json server 'https://auth02-esy1.srv.symbionetworks.com/ipa/json'
Could not update DNS SSHFP records.
SSSD enabled
Configured /etc/openldap/ldap.conf
Configured /etc/ssh/ssh_config
Configured /etc/ssh/sshd_config
Configuring auth.mnfgroup.limited as NIS domain.
Client configuration complete.

WARNING: conflicting time&date synchronization service 'chronyd' will
be disabled in favor of ntpd

ipa         : ERROR    Could not resolve hostname auth02-esy1.srv.symbionetworks.com using DNS. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.)
Continue? [no]: yes
Run connection check to master
Connection check OK
Configuring NTP daemon (ntpd)
  [1/4]: stopping ntpd
  [2/4]: writing configuration
  [3/4]: configuring ntpd to start on boot
  [4/4]: starting ntpd
Done configuring NTP daemon (ntpd).
Configuring directory server (dirsrv). Estimated time: 1 minute
  [1/44]: creating directory server user
  [2/44]: creating directory server instance
  [3/44]: updating configuration in dse.ldif
  [4/44]: restarting directory server
  [5/44]: adding default schema
  [6/44]: enabling memberof plugin
  [7/44]: enabling winsync plugin
  [8/44]: configuring replication version plugin
  [9/44]: enabling IPA enrollment plugin
  [10/44]: enabling ldapi
  [11/44]: configuring uniqueness plugin
  [12/44]: configuring uuid plugin
  [13/44]: configuring modrdn plugin
  [14/44]: configuring DNS plugin
  [15/44]: enabling entryUSN plugin
  [16/44]: configuring lockout plugin
  [17/44]: configuring topology plugin
  [18/44]: creating indices
  [19/44]: enabling referential integrity plugin
  [20/44]: configuring certmap.conf
  [21/44]: configure autobind for root
  [22/44]: configure new location for managed entries
  [23/44]: configure dirsrv ccache
  [24/44]: enabling SASL mapping fallback
  [25/44]: restarting directory server
  [26/44]: creating DS keytab
  [error] NotFound: ldap/[e-mail]: service not found
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

ipa.ipapython.install.cli.install_tool(Replica): ERROR    ldap/[e-mail]: service not found
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

Additional Infomation:
Form /var/log/ipareplica-install.log,
2017-09-12T01:36:13Z DEBUG stderr=ldap_initialize( ldap://auth03-esy1.srv.symbionetworks.com:389/??base )

2017-09-12T01:36:13Z DEBUG   duration: 0 seconds
2017-09-12T01:36:13Z DEBUG   [23/44]: configure dirsrv ccache
2017-09-12T01:36:13Z DEBUG Backing up system configuration file '/etc/sysconfig/dirsrv'
2017-09-12T01:36:13Z DEBUG Saving Index File to '/var/lib/ipa/sysrestore/sysrestore.index'
2017-09-12T01:36:13Z DEBUG Starting external process
2017-09-12T01:36:13Z DEBUG args=/usr/sbin/selinuxenabled
2017-09-12T01:36:13Z DEBUG Process finished, return code=1
2017-09-12T01:36:13Z DEBUG stdout=
2017-09-12T01:36:13Z DEBUG stderr=
2017-09-12T01:36:13Z DEBUG   duration: 0 seconds
2017-09-12T01:36:13Z DEBUG   [24/44]: enabling SASL mapping fallback
2017-09-12T01:36:13Z DEBUG Starting external process
2017-09-12T01:36:13Z DEBUG args=/usr/bin/ldapmodify -v -f /tmp/tmpEjW0XE -H ldap://auth03-esy1.srv.symbionetworks.com:389 -x -D cn=Directory Manager -y /tmp/tmpED2rPP
2017-09-12T01:36:13Z DEBUG Process finished, return code=0
2017-09-12T01:36:13Z DEBUG stdout=replace nsslapd-sasl-mapping-fallback:
        on
modifying entry "cn=config"
modify complete

2017-09-12T01:36:13Z DEBUG stderr=ldap_initialize( ldap://auth03-esy1.srv.symbionetworks.com:389/??base )

2017-09-12T01:36:13Z DEBUG   duration: 0 seconds
2017-09-12T01:36:13Z DEBUG   [25/44]: restarting directory server
2017-09-12T01:36:13Z DEBUG Starting external process
2017-09-12T01:36:13Z DEBUG args=/bin/systemctl --system daemon-reload
2017-09-12T01:36:13Z DEBUG Process finished, return code=0
2017-09-12T01:36:13Z DEBUG stdout=
2017-09-12T01:36:13Z DEBUG stderr=
2017-09-12T01:36:13Z DEBUG Starting external process
2017-09-12T01:36:13Z DEBUG args=/bin/systemctl restart [e-mail]
2017-09-12T01:36:14Z DEBUG Process finished, return code=0
2017-09-12T01:36:14Z DEBUG stdout=
2017-09-12T01:36:14Z DEBUG stderr=
2017-09-12T01:36:14Z DEBUG Starting external process
2017-09-12T01:36:14Z DEBUG args=/bin/systemctl is-active [e-mail]
2017-09-12T01:36:14Z DEBUG Process finished, return code=0
2017-09-12T01:36:14Z DEBUG stdout=active

2017-09-12T01:36:14Z DEBUG stderr=
2017-09-12T01:36:14Z DEBUG wait_for_open_ports: localhost [389] timeout 300
2017-09-12T01:36:14Z DEBUG Starting external process
2017-09-12T01:36:14Z DEBUG args=/bin/systemctl is-active [e-mail]
2017-09-12T01:36:14Z DEBUG Process finished, return code=0
2017-09-12T01:36:14Z DEBUG stdout=active

2017-09-12T01:36:14Z DEBUG stderr=
2017-09-12T01:36:14Z DEBUG   duration: 0 seconds
2017-09-12T01:36:14Z DEBUG   [26/44]: creating DS keytab
2017-09-12T01:36:14Z DEBUG Backing up system configuration file '/etc/dirsrv/ds.keytab'
2017-09-12T01:36:14Z DEBUG   -> Not backing up - '/etc/dirsrv/ds.keytab' doesn't exist
2017-09-12T01:36:14Z DEBUG raw: service_add(u'ldap/[e-mail]', force=True, version=u'2.213')
2017-09-12T01:36:14Z DEBUG service_add(<ipapython.kerberos.Principal object at 0x794e7d0>, force=True, all=False, raw=False, version=u'2.213', no_members=False)
2017-09-12T01:36:14Z DEBUG flushing ldaps://auth02-esy1.srv.symbionetworks.com from SchemaCache
2017-09-12T01:36:14Z DEBUG retrieving schema for SchemaCache url=ldaps://auth02-esy1.srv.symbionetworks.com conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x76610e0>
2017-09-12T01:36:15Z DEBUG raw: host_show(u'auth03-esy1.srv.symbionetworks.com', version=u'2.213')
2017-09-12T01:36:15Z DEBUG host_show(u'auth03-esy1.srv.symbionetworks.com', rights=False, all=False, raw=False, version=u'2.213', no_members=False)
2017-09-12T01:36:15Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 449, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 439, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 1230, in __get_ds_keytab
    force_service_add=True)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 1129, in install_service_keytab
    api.Command.service_add(principal, force=force_service_add)
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 449, in __call__
    return self.__do_call(*args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 477, in __do_call
    ret = self.run(*args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 799, in run
    return self.execute(*args, **options)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/baseldap.py", line 1221, in execute
    self.obj.handle_not_found(*keys)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/baseldap.py", line 759, in handle_not_found
    'pkey': pkey, 'oname': self.object_name,
NotFound: ldap/[e-mail]: service not found

2017-09-12T01:36:15Z DEBUG   [error] NotFound: ldap/[e-mail]: service not found
2017-09-12T01:36:15Z DEBUG Destroyed connection context.ldap2_89533776
2017-09-12T01:36:15Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, in run
    cfgr.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, in run
    self.execute()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, in execute
    for nothing in self._executor():
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 586, in _configure
    next(executor)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, in _handle_exception
    self.__parent._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, in _handle_exception
    super(ComponentBase, self)._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install
    for nothing in self._installer(self.parent):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1722, in main
    promote(self)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 372, in decorated
    func(installer)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1423, in promote
    promote=True, pkcs12_info=dirsrv_pkcs12_info)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 135, in install_replica_ds
    api=remote_api,
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 401, in create_replica
    self.start_creation(runtime=60)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 449, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 439, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 1230, in __get_ds_keytab
    force_service_add=True)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 1129, in install_service_keytab
    api.Command.service_add(principal, force=force_service_add)
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 449, in __call__
    return self.__do_call(*args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 477, in __do_call
    ret = self.run(*args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 799, in run
    return self.execute(*args, **options)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/baseldap.py", line 1221, in execute
    self.obj.handle_not_found(*keys)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/baseldap.py", line 759, in handle_not_found
    'pkey': pkey, 'oname': self.object_name,

2017-09-12T01:36:15Z DEBUG The ipa-replica-install command failed, exception: NotFound: ldap/[e-mail]: service not found
2017-09-12T01:36:15Z ERROR ldap/[e-mail]: service not found
2017-09-12T01:36:15Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

Can anyone please help with this issue?

Regards

Shahriar Rahman
Systems Engineer
MNF Group Limited