keytab file deleted and sssd not starting

[Jeremy Tourville]

I don't know how but somehow /etc/krb5.keytab was deleted on my ipa server.  As a
result my sssd service is dead because the keytab file is missing

I understand the process to fix this is:
1. kinit admin and provide the password
2. ipa-getkeytab -s <FreeIPA server> -p host/<hostname>@REALM -k <keytab
file>.

if command #2 is successful you should get a keytab file and you can run systemctl restart
sssd and all will be happy again.

My problem is that when running command #2 I get an error if I specify -p host/ for the
service prinicpal.  
SASL bind failed
invalid credentials
failed to bind to server
Retrying with pre-4.0 keytab retrieval method... 

If I specify -p ldap/ I can get it to complete and a keytab gets generated.

As a side note I have a bunch of ipa-healthcheck erors but maybe those are mostly related
to the fact that sssd is not running and keytab is missing.
The affected server is my primary but I do have a replica that is in good heatlh except
for one healthcheck issue for dns.

Both servers contain the following roles:
AD trust agent
AD trust controller
CA server
DNS server

The IPA server also has a one way trust established with AD.

What is the correct principal to specify?  Is it host or ldap or dns or something else?

[Rob Crittenden]

Jeremy Tourville via FreeIPA-users wrote:
>  I don't know how but somehow /etc/krb5.keytab was deleted on my
> ipa server.  As a result my sssd service is dead because the keytab file is missing
>  
>  I understand the process to fix this is:
>  1. kinit admin and provide the password
>  2. ipa-getkeytab -s <FreeIPA server> -p host/<hostname>@REALM -k <keytab
> file>.
>  
>  if command #2 is successful you should get a keytab file and you can run systemctl
> restart sssd and all will be happy again.
>  
>  My problem is that when running command #2 I get an error if I specify -p host/ for the
> service prinicpal.  
>  SASL bind failed
>  invalid credentials
>  failed to bind to server
>  Retrying with pre-4.0 keytab retrieval method... 
>  
>  If I specify -p ldap/ I can get it to complete and a keytab gets generated.
>  
>  As a side note I have a bunch of ipa-healthcheck erors but maybe those are mostly related
> to the fact that sssd is not running and keytab is missing.
>  The affected server is my primary but I do have a replica that is in good heatlh except
> for one healthcheck issue for dns.
>  
>  Both servers contain the following roles:
>  AD trust agent
>  AD trust controller
>  CA server
>  DNS server
>  
>  The IPA server also has a one way trust established with AD.
>  
>  What is the correct principal to specify?  Is it host or ldap or dns or something else?
host is the correct principal. Have you tried specifying the other
server with -s in ipa-getkeytab?

Does the IPA server host entry still exist?

rob

[Jeremy Tourville]

> >>Does the IPA server host entry still exist? 
Yes, I see
it under Identity>Hosts


Specifying the other server did work.  SSSD is working again.
[root@gsil-ipa01 etc]# ipa-getkeytab -s gsil-ipa02.idm.x.x -p
host/gsil-ipa01.idm.x.x(a)IDM.x.x -k /etc/krb5.keytab

After doing this I restarted the server and tried to run health check again.  Now
healthcheck has a lot of errors.  Where do you suggest I start?  

I was also reading that  "If your IdM topology contains an integrated CA, one server
has the role of the Certificate revocation list (CRL) publisher server and one server has
the role of the CA renewal server.

By default, the first CA server installed fulfills these two roles..."
It was my first installed server that failed.  Should I move the roles to my replica? 
(Assuming I can)

Here is the healthcheck- 
I presume that if I can fix dirsrv it will help clear up many of the other issues.  Let me
know if you think there is something more critical to be fixed first.

[root@gsil-ipa01 ~]# ipa-healthcheck --failures-only
caSigningCert External CA not found, assuming 3rd party
[
  {
    "source": "ipahealthcheck.meta.services",
    "check": "dirsrv",
    "result": "ERROR",
    "uuid": "c7a2bf32-5878-44f5-b7a5-87b69e4149fa",
    "when": "20230316130703Z",
    "duration": "498.704933",
    "kw": {
      "status": false,
      "msg": "dirsrv: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "httpd",
    "result": "ERROR",
    "uuid": "edf67606-4326-4e37-b860-dc3992eb3bc7",
    "when": "20230316130703Z",
    "duration": "0.105259",
    "kw": {
      "status": false,
      "msg": "httpd: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "ipa_custodia",
    "result": "ERROR",
    "uuid": "87418dcb-c07c-4914-9296-ae5a2baca99d",
    "when": "20230316130703Z",
    "duration": "0.109029",
    "kw": {
      "status": false,
      "msg": "ipa-custodia: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "ipa_dnskeysyncd",
    "result": "ERROR",
    "uuid": "97d08885-7dac-4c60-a447-b669a4cc6e09",
    "when": "20230316130703Z",
    "duration": "0.100007",
    "kw": {
      "status": false,
      "msg": "ipa-dnskeysyncd: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "ipa_otpd",
    "result": "ERROR",
    "uuid": "c83f8cd2-6553-4934-99dd-de4cfa3f515f",
    "when": "20230316130703Z",
    "duration": "0.103143",
    "kw": {
      "status": false,
      "msg": "ipa-otpd: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "kadmin",
    "result": "ERROR",
    "uuid": "657130bf-396f-48a1-968f-3f89170313a3",
    "when": "20230316130703Z",
    "duration": "0.104496",
    "kw": {
      "status": false,
      "msg": "kadmin: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "krb5kdc",
    "result": "ERROR",
    "uuid": "5e82b86c-ac90-49f9-a8fe-1a8c35909f91",
    "when": "20230316130703Z",
    "duration": "0.097375",
    "kw": {
      "status": false,
      "msg": "krb5kdc: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "named",
    "result": "ERROR",
    "uuid": "3ca8751b-803a-46af-a849-fcb309eb65db",
    "when": "20230316130703Z",
    "duration": "0.093471",
    "kw": {
      "status": false,
      "msg": "named: not running"
    }
  },
  {
    "source": "ipahealthcheck.meta.services",
    "check": "pki_tomcatd",
    "result": "ERROR",
    "uuid": "9c57a360-bd4c-48be-af8a-6e738c79c486",
    "when": "20230316130703Z",
    "duration": "0.002969",
    "kw": {
      "status": false,
      "msg": "pki_tomcatd: not running"
    }
  },
  {
    "source": "ipahealthcheck.ipa.files",
    "check": "IPAFileCheck",
    "result": "WARNING",
    "uuid": "4a9001d5-1ad5-4a4c-99fd-48273dd9f822",
    "when": "20230316130707Z",
    "duration": "0.005756",
    "kw": {
      "key": "_var_log_ipaupgrade.log_mode",
      "path": "/var/log/ipaupgrade.log",
      "type": "mode",
      "expected": "0600",
      "got": "0644",
      "msg": "Permissions of /var/log/ipaupgrade.log are too permissive:
0644 and should be 0600"
    }
  }
]

[Jeremy Tourville]

UPDATE:
I did a little more troubleshooting and was able to get dirsrv to start.  Now I need to
figure out why named service won't start.  Here's the output from starting
services and ipa-healthcheck.  I presume several of the healthcheck failures are due to
named service not running.  Can anyone confirm?

[root@gsil-ipa01 ipa]# ipactl status
Directory Service: STOPPED
Directory Service must be running in order to obtain status of other services
[root@gsil-ipa01 ipa]# ipactl start --ignore-service-failures
Existing service file detected!
Assuming stale, cleaning and proceeding
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Failed to start named Service
Forced start, ignoring named Service, continuing normal operation
Starting httpd Service
Starting ipa-custodia Service
Starting pki-tomcatd Service
Starting smb Service
Starting winbind Service
Starting ipa-otpd Service
Starting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful
[root@gsil-ipa01 ipa]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: STOPPED
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
smb Service: RUNNING
winbind Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
1 service(s) are not running
[root@gsil-ipa01 ipa]# ipa-healthcheck --failures-only
caSigningCert External CA not found, assuming 3rd party
[
  {
    "source": "ipahealthcheck.meta.services",
    "check": "named",
    "result": "ERROR",
    "uuid": "b5bfa450-77f4-4655-a4e2-fccbf88aa43a",
    "when": "20230316153125Z",
    "duration": "0.111160",
    "kw": {
      "status": false,
      "msg": "named: not running"
    }
  },
  {
    "source": "ipahealthcheck.ds.replication",
    "check": "ReplicationCheck",
    "result": "CRITICAL",
    "uuid": "dcaa538c-a5e2-4247-9210-d6047a0d65f5",
    "when": "20230316153132Z",
    "duration": "0.281251",
    "kw": {
      "key": "DSREPLLE0001",
      "items": [
        "Replication",
        "Agreement"
      ],
      "msg": "The replication agreement (metogsil-ipa02.idm.x.xl) under
\"dc=idm,dc=x,dc=x\" is not in synchronization."
    }
  },
  {
    "source": "ipahealthcheck.ds.replication",
    "check": "ReplicationCheck",
    "result": "CRITICAL",
    "uuid": "556f572a-0ee9-42fa-8c06-b90e33ed961d",
    "when": "20230316153132Z",
    "duration": "0.281301",
    "kw": {
      "key": "DSREPLLE0001",
      "items": [
        "Replication",
        "Agreement"
      ],
      "msg": "The replication agreement (catogsil-ipa02.idm.x.x) under
\"o=ipaca\" is not in synchronization."
    }
  },
  {
    "source": "ipahealthcheck.ipa.dna",
    "check": "IPADNARangeCheck",
    "result": "CRITICAL",
    "uuid": "7b88f564-dac5-4191-96ec-b9ad922c0f5e",
    "when": "20230316153142Z",
    "duration": "0.027683",
    "kw": {
      "exception": "Insufficient access: SASL(-1): generic failure: GSSAPI
Error: Unspecified GSS failure.  Minor code may provide more information
(Preauthentication failed)"
    }
  },
  {
    "source": "ipahealthcheck.ipa.idns",
    "check": "IPADNSSystemRecordsCheck",
    "result": "WARNING",
    "uuid": "6b0bc0c1-d505-4f5a-944d-42dd044b2365",
    "when": "20230316153426Z",
    "duration": "164.364540",
    "kw": {
      "msg": "Got {count} ipa-ca A records, expected {expected}",
      "count": 1,
      "expected": 2
    }
  },
  {
    "source": "ipahealthcheck.ipa.files",
    "check": "IPAFileCheck",
    "result": "WARNING",
    "uuid": "ea3fcb5d-a280-4a29-ab5b-60abe15febdb",
    "when": "20230316153426Z",
    "duration": "0.003201",
    "kw": {
      "key": "_var_log_ipaupgrade.log_mode",
      "path": "/var/log/ipaupgrade.log",
      "type": "mode",
      "expected": "0600",
      "got": "0644",
      "msg": "Permissions of /var/log/ipaupgrade.log are too permissive:
0644 and should be 0600"
    }
  },
  {
    "source": "ipahealthcheck.ipa.host",
    "check": "IPAHostKeytab",
    "result": "ERROR",
    "uuid": "9e43e0d9-7143-40b1-8411-c0aa4b53bb1e",
    "when": "20230316153426Z",
    "duration": "0.027001",
    "kw": {
      "msg": "Failed to obtain host TGT: Major (851968): Unspecified GSS
failure.  Minor code may provide more information, Minor (2529638936): Preauthentication
failed"
    }
  },
  {
    "source": "ipahealthcheck.ipa.trust",
    "check": "IPATrustDomainsCheck",
    "result": "ERROR",
    "uuid": "a0ed3f4b-c409-42e4-b730-d9964ed46f64",
    "when": "20230316153427Z",
    "duration": "0.336395",
    "kw": {
      "key": "domain-list",
      "sssctl": "/usr/sbin/sssctl",
      "sssd_domains": "",
      "trust_domains": "gx.x",
      "msg": "{sssctl} {key} reports mismatch: sssd domains {sssd_domains}
trust domains {trust_domains}"
    }
  },
  {
    "source": "ipahealthcheck.ipa.trust",
    "check": "IPATrustCatalogCheck",
    "result": "WARNING",
    "uuid": "fd1ff67b-48b3-49dd-a3b4-32631a51672f",
    "when": "20230316153427Z",
    "duration": "0.013619",
    "kw": {
      "key": "S-1-5-21-3568498085-2952124370-1649233135",
      "error": "returned nothing",
      "msg": "Look up of {key} {error}"
    }
  },
  {
    "source": "ipahealthcheck.ipa.trust",
    "check": "IPATrustCatalogCheck",
    "result": "ERROR",
    "uuid": "c478454c-f94c-4089-ade4-7c3bd73d6b65",
    "when": "20230316153427Z",
    "duration": "0.127239",
    "kw": {
      "key": "domain-status",
      "error": "CalledProcessError(Command ['/usr/sbin/sssctl',
'domain-status', 'gx.x', '--active-server'] returned non-zero exit
status 1: 'Unable to get online status\\n')",
      "msg": "Execution of {key} failed: {error}"
    }
  }
]

[Rob Crittenden]

Jeremy Tourville via FreeIPA-users wrote:
>  UPDATE:
>  I did a little more troubleshooting and was able to get dirsrv to start.  Now I need to
> figure out why named service won't start.  Here's the output from starting
> services and ipa-healthcheck.  I presume several of the healthcheck failures are due to
> named service not running.  Can anyone confirm? 
It's likely. Kerberos and TLS rely on working name resolution. If your
server has a valid entry in /etc/hosts that may mitigate some issues but
but I'd still focus on getting named to start as a first step.

rob

>  
>  [root@gsil-ipa01 ipa]# ipactl status
>  Directory Service: STOPPED
>  Directory Service must be running in order to obtain status of other services
>  [root@gsil-ipa01 ipa]# ipactl start --ignore-service-failures
>  Existing service file detected!
>  Assuming stale, cleaning and proceeding
>  Starting Directory Service
>  Starting krb5kdc Service
>  Starting kadmin Service
>  Starting named Service
>  Failed to start named Service
>  Forced start, ignoring named Service, continuing normal operation
>  Starting httpd Service
>  Starting ipa-custodia Service
>  Starting pki-tomcatd Service
>  Starting smb Service
>  Starting winbind Service
>  Starting ipa-otpd Service
>  Starting ipa-dnskeysyncd Service
>  ipa: INFO: The ipactl command was successful
>  [root@gsil-ipa01 ipa]# ipactl status
>  Directory Service: RUNNING
>  krb5kdc Service: RUNNING
>  kadmin Service: RUNNING
>  named Service: STOPPED
>  httpd Service: RUNNING
>  ipa-custodia Service: RUNNING
>  pki-tomcatd Service: RUNNING
>  smb Service: RUNNING
>  winbind Service: RUNNING
>  ipa-otpd Service: RUNNING
>  ipa-dnskeysyncd Service: RUNNING
>  1 service(s) are not running
>  [root@gsil-ipa01 ipa]# ipa-healthcheck --failures-only
>  caSigningCert External CA not found, assuming 3rd party
>  [
>    {
>      "source": "ipahealthcheck.meta.services",
>      "check": "named",
>      "result": "ERROR",
>      "uuid": "b5bfa450-77f4-4655-a4e2-fccbf88aa43a",
>      "when": "20230316153125Z",
>      "duration": "0.111160",
>      "kw": {
>        "status": false,
>        "msg": "named: not running"
>      }
>    },
>    {
>      "source": "ipahealthcheck.ds.replication",
>      "check": "ReplicationCheck",
>      "result": "CRITICAL",
>      "uuid": "dcaa538c-a5e2-4247-9210-d6047a0d65f5",
>      "when": "20230316153132Z",
>      "duration": "0.281251",
>      "kw": {
>        "key": "DSREPLLE0001",
>        "items": [
>          "Replication",
>          "Agreement"
>        ],
>        "msg": "The replication agreement (metogsil-ipa02.idm.x.xl) under
> \"dc=idm,dc=x,dc=x\" is not in synchronization."
>      }
>    },
>    {
>      "source": "ipahealthcheck.ds.replication",
>      "check": "ReplicationCheck",
>      "result": "CRITICAL",
>      "uuid": "556f572a-0ee9-42fa-8c06-b90e33ed961d",
>      "when": "20230316153132Z",
>      "duration": "0.281301",
>      "kw": {
>        "key": "DSREPLLE0001",
>        "items": [
>          "Replication",
>          "Agreement"
>        ],
>        "msg": "The replication agreement (catogsil-ipa02.idm.x.x) under
> \"o=ipaca\" is not in synchronization."
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.dna",
>      "check": "IPADNARangeCheck",
>      "result": "CRITICAL",
>      "uuid": "7b88f564-dac5-4191-96ec-b9ad922c0f5e",
>      "when": "20230316153142Z",
>      "duration": "0.027683",
>      "kw": {
>        "exception": "Insufficient access: SASL(-1): generic failure: GSSAPI
> Error: Unspecified GSS failure.  Minor code may provide more information
> (Preauthentication failed)"
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.idns",
>      "check": "IPADNSSystemRecordsCheck",
>      "result": "WARNING",
>      "uuid": "6b0bc0c1-d505-4f5a-944d-42dd044b2365",
>      "when": "20230316153426Z",
>      "duration": "164.364540",
>      "kw": {
>        "msg": "Got {count} ipa-ca A records, expected {expected}",
>        "count": 1,
>        "expected": 2
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.files",
>      "check": "IPAFileCheck",
>      "result": "WARNING",
>      "uuid": "ea3fcb5d-a280-4a29-ab5b-60abe15febdb",
>      "when": "20230316153426Z",
>      "duration": "0.003201",
>      "kw": {
>        "key": "_var_log_ipaupgrade.log_mode",
>        "path": "/var/log/ipaupgrade.log",
>        "type": "mode",
>        "expected": "0600",
>        "got": "0644",
>        "msg": "Permissions of /var/log/ipaupgrade.log are too permissive:
> 0644 and should be 0600"
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.host",
>      "check": "IPAHostKeytab",
>      "result": "ERROR",
>      "uuid": "9e43e0d9-7143-40b1-8411-c0aa4b53bb1e",
>      "when": "20230316153426Z",
>      "duration": "0.027001",
>      "kw": {
>        "msg": "Failed to obtain host TGT: Major (851968): Unspecified GSS
> failure.  Minor code may provide more information, Minor (2529638936): Preauthentication
> failed"
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.trust",
>      "check": "IPATrustDomainsCheck",
>      "result": "ERROR",
>      "uuid": "a0ed3f4b-c409-42e4-b730-d9964ed46f64",
>      "when": "20230316153427Z",
>      "duration": "0.336395",
>      "kw": {
>        "key": "domain-list",
>        "sssctl": "/usr/sbin/sssctl",
>        "sssd_domains": "",
>        "trust_domains": "gx.x",
>        "msg": "{sssctl} {key} reports mismatch: sssd domains {sssd_domains}
> trust domains {trust_domains}"
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.trust",
>      "check": "IPATrustCatalogCheck",
>      "result": "WARNING",
>      "uuid": "fd1ff67b-48b3-49dd-a3b4-32631a51672f",
>      "when": "20230316153427Z",
>      "duration": "0.013619",
>      "kw": {
>        "key": "S-1-5-21-3568498085-2952124370-1649233135",
>        "error": "returned nothing",
>        "msg": "Look up of {key} {error}"
>      }
>    },
>    {
>      "source": "ipahealthcheck.ipa.trust",
>      "check": "IPATrustCatalogCheck",
>      "result": "ERROR",
>      "uuid": "c478454c-f94c-4089-ade4-7c3bd73d6b65",
>      "when": "20230316153427Z",
>      "duration": "0.127239",
>      "kw": {
>        "key": "domain-status",
>        "error": "CalledProcessError(Command ['/usr/sbin/sssctl',
> 'domain-status', 'gx.x', '--active-server'] returned non-zero exit
> status 1: 'Unable to get online status\\n')",
>        "msg": "Execution of {key} failed: {error}"
>      }
>    }
>  ]
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
>

[Jeremy Tourville]

I have noted that klist and kvno don't match for the keytab I fetched earlier.  Could
this cause issues with named or are those two separate issues?  How do I get them to
match?

[root@gsil-ipa01 data]# klist -ek /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes256-cts-hmac-sha384-192) 
   5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes128-cts-hmac-sha256-128) 
   5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes256-cts-hmac-sha1-96) 
   5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes128-cts-hmac-sha1-96) 

[root@gsil-ipa01 data]# kvno host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL
host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL: kvno = 2

[Rob Crittenden]

Jeremy Tourville via FreeIPA-users wrote:
>  I have noted that klist and kvno don't match for the keytab I
> fetched earlier.  Could this cause issues with named or are those two separate issues? 
> How do I get them to match?
>  
>  [root@gsil-ipa01 data]# klist -ek /etc/krb5.keytab
>  Keytab name: FILE:/etc/krb5.keytab
>  KVNO Principal
>  ---- --------------------------------------------------------------------------
>     5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes256-cts-hmac-sha384-192) 
>     5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes128-cts-hmac-sha256-128) 
>     5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes256-cts-hmac-sha1-96) 
>     5 host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL (aes128-cts-hmac-sha1-96) 
>  
>  [root@gsil-ipa01 data]# kvno host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL
>  host/gsil-ipa01.idm.gsil.smil(a)IDM.GSIL.SMIL: kvno = 2 
Yes, it is basically: the passwords don't match. In a previous e-mail
healthcehck reported that replication wasn't working, that might account
for it.

rob

[Jeremy Tourville]

OK, but how do i get them to match again?  Running  ipa-getkeytab doesn't fix it. 
klist just keeps incrementing and kvno stays the same.

[Florence Blanc-Renaud]

Hi,

On Fri, Mar 17, 2023 at 2:43 PM Jeremy Tourville via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org&gt; wrote:

>  OK, but how do i get them to match again?  Running  ipa-getkeytab
> doesn't
>  fix it.  klist just keeps incrementing and kvno stays the same.
>  
The command ipa-getkeytab creates a new key, unless it is called with
--retrieve (in which case it downloads the existing keys to a keytab file).

In your case, a new key is generated for host/gsil-ipa01.idm.gsil.smil on
server gsil-ipa02.idm.x.x. It means that the new key is updated in the LDAP
entry on gsil-ipa02.idm.x.x. If the replication is broken, the LDAP entry
on gsil-ipa01.idm.gsil.smil still contains the old key, and any kinit
against this server will fail if using the new key.

You need to fix replication first, you may give a try at the command "ipa
topologysegment-reinitialize".

flo

>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>

[Jeremy Tourville]

To make a long story short, we tried that command and it sort of appeared to work but
ultimately failed when running the health check.  Knowing to focus on topology was crucial
in resolving things.  There were several other commands we tried as well but just
couldn't get things back in sync for whatever reason.  Ultimately, we decided to
remove the server altogether and rebuild it again.  This allowed replication to work as
needed once we had all the proper roles installed.  Both servers are healthy and
replicating properly!  :)  Thank you all for your help.