sss_ssh_knownhostsproxy
by Ronald Wimmer
Today, I have seen sss_ssh_knownhostsproxy for the first time. I can see
what it does but I do not get the point why it is needed. I would highly
appreciate if someone could clarify this a little.
Cheers,
Ronald
2 years, 4 months
ipa-client-install error: Failed to obtain host TGT: Major (851968)
by Phinees Garandi
Hello everyone
I encountered a bug while installing freeipa client.
the command fail and I have this as an error message :
`Please make sure the following ports are opened in the firewall settings:
TCP: 80, 88, 389
UDP: 88 (at least one of TCP/UDP ports 88 has to be open)
Also note that following ports are necessary for ipa-client working properly after enrollment:
TCP: 464
UDP: 464, 123 (if NTP enabled)
Failed to obtain host TGT: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2529639107): No credentials cache found
Installation failed. Force set so not rolling back changes.`
This is my command :
ipa-client-install \
--mkhomedir \
--ntp-server=my-ntp-server \
--server=my-ipa-server \
--domain=my-domain \
--realm=MYREALM \
--principal my-user \
--ssh-trust-dns \
--hostname=my-hostname
thank you so much for your help.
2 years, 4 months
Re: ipa-healthcheck: ReplicationCheck ERROR
by Rob Crittenden
Dungan, Scott A. via FreeIPA-users wrote:
> Rob,
>
> That worked!
>
> No errors on healthcheck now. getcert list shows the expected output as well (9 certificates tracked just like on the other two servers).
>
> I am not sure how to look for the LDAP entry for the HTTP service on ipa1 to remove the old certificate blob.
I'd suggest using Apache Studio. It is a more user-friendly LDAP viewer
than ldapmodify/search.
What you're looking for is the userCertificate attribute for the HTTP
service entry. You'll need the dn for this. To get that easily you can run:
ipa service-show --all HTTP/ipa1.<blah>
The dn value will be displayed first.
If you look in /var/lib/ipa/certs/httpd.pem you'll see the
base64-encoded certificate in PEM format. A similar format is stored in
LDAP, just without the headers, footers and newlines.
userCertificate is a multi-valued which should have two certificates.
Remove the one that doesn't match /var/lib/ipa/certs/httpd.pem and you
should be good.
rob
>
> Thanks,
>
> -Scott
>
> -----Original Message-----
> From: Rob Crittenden <rcritten(a)redhat.com>
> Sent: Thursday, December 2, 2021 12:33 PM
> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
> Subject: Re: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck ERROR
>
> Try this. Move /var/lib/ipa/certs/httpd.crt to some place safe.
>
> # getcert resubmit -i 20211130180109 -w -v
>
> You'll be able to watch the progress. It should end up in MONITORING with a new certificate.
>
> Restart httpd.
>
> So now you have a new, valid, known certificate for the service.
>
> You'll need to modify the LDAP entry for this service to remove the original certificate because it is unknown and likely to fail revocation by IPA tools as a result (e.g. uninstallation). Figure out which of the userCertificate blobs is the old cert and remove that value.
>
> rob
>
> Dungan, Scott A. via FreeIPA-users wrote:
>> So, that certificate appears to be the newly installed web cert, signed by the internal ipa CA for the ipa1 server itself:
>>
>> [root(a)ipa1.id.example.com ~]# getcert list -i 20211130180109 Number of
>> certificates and requests being tracked: 9.
>> Request ID '20211130180109':
>> status: MONITORING
>> stuck: no
>> key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa1.id.example.com-443-RSA'
>> certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
>> CA: IPA
>> issuer: CN=Certificate Authority,O=ID.EXAMPLE.COM
>> subject: CN= ipa1.id.example.com,O=ID.EXAMPLE.COM
>> expires: 2023-12-01 10:01:12 PST
>> dns: ipa1.id.example.com,ipa-ca.id.example.com
>> principal name: HTTP/ipa1.id.example.com(a)ID.EXAMPLE.COM
>> key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>> track: yes
>> auto-renew: yes
>>
>> -Scott
>>
>> -----Original Message-----
>> From: Rob Crittenden <rcritten(a)redhat.com>
>> Sent: Thursday, December 2, 2021 5:37 AM
>> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
>> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
>> Subject: Re: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck
>> ERROR
>>
>> So there was some data loss.
>>
>> Re-issuing a cert generally involves revoking the existing one but since that isn't there, this could be interesting.
>>
>> What does the tracking for that cert look like? getcert list -i
>> 20211130180109
>>
>> rob
>>
>> Dungan, Scott A. via FreeIPA-users wrote:
>>> Update: after 20-30 minutes, the ca replication errors stopped, leaving only the new IPACertRevocation error and the 404 code:
>>>
>>> [root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only ...
>>> ra.get_certificate(): Request failed with status 404: Non-2xx
>>> response from CA REST API: 404. Certificate ID 0x13 not found (404) [
>>> {
>>> "source": "ipahealthcheck.ipa.certs",
>>> "check": "IPACertRevocation",
>>> "result": "ERROR",
>>> "uuid": "c447d225-c712-4a77-b174-81f6ba008128",
>>> "when": "20211202025711Z",
>>> "duration": "3.825151",
>>> "kw": {
>>> "key": "20211130180109",
>>> "serial": 19,
>>> "error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404)",
>>> "msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
>>> }
>>> }
>>> ]
>>
>> So there was some data loss.
>>
>> Re-issuing a cert generally involves revoking the existing one but since that isn't there, this could be interesting.
>>
>> What does the tracking for that cert look like? getcert list -i
>> 20211130180109
>>
>> rob
>>>
>>> -----Original Message-----
>>> From: Dungan, Scott A. via FreeIPA-users
>>> <freeipa-users(a)lists.fedorahosted.org>
>>> Sent: Wednesday, December 1, 2021 5:18 PM
>>> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
>>> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
>>> Subject: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck ERROR
>>>
>>> Hi, Rob.
>>>
>>> I think I got ahead of myself and may have made things worse:
>>>
>>> Because servers iap2 and ipa3 are in agreement (ldif matches) and had no healthcheck errors, and ipa1 did, I assumed that ipa1 needed to be re-initialized. The small number of changes that would be lost on ipa1 was/is considered negligible. To re-initialize ipa1, I ran the following:
>>>
>>> [root(a)ipa1.id.example.com ~]# ipa-csreplica-manage re-initialize --from ipa3.id.example.com Directory Manager password:
>>>
>>> Update in progress, 7 seconds elapsed Update succeeded
>>>
>>> Looks good, but when I do a healthcheck again, now I get additional errors as well as the original:
>>>
>>> [root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only ...
>>> ra.get_certificate(): Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404 [
>>> {
>>> "source": "ipahealthcheck.ds.replication",
>>> "check": "ReplicationCheck",
>>> "result": "ERROR",
>>> "uuid": "023f04e2-e8d8-459e-b2cb-bb516e30db07",
>>> "when": "20211202005925Z",
>>> "duration": "0.301820",
>>> "kw": {
>>> "key": "DSREPLLE0003",
>>> "items": [
>>> "Replication",
>>> "Agreement"
>>> ],
>>> "msg": "The replication agreement (catoipa2.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
>>> }
>>> },
>>> {
>>> "source": "ipahealthcheck.ds.replication",
>>> "check": "ReplicationCheck",
>>> "result": "ERROR",
>>> "uuid": "6b7499ce-2138-415d-88ac-bf24f6f3a6db",
>>> "when": "20211202005925Z",
>>> "duration": "0.301851",
>>> "kw": {
>>> "key": "DSREPLLE0003",
>>> "items": [
>>> "Replication",
>>> "Agreement"
>>> ],
>>> "msg": "The replication agreement (catoipa3.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
>>> }
>>> },
>>> {
>>> "source": "ipahealthcheck.ipa.certs",
>>> "check": "IPACertRevocation",
>>> "result": "ERROR",
>>> "uuid": "b02a8f85-985f-4b80-a099-fb3d415b2005",
>>> "when": "20211202005935Z",
>>> "duration": "2.663208",
>>> "kw": {
>>> "key": "20211130180109",
>>> "serial": 19,
>>> "error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 40 Certificate ID 0x13 not found (404)",
>>> "msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
>>> }
>>> }
>>> ]
>>>
>>> -----Original Message-----
>>> From: Rob Crittenden <rcritten(a)redhat.com>
>>> Sent: Wednesday, December 1, 2021 11:50 AM
>>> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
>>> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
>>> Subject: Re: [Freeipa-users] ipa-healthcheck: ReplicationCheck ERROR
>>>
>>> Dungan, Scott A. via FreeIPA-users wrote:
>>>> We have 3 ipa servers, one of which is throwing an ERROR condition
>>>> during ipa-healthcheck for the "ReplicationCheck" test.
>>>> Ipa-healthcheck shows no errors when run from the other two replicas.
>>>> Looking back at the logs, it appears this started about ten days
>>>> ago, so it is not a transient issue as the output suggests:
>>>>
>>>>
>>>>
>>>> [root(a)ipa1.id.example.com]# ipa-healthcheck --failures-only
>>>>
>>>>
>>>>
>>>> [
>>>>
>>>> {
>>>>
>>>> "source": "ipahealthcheck.ds.replication",
>>>>
>>>> "check": "ReplicationCheck",
>>>>
>>>> "result": "ERROR",
>>>>
>>>> "uuid": "2b971ca3-678e-4c26-86a0-5b352027e7e8",
>>>>
>>>> "when": "20211201180013Z",
>>>>
>>>> "duration": "0.687812",
>>>>
>>>> "kw": {
>>>>
>>>> "key": "DSREPLLE0003",
>>>>
>>>> "items": [
>>>>
>>>> "Replication",
>>>>
>>>> "Agreement"
>>>>
>>>> ],
>>>>
>>>> "msg": "The replication agreement (catoipa2.id.example.com)
>>>> under \"o=ipaca\" is not in synchronization.\nStatus message: error
>>>> (18) can't acquire replica (incremental update transient warning.
>>>> backing off, will retry update later.)"
>>>>
>>>> }
>>>>
>>>> },
>>>>
>>>> {
>>>>
>>>> "source": "ipahealthcheck.ds.replication",
>>>>
>>>> "check": "ReplicationCheck",
>>>>
>>>> "result": "ERROR",
>>>>
>>>> "uuid": "99436870-bc98-4ce8-84b1-c0b0806945c8",
>>>>
>>>> "when": "20211201180013Z",
>>>>
>>>> "duration": "0.687829",
>>>>
>>>> "kw": {
>>>>
>>>> "key": "DSREPLLE0003",
>>>>
>>>> "items": [
>>>>
>>>> "Replication",
>>>>
>>>> "Agreement"
>>>>
>>>> ],
>>>>
>>>> "msg": "The replication agreement (catoipa3.id.example.com)
>>>> under \"o=ipaca\" is not in synchronization.\nStatus message: error
>>>> (18) can't acquire replica (incremental update transient warning.
>>>> backing off, will retry update later.)"
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> ]
>>>>
>>>>
>>>>
>>>> 389-ds error logs show a slew of these:
>>>>
>>>>
>>>>
>>>> [30/Nov/2021:23:41:35.277399980 -0800] - ERR - NSMMReplicationPlugin
>>>> - send_updates - agmt="cn=caToipa3.id.example.com" (ipa2:389):
>>>> Missing data encountered. If the error persists the replica must be reinitialized.
>>>>
>>>> [30/Nov/2021:23:41:38.288003253 -0800] - ERR -
>>>> agmt="cn=caToipa3.id.example.com" (ipa3:389) - clcache_load_buffer -
>>>> Can't locate CSN 6197e149000000060000 in the changelog (DB rc=-30988).
>>>> If replication stops, the consumer may need to be reinitialized.
>>>>
>>>> [30/Nov/2021:23:41:38.289713999 -0800] - ERR - NSMMReplicationPlugin
>>>> - send_updates - agmt="cn=caToipa3.id.example.com" (ipa3:389):
>>>> Missing data encountered. If the error persists the replica must be reinitialized.
>>>>
>>>>
>>>>
>>>> That would seem to suggest running a "ipa-replica-manage
>>>> re-initialize --from $SERVER_TO_PULL_FROM" may resolve the issue,
>>>> but before we try that, is there anything else we should look at?
>>>
>>> You use ipa-csreplica-manage to manage the CA replication agreements.
>>> But yes, it looks like you need to re-initialize some of them.
>>>
>>> I'd suggest dump the ldif of the two to be re-inited and see if there are any entries there not recorded in the one you will re-init from to see if there is any potential data loss.
>>>
>>> rob
>>> _______________________________________________
>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>> To unsubscribe send an email to
>>> freeipa-users-leave(a)lists.fedorahosted.org
>>> Fedora Code of Conduct:
>>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>> List Guidelines:
>>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>>> List Archives:
>>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedo
>>> r ahosted.org Do not reply to spam on the list, report it:
>>> https://pagure.io/fedora-infrastructure
>>> _______________________________________________
>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>> To unsubscribe send an email to
>>> freeipa-users-leave(a)lists.fedorahosted.org
>>> Fedora Code of Conduct:
>>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>> List Guidelines:
>>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>>> List Archives:
>>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedo
>>> r ahosted.org Do not reply to spam on the list, report it:
>>> https://pagure.io/fedora-infrastructure
>>>
>>
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to
>> freeipa-users-leave(a)lists.fedorahosted.org
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedor
>> ahosted.org Do not reply to spam on the list, report it:
>> https://pagure.io/fedora-infrastructure
>>
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>
2 years, 4 months
Re: ipa-healthcheck: ReplicationCheck ERROR
by Rob Crittenden
Try this. Move /var/lib/ipa/certs/httpd.crt to some place safe.
# getcert resubmit -i 20211130180109 -w -v
You'll be able to watch the progress. It should end up in MONITORING
with a new certificate.
Restart httpd.
So now you have a new, valid, known certificate for the service.
You'll need to modify the LDAP entry for this service to remove the
original certificate because it is unknown and likely to fail revocation
by IPA tools as a result (e.g. uninstallation). Figure out which of the
userCertificate blobs is the old cert and remove that value.
rob
Dungan, Scott A. via FreeIPA-users wrote:
> So, that certificate appears to be the newly installed web cert, signed by the internal ipa CA for the ipa1 server itself:
>
> [root(a)ipa1.id.example.com ~]# getcert list -i 20211130180109
> Number of certificates and requests being tracked: 9.
> Request ID '20211130180109':
> status: MONITORING
> stuck: no
> key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa1.id.example.com-443-RSA'
> certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
> CA: IPA
> issuer: CN=Certificate Authority,O=ID.EXAMPLE.COM
> subject: CN= ipa1.id.example.com,O=ID.EXAMPLE.COM
> expires: 2023-12-01 10:01:12 PST
> dns: ipa1.id.example.com,ipa-ca.id.example.com
> principal name: HTTP/ipa1.id.example.com(a)ID.EXAMPLE.COM
> key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> eku: id-kp-serverAuth,id-kp-clientAuth
> pre-save command:
> post-save command: /usr/libexec/ipa/certmonger/restart_httpd
> track: yes
> auto-renew: yes
>
> -Scott
>
> -----Original Message-----
> From: Rob Crittenden <rcritten(a)redhat.com>
> Sent: Thursday, December 2, 2021 5:37 AM
> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
> Subject: Re: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck ERROR
>
> So there was some data loss.
>
> Re-issuing a cert generally involves revoking the existing one but since that isn't there, this could be interesting.
>
> What does the tracking for that cert look like? getcert list -i
> 20211130180109
>
> rob
>
> Dungan, Scott A. via FreeIPA-users wrote:
>> Update: after 20-30 minutes, the ca replication errors stopped, leaving only the new IPACertRevocation error and the 404 code:
>>
>> [root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only ...
>> ra.get_certificate(): Request failed with status 404: Non-2xx response
>> from CA REST API: 404. Certificate ID 0x13 not found (404) [
>> {
>> "source": "ipahealthcheck.ipa.certs",
>> "check": "IPACertRevocation",
>> "result": "ERROR",
>> "uuid": "c447d225-c712-4a77-b174-81f6ba008128",
>> "when": "20211202025711Z",
>> "duration": "3.825151",
>> "kw": {
>> "key": "20211130180109",
>> "serial": 19,
>> "error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404)",
>> "msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
>> }
>> }
>> ]
>
> So there was some data loss.
>
> Re-issuing a cert generally involves revoking the existing one but since that isn't there, this could be interesting.
>
> What does the tracking for that cert look like? getcert list -i
> 20211130180109
>
> rob
>>
>> -----Original Message-----
>> From: Dungan, Scott A. via FreeIPA-users
>> <freeipa-users(a)lists.fedorahosted.org>
>> Sent: Wednesday, December 1, 2021 5:18 PM
>> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
>> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
>> Subject: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck ERROR
>>
>> Hi, Rob.
>>
>> I think I got ahead of myself and may have made things worse:
>>
>> Because servers iap2 and ipa3 are in agreement (ldif matches) and had no healthcheck errors, and ipa1 did, I assumed that ipa1 needed to be re-initialized. The small number of changes that would be lost on ipa1 was/is considered negligible. To re-initialize ipa1, I ran the following:
>>
>> [root(a)ipa1.id.example.com ~]# ipa-csreplica-manage re-initialize --from ipa3.id.example.com Directory Manager password:
>>
>> Update in progress, 7 seconds elapsed
>> Update succeeded
>>
>> Looks good, but when I do a healthcheck again, now I get additional errors as well as the original:
>>
>> [root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only ...
>> ra.get_certificate(): Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404 [
>> {
>> "source": "ipahealthcheck.ds.replication",
>> "check": "ReplicationCheck",
>> "result": "ERROR",
>> "uuid": "023f04e2-e8d8-459e-b2cb-bb516e30db07",
>> "when": "20211202005925Z",
>> "duration": "0.301820",
>> "kw": {
>> "key": "DSREPLLE0003",
>> "items": [
>> "Replication",
>> "Agreement"
>> ],
>> "msg": "The replication agreement (catoipa2.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
>> }
>> },
>> {
>> "source": "ipahealthcheck.ds.replication",
>> "check": "ReplicationCheck",
>> "result": "ERROR",
>> "uuid": "6b7499ce-2138-415d-88ac-bf24f6f3a6db",
>> "when": "20211202005925Z",
>> "duration": "0.301851",
>> "kw": {
>> "key": "DSREPLLE0003",
>> "items": [
>> "Replication",
>> "Agreement"
>> ],
>> "msg": "The replication agreement (catoipa3.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
>> }
>> },
>> {
>> "source": "ipahealthcheck.ipa.certs",
>> "check": "IPACertRevocation",
>> "result": "ERROR",
>> "uuid": "b02a8f85-985f-4b80-a099-fb3d415b2005",
>> "when": "20211202005935Z",
>> "duration": "2.663208",
>> "kw": {
>> "key": "20211130180109",
>> "serial": 19,
>> "error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 40 Certificate ID 0x13 not found (404)",
>> "msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
>> }
>> }
>> ]
>>
>> -----Original Message-----
>> From: Rob Crittenden <rcritten(a)redhat.com>
>> Sent: Wednesday, December 1, 2021 11:50 AM
>> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
>> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
>> Subject: Re: [Freeipa-users] ipa-healthcheck: ReplicationCheck ERROR
>>
>> Dungan, Scott A. via FreeIPA-users wrote:
>>> We have 3 ipa servers, one of which is throwing an ERROR condition
>>> during ipa-healthcheck for the "ReplicationCheck" test.
>>> Ipa-healthcheck shows no errors when run from the other two replicas.
>>> Looking back at the logs, it appears this started about ten days ago,
>>> so it is not a transient issue as the output suggests:
>>>
>>>
>>>
>>> [root(a)ipa1.id.example.com]# ipa-healthcheck --failures-only
>>>
>>>
>>>
>>> [
>>>
>>> {
>>>
>>> "source": "ipahealthcheck.ds.replication",
>>>
>>> "check": "ReplicationCheck",
>>>
>>> "result": "ERROR",
>>>
>>> "uuid": "2b971ca3-678e-4c26-86a0-5b352027e7e8",
>>>
>>> "when": "20211201180013Z",
>>>
>>> "duration": "0.687812",
>>>
>>> "kw": {
>>>
>>> "key": "DSREPLLE0003",
>>>
>>> "items": [
>>>
>>> "Replication",
>>>
>>> "Agreement"
>>>
>>> ],
>>>
>>> "msg": "The replication agreement (catoipa2.id.example.com)
>>> under \"o=ipaca\" is not in synchronization.\nStatus message: error
>>> (18) can't acquire replica (incremental update transient warning.
>>> backing off, will retry update later.)"
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "source": "ipahealthcheck.ds.replication",
>>>
>>> "check": "ReplicationCheck",
>>>
>>> "result": "ERROR",
>>>
>>> "uuid": "99436870-bc98-4ce8-84b1-c0b0806945c8",
>>>
>>> "when": "20211201180013Z",
>>>
>>> "duration": "0.687829",
>>>
>>> "kw": {
>>>
>>> "key": "DSREPLLE0003",
>>>
>>> "items": [
>>>
>>> "Replication",
>>>
>>> "Agreement"
>>>
>>> ],
>>>
>>> "msg": "The replication agreement (catoipa3.id.example.com)
>>> under \"o=ipaca\" is not in synchronization.\nStatus message: error
>>> (18) can't acquire replica (incremental update transient warning.
>>> backing off, will retry update later.)"
>>>
>>> }
>>>
>>> }
>>>
>>> ]
>>>
>>>
>>>
>>> 389-ds error logs show a slew of these:
>>>
>>>
>>>
>>> [30/Nov/2021:23:41:35.277399980 -0800] - ERR - NSMMReplicationPlugin
>>> - send_updates - agmt="cn=caToipa3.id.example.com" (ipa2:389):
>>> Missing data encountered. If the error persists the replica must be reinitialized.
>>>
>>> [30/Nov/2021:23:41:38.288003253 -0800] - ERR -
>>> agmt="cn=caToipa3.id.example.com" (ipa3:389) - clcache_load_buffer -
>>> Can't locate CSN 6197e149000000060000 in the changelog (DB rc=-30988).
>>> If replication stops, the consumer may need to be reinitialized.
>>>
>>> [30/Nov/2021:23:41:38.289713999 -0800] - ERR - NSMMReplicationPlugin
>>> - send_updates - agmt="cn=caToipa3.id.example.com" (ipa3:389):
>>> Missing data encountered. If the error persists the replica must be reinitialized.
>>>
>>>
>>>
>>> That would seem to suggest running a "ipa-replica-manage
>>> re-initialize --from $SERVER_TO_PULL_FROM" may resolve the issue, but
>>> before we try that, is there anything else we should look at?
>>
>> You use ipa-csreplica-manage to manage the CA replication agreements.
>> But yes, it looks like you need to re-initialize some of them.
>>
>> I'd suggest dump the ldif of the two to be re-inited and see if there are any entries there not recorded in the one you will re-init from to see if there is any potential data loss.
>>
>> rob
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to
>> freeipa-users-leave(a)lists.fedorahosted.org
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedor
>> ahosted.org Do not reply to spam on the list, report it:
>> https://pagure.io/fedora-infrastructure
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to
>> freeipa-users-leave(a)lists.fedorahosted.org
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedor
>> ahosted.org Do not reply to spam on the list, report it:
>> https://pagure.io/fedora-infrastructure
>>
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>
2 years, 4 months
Trusted AD users cannot login
by tizo
We have a test environment with a FreeIPA server with a cross forest trust
with an AD (that is in fact a Samba AD DC). Both servers are Rocky Linux 8.
Everything works fine when we try to login to the FreeIPA server with an AD
user (and with IPA users too). However, in another Rocky Linux 8 acting as
an IPA client, we cannot do that. In this case, we can login with IPA users
(admin for example), but we cannot login with AD users.
More details:
* "id userad(a)ad.xx.xx" and "getent passwd user(a)ad.xx.xx" are not working
in IPA client.
* Both are working for IPA users in IPA client.
* "kinit userad(a)ad.xx.xx" is working in IPA client. It is also working for
IPA users.
* Everything is working on IPA server.
Any help is appreciated,
tizo
2 years, 4 months
IPA<->AD performance
by Ronald Wimmer
We do have several hundred IPA clients at the moment. People are advised
to use their AD users when connecting to an IPA client via SSH.
Colleagues are reporting constantly that uncached logins are taking way
to long. I did even see colleagues returning to creating local user
accounts...
I am aware of the tuning guide
https://jhrozek.wordpress.com/2015/08/19/performance-tuning-sssd-for-larg...
. We have implemented all tunables described there on the server side.
We have tens of thousands of users in our AD. Maybe the whole AD<->IPA
setup should be implemented in a completely different way?
Please help me to further analyze and tune our IPA installation in order
to minimize SSH login times.
Cheers,
Ronald
2 years, 4 months
Mixed case user names
by Jim Kinney
I've seen that lower case is the norm for freeipa and idm, and with good reasoning.
However, a conversion project is using mixed case by choice in an existing openldap process. Users have done "user things" like hard coded paths with their mixed case names.
User names are assigned by an AD process but the openldap and now freeipa will not be sharing anything with the AD setup.
Is there a flag that can be set somewhere to allow username creation as entered? Or should I look at doing a direct ldif import (yuck!).
Currently in testing with a rocky 8 vm and IdM and customer proof on concept phase and final deploy to be RHEL/IdM.
--
Computers amplify human error
Super computers are really cool
2 years, 4 months
Re: ipa-healthcheck: ReplicationCheck ERROR
by Rob Crittenden
Dungan, Scott A. via FreeIPA-users wrote:
> Update: after 20-30 minutes, the ca replication errors stopped, leaving only the new IPACertRevocation error and the 404 code:
>
> [root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only
> ...
> ra.get_certificate(): Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404)
> [
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPACertRevocation",
> "result": "ERROR",
> "uuid": "c447d225-c712-4a77-b174-81f6ba008128",
> "when": "20211202025711Z",
> "duration": "3.825151",
> "kw": {
> "key": "20211130180109",
> "serial": 19,
> "error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404)",
> "msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
> }
> }
> ]
So there was some data loss.
Re-issuing a cert generally involves revoking the existing one but since
that isn't there, this could be interesting.
What does the tracking for that cert look like? getcert list -i
20211130180109
rob
>
> -----Original Message-----
> From: Dungan, Scott A. via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org>
> Sent: Wednesday, December 1, 2021 5:18 PM
> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
> Subject: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck ERROR
>
> Hi, Rob.
>
> I think I got ahead of myself and may have made things worse:
>
> Because servers iap2 and ipa3 are in agreement (ldif matches) and had no healthcheck errors, and ipa1 did, I assumed that ipa1 needed to be re-initialized. The small number of changes that would be lost on ipa1 was/is considered negligible. To re-initialize ipa1, I ran the following:
>
> [root(a)ipa1.id.example.com ~]# ipa-csreplica-manage re-initialize --from ipa3.id.example.com Directory Manager password:
>
> Update in progress, 7 seconds elapsed
> Update succeeded
>
> Looks good, but when I do a healthcheck again, now I get additional errors as well as the original:
>
> [root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only ...
> ra.get_certificate(): Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404 [
> {
> "source": "ipahealthcheck.ds.replication",
> "check": "ReplicationCheck",
> "result": "ERROR",
> "uuid": "023f04e2-e8d8-459e-b2cb-bb516e30db07",
> "when": "20211202005925Z",
> "duration": "0.301820",
> "kw": {
> "key": "DSREPLLE0003",
> "items": [
> "Replication",
> "Agreement"
> ],
> "msg": "The replication agreement (catoipa2.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
> }
> },
> {
> "source": "ipahealthcheck.ds.replication",
> "check": "ReplicationCheck",
> "result": "ERROR",
> "uuid": "6b7499ce-2138-415d-88ac-bf24f6f3a6db",
> "when": "20211202005925Z",
> "duration": "0.301851",
> "kw": {
> "key": "DSREPLLE0003",
> "items": [
> "Replication",
> "Agreement"
> ],
> "msg": "The replication agreement (catoipa3.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
> }
> },
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPACertRevocation",
> "result": "ERROR",
> "uuid": "b02a8f85-985f-4b80-a099-fb3d415b2005",
> "when": "20211202005935Z",
> "duration": "2.663208",
> "kw": {
> "key": "20211130180109",
> "serial": 19,
> "error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 40 Certificate ID 0x13 not found (404)",
> "msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
> }
> }
> ]
>
> -----Original Message-----
> From: Rob Crittenden <rcritten(a)redhat.com>
> Sent: Wednesday, December 1, 2021 11:50 AM
> To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
> Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
> Subject: Re: [Freeipa-users] ipa-healthcheck: ReplicationCheck ERROR
>
> Dungan, Scott A. via FreeIPA-users wrote:
>> We have 3 ipa servers, one of which is throwing an ERROR condition
>> during ipa-healthcheck for the "ReplicationCheck" test.
>> Ipa-healthcheck shows no errors when run from the other two replicas.
>> Looking back at the logs, it appears this started about ten days ago,
>> so it is not a transient issue as the output suggests:
>>
>>
>>
>> [root(a)ipa1.id.example.com]# ipa-healthcheck --failures-only
>>
>>
>>
>> [
>>
>> {
>>
>> "source": "ipahealthcheck.ds.replication",
>>
>> "check": "ReplicationCheck",
>>
>> "result": "ERROR",
>>
>> "uuid": "2b971ca3-678e-4c26-86a0-5b352027e7e8",
>>
>> "when": "20211201180013Z",
>>
>> "duration": "0.687812",
>>
>> "kw": {
>>
>> "key": "DSREPLLE0003",
>>
>> "items": [
>>
>> "Replication",
>>
>> "Agreement"
>>
>> ],
>>
>> "msg": "The replication agreement (catoipa2.id.example.com)
>> under \"o=ipaca\" is not in synchronization.\nStatus message: error
>> (18) can't acquire replica (incremental update transient warning.
>> backing off, will retry update later.)"
>>
>> }
>>
>> },
>>
>> {
>>
>> "source": "ipahealthcheck.ds.replication",
>>
>> "check": "ReplicationCheck",
>>
>> "result": "ERROR",
>>
>> "uuid": "99436870-bc98-4ce8-84b1-c0b0806945c8",
>>
>> "when": "20211201180013Z",
>>
>> "duration": "0.687829",
>>
>> "kw": {
>>
>> "key": "DSREPLLE0003",
>>
>> "items": [
>>
>> "Replication",
>>
>> "Agreement"
>>
>> ],
>>
>> "msg": "The replication agreement (catoipa3.id.example.com)
>> under \"o=ipaca\" is not in synchronization.\nStatus message: error
>> (18) can't acquire replica (incremental update transient warning.
>> backing off, will retry update later.)"
>>
>> }
>>
>> }
>>
>> ]
>>
>>
>>
>> 389-ds error logs show a slew of these:
>>
>>
>>
>> [30/Nov/2021:23:41:35.277399980 -0800] - ERR - NSMMReplicationPlugin -
>> send_updates - agmt="cn=caToipa3.id.example.com" (ipa2:389): Missing
>> data encountered. If the error persists the replica must be reinitialized.
>>
>> [30/Nov/2021:23:41:38.288003253 -0800] - ERR -
>> agmt="cn=caToipa3.id.example.com" (ipa3:389) - clcache_load_buffer -
>> Can't locate CSN 6197e149000000060000 in the changelog (DB rc=-30988).
>> If replication stops, the consumer may need to be reinitialized.
>>
>> [30/Nov/2021:23:41:38.289713999 -0800] - ERR - NSMMReplicationPlugin -
>> send_updates - agmt="cn=caToipa3.id.example.com" (ipa3:389): Missing
>> data encountered. If the error persists the replica must be reinitialized.
>>
>>
>>
>> That would seem to suggest running a "ipa-replica-manage re-initialize
>> --from $SERVER_TO_PULL_FROM" may resolve the issue, but before we try
>> that, is there anything else we should look at?
>
> You use ipa-csreplica-manage to manage the CA replication agreements.
> But yes, it looks like you need to re-initialize some of them.
>
> I'd suggest dump the ldif of the two to be re-inited and see if there are any entries there not recorded in the one you will re-init from to see if there is any potential data loss.
>
> rob
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>
2 years, 4 months
Certificate Authority master cert shows revoked somehow (refresh issue due bad hosts file?)
by Andrius Jurkus
ipa --version
VERSION: 4.8.7
So I have no idea what actually happened, but today I cant login into freeipa webui.
There is 3 instances on 2 and 3rd I can login. Sync is on and working.
I noticed that in webui it shows that first certificate
CN=Certificate Authority,O=xx ipa REVOKED
Obviously I didint revoke it.
ipa-getcert list shows only two certificates and noone is master
getcert list shows more and
this is master cert as you can see server was installed ~2 years ago.
Request ID '20201218102240':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=xxx
subject: CN=Certificate Authority,O=xxx
expires: 2039-12-16 13:38:03 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
CN=localhost... :facepalm:
This is I guess new(not really) and probably this was tickint time bomb for quite some time.
Request ID '20210317111052':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=INT.O4.LT
subject: CN=localhost
expires: 2041-03-17 11:11:01 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
profile: caCACert
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09"
track: yes
auto-renew: yes
I have bad feeling that it is because hosts file is not cleared from localhost entries?
cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
ipa-healthcheck complainas about
this is SUBCA caSigningCert cert-pki-ca 86623fac-5d11-4ac6-9972-ea80ef16f711 not found, assuming 3rd party
{
"source": "ipahealthcheck.ipa.certs",
"check": "IPACertfileExpirationCheck",
"result": "ERROR",
"uuid": "a77e06d1-ee90-4574-8c68-750ec3f5d6bd",
"when": "20211124184538Z",
"duration": "0.366900",
"kw": {
"key": "20210317111052",
"dbdir": "/etc/pki/pki-tomcat/alias",
"nickname": "caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09",
"error": "Failed to get caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09",
"msg": "Unable to retrieve cert 'caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09' from '/etc/pki/pki-tomcat/alias': Failed to get caSigningCert cert-pki-ca bd372c7d-62d0-469d-a107-da3b5a782c09"
}
},
{
"source": "ipahealthcheck.ipa.certs",
"check": "IPACertTracking",
"result": "ERROR",
"uuid": "c8e78e6d-794c-41cb-a12f-f57d0dda23d6",
"when": "20211124184538Z",
"duration": "0.416846",
"kw": {
"key": "cert-database=/etc/pki/pki-tomcat/alias, cert-nickname=caSigningCert cert-pki-ca, ca-name=dogtag-ipa-ca-renew-agent, cert-presave-command=/usr/libexec/ipa/certmonger/stop_pkicad, cert-postsave-command=/usr/libexec/ipa/certmonger/renew_ca_cert \"caSigningCer
t cert-pki-ca\", template-profile=caCACert",
"msg": "Missing tracking for cert-database=/etc/pki/pki-tomcat/alias, cert-nickname=caSigningCert cert-pki-ca, ca-name=dogtag-ipa-ca-renew-agent, cert-presave-command=/usr/libexec/ipa/certmonger/stop_pkicad, cert-postsave-command=/usr/libexec/ipa/certmonger/renew_c
a_cert \"caSigningCert cert-pki-ca\", template-profile=caCACert"
}
},
and more
question how I can fix this?
2 years, 4 months
Re: ipa-healthcheck: ReplicationCheck ERROR
by Dungan, Scott A.
Update: after 20-30 minutes, the ca replication errors stopped, leaving only the new IPACertRevocation error and the 404 code:
[root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only
...
ra.get_certificate(): Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404)
[
{
"source": "ipahealthcheck.ipa.certs",
"check": "IPACertRevocation",
"result": "ERROR",
"uuid": "c447d225-c712-4a77-b174-81f6ba008128",
"when": "20211202025711Z",
"duration": "3.825151",
"kw": {
"key": "20211130180109",
"serial": 19,
"error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404)",
"msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
}
}
]
-----Original Message-----
From: Dungan, Scott A. via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org>
Sent: Wednesday, December 1, 2021 5:18 PM
To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
Subject: [Freeipa-users] Re: ipa-healthcheck: ReplicationCheck ERROR
Hi, Rob.
I think I got ahead of myself and may have made things worse:
Because servers iap2 and ipa3 are in agreement (ldif matches) and had no healthcheck errors, and ipa1 did, I assumed that ipa1 needed to be re-initialized. The small number of changes that would be lost on ipa1 was/is considered negligible. To re-initialize ipa1, I ran the following:
[root(a)ipa1.id.example.com ~]# ipa-csreplica-manage re-initialize --from ipa3.id.example.com Directory Manager password:
Update in progress, 7 seconds elapsed
Update succeeded
Looks good, but when I do a healthcheck again, now I get additional errors as well as the original:
[root(a)ipa1.id.example.com ~]# ipa-healthcheck --failures-only ...
ra.get_certificate(): Request failed with status 404: Non-2xx response from CA REST API: 404. Certificate ID 0x13 not found (404 [
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "ERROR",
"uuid": "023f04e2-e8d8-459e-b2cb-bb516e30db07",
"when": "20211202005925Z",
"duration": "0.301820",
"kw": {
"key": "DSREPLLE0003",
"items": [
"Replication",
"Agreement"
],
"msg": "The replication agreement (catoipa2.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
}
},
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "ERROR",
"uuid": "6b7499ce-2138-415d-88ac-bf24f6f3a6db",
"when": "20211202005925Z",
"duration": "0.301851",
"kw": {
"key": "DSREPLLE0003",
"items": [
"Replication",
"Agreement"
],
"msg": "The replication agreement (catoipa3.id.example.com) under \"o=ipaca\" is not in synchronization.\nStatus messa: error (18) can't acquire replica (incremental update transient warning. backing off, will retry update later.)"
}
},
{
"source": "ipahealthcheck.ipa.certs",
"check": "IPACertRevocation",
"result": "ERROR",
"uuid": "b02a8f85-985f-4b80-a099-fb3d415b2005",
"when": "20211202005935Z",
"duration": "2.663208",
"kw": {
"key": "20211130180109",
"serial": 19,
"error": "Certificate operation cannot be completed: Request failed with status 404: Non-2xx response from CA REST API: 40 Certificate ID 0x13 not found (404)",
"msg": "Request for certificate serial number {serial} in request {key} failed: {error}"
}
}
]
-----Original Message-----
From: Rob Crittenden <rcritten(a)redhat.com>
Sent: Wednesday, December 1, 2021 11:50 AM
To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
Cc: Dungan, Scott A. <sdungan(a)caltech.edu>
Subject: Re: [Freeipa-users] ipa-healthcheck: ReplicationCheck ERROR
Dungan, Scott A. via FreeIPA-users wrote:
> We have 3 ipa servers, one of which is throwing an ERROR condition
> during ipa-healthcheck for the "ReplicationCheck" test.
> Ipa-healthcheck shows no errors when run from the other two replicas.
> Looking back at the logs, it appears this started about ten days ago,
> so it is not a transient issue as the output suggests:
>
>
>
> [root(a)ipa1.id.example.com]# ipa-healthcheck --failures-only
>
>
>
> [
>
> {
>
> "source": "ipahealthcheck.ds.replication",
>
> "check": "ReplicationCheck",
>
> "result": "ERROR",
>
> "uuid": "2b971ca3-678e-4c26-86a0-5b352027e7e8",
>
> "when": "20211201180013Z",
>
> "duration": "0.687812",
>
> "kw": {
>
> "key": "DSREPLLE0003",
>
> "items": [
>
> "Replication",
>
> "Agreement"
>
> ],
>
> "msg": "The replication agreement (catoipa2.id.example.com)
> under \"o=ipaca\" is not in synchronization.\nStatus message: error
> (18) can't acquire replica (incremental update transient warning.
> backing off, will retry update later.)"
>
> }
>
> },
>
> {
>
> "source": "ipahealthcheck.ds.replication",
>
> "check": "ReplicationCheck",
>
> "result": "ERROR",
>
> "uuid": "99436870-bc98-4ce8-84b1-c0b0806945c8",
>
> "when": "20211201180013Z",
>
> "duration": "0.687829",
>
> "kw": {
>
> "key": "DSREPLLE0003",
>
> "items": [
>
> "Replication",
>
> "Agreement"
>
> ],
>
> "msg": "The replication agreement (catoipa3.id.example.com)
> under \"o=ipaca\" is not in synchronization.\nStatus message: error
> (18) can't acquire replica (incremental update transient warning.
> backing off, will retry update later.)"
>
> }
>
> }
>
> ]
>
>
>
> 389-ds error logs show a slew of these:
>
>
>
> [30/Nov/2021:23:41:35.277399980 -0800] - ERR - NSMMReplicationPlugin -
> send_updates - agmt="cn=caToipa3.id.example.com" (ipa2:389): Missing
> data encountered. If the error persists the replica must be reinitialized.
>
> [30/Nov/2021:23:41:38.288003253 -0800] - ERR -
> agmt="cn=caToipa3.id.example.com" (ipa3:389) - clcache_load_buffer -
> Can't locate CSN 6197e149000000060000 in the changelog (DB rc=-30988).
> If replication stops, the consumer may need to be reinitialized.
>
> [30/Nov/2021:23:41:38.289713999 -0800] - ERR - NSMMReplicationPlugin -
> send_updates - agmt="cn=caToipa3.id.example.com" (ipa3:389): Missing
> data encountered. If the error persists the replica must be reinitialized.
>
>
>
> That would seem to suggest running a "ipa-replica-manage re-initialize
> --from $SERVER_TO_PULL_FROM" may resolve the issue, but before we try
> that, is there anything else we should look at?
You use ipa-csreplica-manage to manage the CA replication agreements.
But yes, it looks like you need to re-initialize some of them.
I'd suggest dump the ldif of the two to be re-inited and see if there are any entries there not recorded in the one you will re-init from to see if there is any potential data loss.
rob
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
2 years, 4 months