dirsrv hangs soon after reboot
by Kees Bakker
Hey,
I'm looking for advice how to analyse/debug this.
On one of the masters the dirsrv is unresponsive. It runs, but every
attempt to connect it hangs.
The command "systemctl status" does not show anything alarming
● dirsrv(a)EXAMPLE-COM.service - 389 Directory Server EXAMPLE-COM.
Loaded: loaded (/usr/lib/systemd/system/dirsrv@.service; enabled; vendor preset: disabled)
Active: active (running) since vr 2020-04-17 13:46:25 CEST; 1h 33min ago
Process: 3123 ExecStartPre=/usr/sbin/ds_systemd_ask_password_acl /etc/dirsrv/slapd-%i/dse.ldif (code=exited, status=0/SUCCESS)
Main PID: 3134 (ns-slapd)
Status: "slapd started: Ready to process requests"
CGroup: /system.slice/system-dirsrv.slice/dirsrv(a)EXAMPLE-COM.service
└─3134 /usr/sbin/ns-slapd -D /etc/dirsrv/slapd-EXAMPLE-COM -i /var/run/dirsrv/slapd-EXAMPLE-COM.pid
apr 17 15:13:54 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:13:54 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:13:54 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:13:54 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:13:54 linge.example.com ns-slapd[3134]: GSSAPI client step 2
apr 17 15:18:54 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:18:54 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:18:55 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:18:55 linge.example.com ns-slapd[3134]: GSSAPI client step 1
apr 17 15:18:55 linge.example.com ns-slapd[3134]: GSSAPI client step 2
However, an ldapsearch command hangs forever
[root@rotte ~]# ldapsearch -H ldaps://linge.example.com -D uid=keesbtest,cn=users,cn=accounts,dc=example,dc=com -W -LLL -o ldif-wrap=no -b cn=users,cn=accounts,dc=example,dc=com '(&(objectClass=person)(memberOf=cn=admins,cn=groups,cn=accounts,dc=example,dc=com))' uid
Enter LDAP Password:
Even if I use the socket (ldapi://%2fvar%2frun%2fslapd-EXAMPLE-COM.socket) the ldapsearch
command hangs.
"ipactl status" hangs
"kinit" hangs
--
Kees Bakker
2 months, 1 week
Unable to install ipa client centos 7.5.1804 (Core)
by William Graboyes
Hello List,
I have been searching around for the day and have found an answer for
the error I am getting when I am trying to install the client on a brand
new install:
Version:
ipa-client-4.5.4-10.el7.centos.3.x86_64
ipa-client-common-4.5.4-10.el7.centos.3.noarch
The error is below (run as root, not via sudo):
ipa-client-install
Traceback (most recent call last):
File "/sbin/ipa-client-install", line 22, in <module>
from ipaclient.install import ipa_client_install
File
"/usr/lib/python2.7/site-packages/ipaclient/install/ipa_client_install.py",
line 5, in <module>
from ipaclient.install import client
File "/usr/lib/python2.7/site-packages/ipaclient/install/client.py",
line 34, in <module>
from ipalib import api, errors, x509
File "/usr/lib/python2.7/site-packages/ipalib/x509.py", line 45, in
<module>
from pyasn1_modules import rfc2315, rfc2459
File "/usr/lib/python2.7/site-packages/pyasn1_modules/rfc2315.py",
line 67, in <module>
class DigestedData(univ.Sequence):
File "/usr/lib/python2.7/site-packages/pyasn1_modules/rfc2315.py",
line 72, in DigestedData
namedtype.NamedType('digest', Digest)
File "/usr/lib/python2.7/site-packages/pyasn1/type/namedtype.py",
line 115, in __init__
self.__ambiguousTypes = 'terminal' not in kwargs and
self.__computeAmbiguousTypes() or {}
File "/usr/lib/python2.7/site-packages/pyasn1/type/namedtype.py",
line 232, in __computeAmbiguousTypes
ambigiousTypes[idx] = NamedTypes(*partialAmbigiousTypes,
**dict(terminal=True))
File "/usr/lib/python2.7/site-packages/pyasn1/type/namedtype.py",
line 114, in __init__
self.__tagToPosMap = self.__computeTagToPosMap()
File "/usr/lib/python2.7/site-packages/pyasn1/type/namedtype.py",
line 205, in __computeTagToPosMap
for _tagSet in tagMap.presentTypes:
AttributeError: 'property' object has no attribute 'presentTypes'
Any help would be greatly appreciated.
Thanks,
Bill G.
5 months, 1 week
FreeIPA, OSX, DockerDesktop
by james liu
PREP
====
git clone https://github.com/freeipa/freeipa-container.git
cd freeipa-container
mkdir /tmp/ipa-data
docker run --name freeipa-server-container -ti -h ipa.example.test --read-only -v /tmp/ip-data :/data:Z freeipa-server --sysctl net.ipv6.conf.all.disable_ipv6=1
RESULT
======
tar: etc/sysconfig/selinux: Cannot utime: No such file or directory
tar: Exiting with failure status due to previous errors
QUESTION
=========
I'm running DockerDesktop 2.0.4, OSX 10.13.6.
Is there a set of commands that will work?
Thanks
9 months, 1 week
yum update problem
by Natxo Asenjo
hi,
after patching our centos 7 hosts to the latest version today, one of the
two replicas is having trouble.
[root@kdc2 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: STOPPED
kadmin Service: STOPPED
named Service: STOPPED
httpd Service: RUNNING
ipa-custodia Service: STOPPED
ntpd Service: STOPPED
pki-tomcatd Service: RUNNING
smb Service: STOPPED
winbind Service: STOPPED
ipa-otpd Service: STOPPED
ipa-dnskeysyncd Service: STOPPED
ipa: INFO: The ipactl command was successful
and after digging in the logs I come across this in /var/log/ipaupgrade.log:
2019-11-20T18:18:29Z DEBUG stderr=
2019-11-20T18:18:31Z INFO Certmonger certificate renewal configuration
already up-to-date
2019-11-20T18:18:31Z INFO [Enable PKIX certificate path discovery and
validation]
2019-11-20T18:18:31Z DEBUG Loading StateFile from
'/var/lib/ipa/sysupgrade/sysupgrade.state'
2019-11-20T18:18:31Z INFO PKIX already enabled
2019-11-20T18:18:31Z INFO [Authorizing RA Agent to modify profiles]
2019-11-20T18:18:31Z INFO [Authorizing RA Agent to manage lightweight CAs]
2019-11-20T18:18:31Z INFO [Ensuring Lightweight CAs container exists in
Dogtag database]
2019-11-20T18:18:31Z DEBUG Created connection context.ldap2_139740162547472
2019-11-20T18:18:31Z DEBUG flushing
ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket from SchemaCache
2019-11-20T18:18:31Z DEBUG retrieving schema for SchemaCache
url=ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket
conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7f17cc24b638>
2019-11-20T18:18:31Z DEBUG Destroyed connection
context.ldap2_139740162547472
2019-11-20T18:18:31Z INFO [Adding default OCSP URI configuration]
2019-11-20T18:18:31Z INFO [Ensuring CA is using LDAPProfileSubsystem]
2019-11-20T18:18:31Z INFO [Migrating certificate profiles to LDAP]
2019-11-20T18:18:31Z DEBUG Created connection context.ldap2_139740160021648
2019-11-20T18:18:31Z DEBUG flushing
ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket from SchemaCache
2019-11-20T18:18:31Z DEBUG retrieving schema for SchemaCache
url=ldapi://%2fvar%2frun%2fslapd-L-DOMAIN-IT.socket
conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7f17cc289b00>
2019-11-20T18:18:31Z DEBUG Destroyed connection
context.ldap2_139740160021648
2019-11-20T18:18:31Z DEBUG request GET
https://kdc2.l.domain.it:8443/ca/rest/account/login
2019-11-20T18:18:31Z DEBUG request body ''
2019-11-20T18:18:31Z DEBUG response status 401
2019-11-20T18:18:31Z DEBUG response headers Server: Apache-Coyote/1.1
Cache-Control: private
Expires: Thu, 01 Jan 1970 01:00:00 CET
WWW-Authenticate: Basic realm="Certificate Authority"
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 951
Date: Wed, 20 Nov 2019 18:18:31 GMT
2019-11-20T18:18:31Z DEBUG response body '<html><head><title>Apache
Tomcat/7.0.76 - Error report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
{color : black;}A.name {color : black;}HR {color : #525D76;}--></style>
</head><body><h1>HTTP Status 401 - </h1><HR size="1"
noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b>
<u></u></p><p><b>description</b> <u>This request requires HTTP
authentication.</u></p><HR size="1" noshade="noshade"><h3>Apache
Tomcat/7.0.76</h3></body></html>'
2019-11-20T18:18:31Z ERROR IPA server upgrade failed: Inspect
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2019-11-20T18:18:31Z DEBUG File
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in
execute
return_value = self.run()
File
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
line 54, in run
server.upgrade()
File
"/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 2146, in upgrade
upgrade_configuration()
File
"/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 2018, in upgrade_configuration
ca_enable_ldap_profile_subsystem(ca)
File
"/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 406, in ca_enable_ldap_profile_subsystem
cainstance.migrate_profiles_to_ldap()
File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
line 2027, in migrate_profiles_to_ldap
_create_dogtag_profile(profile_id, profile_data, overwrite=False)
File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
line 2033, in _create_dogtag_profile
with api.Backend.ra_certprofile as profile_api:
File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line
1315, in __enter__
raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA
REST API'))
2019-11-20T18:18:31Z DEBUG The ipa-server-upgrade command failed,
exception: RemoteRetrieveError: Failed to authenticate to CA REST API
2019-11-20T18:18:31Z ERROR Unexpected error - see /var/log/ipaupgrade.log
for details:
RemoteRetrieveError: Failed to authenticate to CA REST API
In this kdc I see these errors in getcert list:
Request ID '20190220182014':
status: MONITORING
ca-error: Invalid cookie: u''
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=L.DOMAIN.IT
subject: CN=CA Audit,O=L.DOMAIN.IT
expires: 2019-12-05 13:58:24 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190220182015':
status: MONITORING
ca-error: Invalid cookie: u''
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=L.DOMAIN.IT
subject: CN=OCSP Subsystem,O=L.DOMAIN.IT
expires: 2019-12-05 13:58:24 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190220182016':
status: MONITORING
ca-error: Invalid cookie: u''
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=L.DOMAIN.IT
subject: CN=CA Subsystem,O=L.DOMAIN.IT
expires: 2019-12-05 13:58:24 UTC
key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190220182018':
status: MONITORING
ca-error: Invalid cookie: u''
stuck: no
key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=L.DOMAIN.IT
subject: CN=IPA RA,O=L.DOMAIN.IT
expires: 2019-12-05 13:58:44 UTC
key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
Request ID '20190220182019':
status: MONITORING
ca-error: Server at "
https://kdc2.l.domain.it:8443/ca/agent/ca/profileProcess" replied: 1:
Invalid Credential.
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=L.DOMAIN.IT
subject: CN=kdc2.l.domain.it,O=L.DOMAIN.IT
expires: 2019-12-10 10:57:52 UTC
key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
I still have a working replica, so I could just reinstall and have a
working set in a couple of minutes, but I would like to find out what has
gone wrong.
The systems are running ipa-server-4.6.5-11.el7.centos.3.x86_64
Any help welcome ;-)
Thanks,
--
Groeten,
natxo
9 months, 3 weeks
Replica not renewing IPA certificates
by Roderick Johnstone
Hi
This is freeipa (ipa-server-4.6.5-11.el7_7.3.x86_64) on RHEL7 with
freeipa's own internal CA.
One of my ipa server replicas (host3) has not renewed its IPA system
certificates and is now showing
ca-error: Invalid cookie: u''
in the 'getcert list' output for certificates:
"auditSigningCert cert-pki-ca", "ocspSigningCert cert-pki-ca",
"subsystemCert cert-pki-ca", and the
certificate in the file /var/lib/ipa/ra-agent.pem
As far as I can see, the sequence of events has been as follows:
host3 noticed the certificates needed renewing at 30 Jan 2020 05:37 and
certmonger initiated a renewal.
The state of those certificates went from MONITORING to CA_WORKING but
the certificates were not renewed.
The CA renewal master (host1) noticed its same set of certificates (plus
"Server-Cert cert-pki-ca") needed renewing at 30 Jan 2020 07:28 and
renewed them successfully.
Another replica (host2) noticed that its certificates needed renewing at
30 Jan 2020 07:32 and renewed them successfully.
At 30 Jan 13:37 on host3 the certificates needing to be renewed went
from CA_WORKING back to MONITORING, but 'getcert list' now shows them with:
ca-error: Invalid cookie: u''
and they still haven't renewed.
I haven't seen certmonger attempt to try the renewal again on host3
(nothing from certmonger in /var/log/messages since 30 Jan 13:37).
While I could try a getcert resubmit on host3 to force it to try again,
I'd like to know if what I am seeing is the expected behaviour when a
replica tried to renew certificates before the renewal master.
How long should I have to wait till certmonger on host3 tries again? - I
couldn't find any reference to how often certmonger tries the renewal.
Rob Crittenden's freeipa-healthcheck script is now showing the following
for host3:
ERROR: ipahealthcheck.ipa.certs.IPARAAgent: RA agent description does
not match 2;16;CN=Certificate Authority,O=EXAMPLE.COM;CN=IPA
RA,O=EXAMPLE.COM in LDAP and 2;7;CN=Certificate
Authority,O=EXAMPLE.COM;CN=IPA RA,O=EXAMPLE.COM expected
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040924:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040920:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040921:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040922:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040923:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040925:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040927:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180926040926:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.ipa.certs.IPACertRevocation.20180831064406:
Request for certificate failed, Certificate operation cannot be
completed: EXCEPTION (Invalid Credential.)
ERROR: ipahealthcheck.dogtag.ca.DogtagCertsConnectivityCheck: Request
for certificate failed, Certificate operation cannot be completed:
EXCEPTION (Invalid Credential.)
Each of host1, host2 and host3 are showing serial number 16 in ldap using:
ldapsearch -D "cn=directory manager" -W -b uid=ipara,ou=people,o=ipaca
description
At this stage I'm not sure whether this will resolve itself when
certmonger tries to renew certificates again or whether I need to be
more proactive.
I'm happy to supply more logs as necessary.
Thanks
Roderick
9 months, 3 weeks
ipa-replica-install failing
by Mitchell Smith
Hi list,
I wanted to repost this issue with a more appropriate subject line, in
case anyone has come across this issue before and has a work around.
To provide some context, I have two FreeIPA instances running FreeIPA
4.3.1 on Ubuntu 16.04 LTS.
I want to migrate to FreeIPA 4.5.4 running on CentOS 7.
I have a way to migrate by dumping all the users out with ldapsearch
and adding them to the new instance with ldapadd but it is a bit messy
and will result in all users having to reset their password, as it
won't let me add in already encrypted passwords.
My initial thought was to add the new instance as a replica and then
eventually retire the old one.
I ran in to some problems with the ‘ipa-replica-install’ command though.
I was able to join as a client no problem, but when I went to run
‘ipa-replica-install’ it failed while configuring the directory server
component.
[25/42]: restarting directory server
[26/42]: creating DS keytab
[27/42]: ignore time skew for initial replication
[28/42]: setting up initial replication
[error] DatabaseError: Server is unwilling to perform: modification
of attribute nsds5replicareleasetimeout is not allowed in replica
entry
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
I thought this might have something to do with differences between
4.3.1 and 4.5.4 but I wasn’t entirely sure.
If there is a work around for this issue, it would be a significantly
easier transition to the new FreeIPA instance.
Cheers,
Mitch
11 months
IPA healthcheck for older versions
by Rob Crittenden
Over the summer we announced the freeipa-healthcheck project which is
designed to look at an IdM cluster and look for common problems so you
can have some level of assurance that the system is running as it should.
It was built against the IPA 4.8.x branch and originally released only
for Fedora 29+. It is also included in the newly released RHEL 8.1.0.
My curious nature led me to see if it would also work in in the IPA
4.6.x branch. It was a bit of a challenge backing down to Python 2 but I
was able to get something working. I tested primarily on Fedora 27 but
it should also work in RHEL/CentOS 7 (I smoke tested 7.8).
I made an EPEL 7 build in COPR,
https://copr.fedorainfracloud.org/coprs/rcritten/ipa-healthcheck/
Enable the repo and do: yum install freeipa-healthcheck
Then run: ipa-healthcheck --failures-only
Ideally there will be no output but an empty list []. Otherwise the
output is JSON and hopefully has enough information to point you in the
right direction. Feel free to ask if need help.
False positives are always a possibility and many of the checks run
independently so it's possible to get multiple issues from a single root
problem. It's hard to predict all possible installations so some
fine-tuning may be required.
I'd recommend running it every now and then at least, like prior to
updating IPA packages, creating a new master, etc, if not daily. It
will, for example, warn of impending cert expiration.
The more feedback I get on it the better and more useful I can make it.
This is my own personal backport and is not officially supported by
anyone but me. It's preferred to report issues on this mailing list.
I'll see them and others may be able to chime in as well.
rob
11 months
Replication issue with CSN generator
by Morgan Marodin
Hi.
Into my environment I have two IPA server, replicating each other.
They are both 7.6 OS systems, ipa-server RPM version is
4.6.4-10.0.1.el7_6.2.x86_64.
The first server installed was srv01 (many years ago), then I installed the
replica into srv02 (like a year later the 1st node).
When I had a single server I did also a trust with my corporate Active
Directory.
VMs are running in 2 different hypervisor clusters.
Now the replication doesn't works. Into log files I have this error:
*[16/Apr/2020:12:25:36.856632697 +0200] - ERR - csngen_adjust_time -
Adjustment limit exceeded; value - 23221226, limit -
86400[16/Apr/2020:12:25:36.857909222 +0200] - ERR - NSMMReplicationPlugin -
repl5_inc_run - agmt="cn=meTosrv01.ipa.mydomain.com
<http://meTosrv01.ipa.mydomain.com>" (srv01:389): Fatal error - too much
time skew between replicas![16/Apr/2020:12:25:36.862233147 +0200] - ERR -
NSMMReplicationPlugin - repl5_inc_run - agmt="cn=meTosrv01.ipa.mydomain.com
<http://meTosrv01.ipa.mydomain.com>" (srv01:389): Incremental update failed
and requires administrator action*
I tried to force the replica, but the limit exceeded problem doesn't allow
the sync.
I know that the problem is that CSN generator has become grossly skewed.
Using the external script readNsState.py I found that there was as offset
time for about a month, so ... I waited for a month and then the issue
disappeared.
But now the offset is about 9 months ... I can't wait so much time :)
*[root@srv01 scripts]# ./readNsState.py
/etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldifnsState is
BAAAAAAAAACCN/xfAAAAAHbiBAAAAAAABCgAAAAAAAANdQAAAAAAAA==Little EndianFor
replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping
tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN
generator state: Replica ID : 4 Sampled Time : 1610364802 Gen
as csn : 5ffc37822996500040000 Time as str : Mon Jan 11 12:33:22
2021 Local Offset : 320118 Remote Offset : 10244 Seq. num :
29965 System time : Tue Apr 21 15:03:45 2020 Diff in sec. :
-22890577 Day:sec diff : -265:5423nsState is
YAAAAAAAAAADLZheAAAAAAAAAAAAAAAAXSgAAAAAAAATAAAAAAAAAA==Little EndianFor
replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state:
Replica ID : 96 Sampled Time : 1587031299 Gen as csn :
5e982d03001900960000 Time as str : Thu Apr 16 12:01:39 2020 Local
Offset : 0 Remote Offset : 10333 Seq. num : 19 System time
: Tue Apr 21 15:03:45 2020 Diff in sec. : 442926 Day:sec diff :
5:10926[root@srv02 scripts]# ./readNsState.py
/etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldifnsState is
AwAAAAAAAABU7p5eAAAAAAAAAAAAAAAAsVNiAQAAAAAAAAAAAAAAAA==Little EndianFor
replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping
tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN
generator state: Replica ID : 3 Sampled Time : 1587474004 Gen
as csn : 5e9eee54000000030000 Time as str : Tue Apr 21 15:00:04
2020 Local Offset : 0 Remote Offset : 23221169 Seq. num : 0
System time : Tue Apr 21 15:02:38 2020 Diff in sec. : 154
Day:sec diff : 0:154nsState is
YQAAAAAAAAAuLZheAAAAAEUBAAAAAAAA7SYAAAAAAAASAAAAAAAAAA==Little EndianFor
replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state:
Replica ID : 97 Sampled Time : 1587031342 Gen as csn :
5e982d2e001800970000 Time as str : Thu Apr 16 12:02:22 2020 Local
Offset : 325 Remote Offset : 9965 Seq. num : 18 System time
: Tue Apr 21 15:02:38 2020 Diff in sec. : 442816 Day:sec diff :
5:10816*
As you can see in the 1st node the Time as str is Jan 11 of 2021.
With timedatectl command I see that both VMs use the same Time zone and the
clock is correct.
I found this old article to fix my issue:
*https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html
<https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html>*
But ... I had the same issue in the past, always in the 1st server. So, in
my mind I don't want to try to use that fix.
I have a new hypervisor cluster, so I would prefer to reinstall the 1st
server, using these steps:
1) check if all roles (also the CA) is installed in srv02
You can find here some data about the VMs:
*[root@srv01 ~]# ipa server-show srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com> Server name: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com> Managed suffixes: domain, ca Min domain
level: 0 Max domain level: 1 Enabled server roles: CA server, IPA master,
DNS server, NTP server, AD trust controller[root@srv02 ~]# ipa server-show
srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> Server name:
srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> Managed suffixes:
domain, ca Min domain level: 0 Max domain level: 1 Enabled server roles:
CA server, IPA master, DNS server, NTP server[root@srv01 ~]# ipa
config-show Maximum username length: 32 Home directory base: /home
Default shell: /bin/bash Default users group: ipausers Default e-mail
domain: ipa.mydomain.com <http://ipa.mydomain.com> Search time limit: 2
Search size limit: 100 User search fields:
uid,givenname,sn,telephonenumber,ou,title Group search fields:
cn,description Enable migration mode: FALSE Certificate Subject base:
O=IPA.MYDOMAIN.COM <http://IPA.MYDOMAIN.COM> Password Expiration
Notification (days): 4 Password plugin features: AllowNThash SELinux user
map order:
guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023
Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types:
MS-PAC, nfs:NONE IPA masters: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com
<http://srv02.ipa.mydomain.com> IPA CA servers: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com
<http://srv02.ipa.mydomain.com> IPA NTP servers: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com
<http://srv02.ipa.mydomain.com> IPA CA renewal master:
srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>[root@srv02 ~]# ipa
config-show Maximum username length: 32 Home directory base: /home
Default shell: /bin/bash Default users group: ipausers Default e-mail
domain: ipa.mydomain.com <http://ipa.mydomain.com> Search time limit: 2
Search size limit: 100 User search fields:
uid,givenname,sn,telephonenumber,ou,title Group search fields:
cn,description Enable migration mode: FALSE Certificate Subject base:
O=IPA.MYDOMAIN.COM <http://IPA.MYDOMAIN.COM> Password Expiration
Notification (days): 4 Password plugin features: AllowNThash SELinux user
map order:
guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023
Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types:
MS-PAC, nfs:NONE IPA masters: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com
<http://srv02.ipa.mydomain.com> IPA CA servers: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com
<http://srv02.ipa.mydomain.com> IPA NTP servers: srv01.ipa.mydomain.com
<http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com
<http://srv02.ipa.mydomain.com> IPA CA renewal master:
srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>[root@srv01 ~]#
ipactl statusDirectory Service: RUNNINGkrb5kdc Service: RUNNINGkadmin
Service: RUNNINGnamed Service: RUNNINGhttpd Service: RUNNINGipa-custodia
Service: RUNNINGntpd Service: RUNNINGpki-tomcatd Service: STOPPEDsmb
Service: RUNNINGwinbind Service: RUNNINGipa-otpd Service:
RUNNINGipa-dnskeysyncd Service: RUNNINGipa: INFO: The ipactl command was
successful[root@srv02 ~]# ipactl statusDirectory Service: RUNNINGkrb5kdc
Service: RUNNINGkadmin Service: RUNNINGnamed Service: RUNNINGhttpd Service:
RUNNINGipa-custodia Service: RUNNINGntpd Service: RUNNINGpki-tomcatd
Service: STOPPEDipa-otpd Service: RUNNINGipa-dnskeysyncd Service:
RUNNINGipa: INFO: The ipactl command was successful[root@srv01 ~]# certutil
-L -d /etc/pki/pki-tomcat/aliasCertificate Nickname
Trust Attributes
SSL,S/MIME,JAR/XPIServer-Cert cert-pki-ca
u,u,usubsystemCert cert-pki-ca
u,u,ucaSigningCert cert-pki-ca
CTu,Cu,CuocspSigningCert cert-pki-ca
u,u,uauditSigningCert cert-pki-ca
u,u,Pu[root@srv02 ~]# certutil -L -d /etc/pki/pki-tomcat/aliasCertificate
Nickname Trust Attributes
SSL,S/MIME,JAR/XPIServer-Cert cert-pki-ca
u,u,usubsystemCert cert-pki-ca
u,u,ucaSigningCert cert-pki-ca
CTu,u,uocspSigningCert cert-pki-ca
u,u,uauditSigningCert cert-pki-ca u,u,Pu*
It seems that AD trust controller role, IPA CA renewal master, smb and
windbind are only in the 1st server.
And also caSigningCert cert-pki-ca entry is different (CTu,Cu,Cu vs
CTu,u,u).
I can see only in the 1st server these DNS records:
*_kerberos._tcp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88
srv01_kerberos._tcp.dc._msdcs SRV 0 100 88
srv01_kerberos._udp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88
srv01_kerberos._udp.dc._msdcs SRV 0 100 88
srv01_ldap._tcp.Default-First-Site-Name._sites.dc._msdcs 0 100 389
srv01_ldap._tcp.dc._msdcs 0 100 389 srv01*
Srv01 is the first master, I know, but is the server VM that has clock
problems, in both situations.
So I want to keep srv02 and install a new one.
What do I have to do to let the 2nd VM be a single server?
Could I use these URLs?
*https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_Master
<https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_M...>https://www.freeipa.org/page/V4/Server_Roles#Upgrade
<https://www.freeipa.org/page/V4/Server_Roles#Upgrade>*
2) uninstall ipa-server from the 1st server (srv01) and then powering off
it, assuming that all data into the 2nd one are ok (srv02)
3) update freeipa and all other RPM packages into the VM srv02
4) install a new fresh VM, always with 7 release, and create a new replica
Could I use the same old hostname (srv01) and IP address for this new VM?
Or is better to use the same IP but a new name, like srv03?
Do you think this is the right way to solve my issue?
Or do you have any better idea?
Please let me know, thanks.
Bye, Morgan
11 months, 2 weeks
kinit -n asking for password on clients
by John Ratliff
When trying to do pkinit, if I do kinit -n on one of the IdM servers, it
works fine. If I try on a client machine, it asks me for the password
for WELLKNOWN/ANONYMOUS@REALM.
I have the pkinit_anchors setup for the realm. As I'm trying to do
anonymous pkinit, I think I don't need a client certificate.
On the server, I get this:
$ KRB5_TRACE="/dev/stderr" kinit -n
[13061] 1518402857.924212: Getting initial credentials for
WELLKNOWN/ANONYMOUS(a)IDM.EXAMPLE.COM
[13061] 1518402857.929673: Sending request (200 bytes) to IDM.EXAMPLE.COM
[13061] 1518402857.931830: Initiating TCP connection to stream
10.77.9.101:88
[13061] 1518402857.932241: Sending TCP request to stream 10.77.9.101:88
[13061] 1518402857.939162: Received answer (359 bytes) from stream
10.77.9.101:88
[13061] 1518402857.939180: Terminating TCP connection to stream
10.77.9.101:88
[13061] 1518402857.939284: Response was from master KDC
[13061] 1518402857.939380: Received error from KDC:
-1765328359/Additional pre-authentication required
[13061] 1518402857.939474: Processing preauth types: 16, 15, 14, 136,
19, 147, 2, 133
[13061] 1518402857.939499: Selected etype info: etype aes256-cts, salt
"IDM.EXAMPLE.COMWELLKNOWNANONYMOUS", params ""
[13061] 1518402857.939509: Received cookie: MIT
[13061] 1518402857.939563: Preauth module pkinit (147) (info) returned:
0/Success
[13061] 1518402857.940352: PKINIT client computed kdc-req-body checksum
9/D98A0144E7E4ACC66B63EBCA98379AB9F055D143
[13061] 1518402857.940369: PKINIT client making DH request
[13061] 1518402858.935: Preauth module pkinit (16) (real) returned:
0/Success
[13061] 1518402858.956: Produced preauth for next request: 133, 16
[13061] 1518402858.994: Sending request (1408 bytes) to IDM.EXAMPLE.COM
[13061] 1518402858.1091: Initiating TCP connection to stream 10.77.9.101:88
[13061] 1518402858.1187: Sending TCP request to stream 10.77.9.101:88
[13061] 1518402858.43063: Received answer (2880 bytes) from stream
10.77.9.101:88
[13061] 1518402858.43088: Terminating TCP connection to stream
10.77.9.101:88
[13061] 1518402858.43198: Response was from master KDC
[13061] 1518402858.43258: Processing preauth types: 17, 19, 147
[13061] 1518402858.43273: Selected etype info: etype aes256-cts, salt
"IDM.EXAMPLE.COMWELLKNOWNANONYMOUS", params ""
[13061] 1518402858.43300: Preauth module pkinit (147) (info) returned:
0/Success
[13061] 1518402858.44150: PKINIT client verified DH reply
[13061] 1518402858.44189: PKINIT client found id-pkinit-san in KDC cert:
krbtgt/IDM.EXAMPLE.COM(a)IDM.EXAMPLE.COM
[13061] 1518402858.44199: PKINIT client matched KDC principal
krbtgt/IDM.EXAMPLE.COM(a)IDM.EXAMPLE.COM against id-pkinit-san; no EKU
check required
[13061] 1518402858.62345: PKINIT client used KDF 2B06010502030602 to
compute reply key aes256-cts/00E0
[13061] 1518402858.62395: Preauth module pkinit (17) (real) returned:
0/Success
[13061] 1518402858.62402: Produced preauth for next request: (empty)
[13061] 1518402858.62414: AS key determined by preauth: aes256-cts/00E0
[13061] 1518402858.62547: Decrypted AS reply; session key is:
aes256-cts/96F0
[13061] 1518402858.62589: FAST negotiation: available
[13061] 1518402858.62692: Initializing
KEYRING:persistent:760400007:krb_ccache_f3PFEy1 with default princ
WELLKNOWN/ANONYMOUS@WELLKNOWN:ANONYMOUS
[13061] 1518402858.62770: Storing
WELLKNOWN/ANONYMOUS@WELLKNOWN:ANONYMOUS ->
krbtgt/IDM.EXAMPLE.COM(a)IDM.EXAMPLE.COM in
KEYRING:persistent:760400007:krb_ccache_f3PFEy1
[13061] 1518402858.62846: Storing config in
KEYRING:persistent:760400007:krb_ccache_f3PFEy1 for
krbtgt/IDM.EXAMPLE.COM(a)IDM.EXAMPLE.COM: fast_avail: yes
[13061] 1518402858.62878: Storing
WELLKNOWN/ANONYMOUS@WELLKNOWN:ANONYMOUS ->
krb5_ccache_conf_data/fast_avail/krbtgt\/IDM.EXAMPLE.COM\@IDM.EXAMPLE.COM(a)X-CACHECONF:
in KEYRING:persistent:760400007:krb_ccache_f3PFEy1
[13061] 1518402858.62933: Storing config in
KEYRING:persistent:760400007:krb_ccache_f3PFEy1 for
krbtgt/IDM.EXAMPLE.COM(a)IDM.EXAMPLE.COM: pa_type: 16
[13061] 1518402858.62954: Storing
WELLKNOWN/ANONYMOUS@WELLKNOWN:ANONYMOUS ->
krb5_ccache_conf_data/pa_type/krbtgt\/IDM.EXAMPLE.COM\@IDM.EXAMPLE.COM(a)X-CACHECONF:
in KEYRING:persistent:760400007:krb_ccache_f3PFEy1
But on the client, I get this:
$ KRB5_TRACE="/dev/stderr" kinit -n
[2941] 1518402820.155827: Getting initial credentials for
WELLKNOWN/ANONYMOUS(a)IDM.EXAMPLE.COM
[2941] 1518402820.156298: Sending request (200 bytes) to IDM.EXAMPLE.COM
[2941] 1518402820.158723: Resolving hostname paine.example.com.
[2941] 1518402820.159975: Resolving hostname phantom.example.com.
[2941] 1518402820.160757: Resolving hostname paine.example.com.
[2941] 1518402820.161411: Initiating TCP connection to stream
204.89.253.101:88
[2941] 1518402820.162065: Sending TCP request to stream 204.89.253.101:88
[2941] 1518402820.168495: Received answer (359 bytes) from stream
204.89.253.101:88
[2941] 1518402820.168532: Terminating TCP connection to stream
204.89.253.101:88
[2941] 1518402820.169917: Response was from master KDC
[2941] 1518402820.169974: Received error from KDC:
-1765328359/Additional pre-authentication required
[2941] 1518402820.170029: Processing preauth types: 16, 15, 14, 136, 19,
147, 2, 133
[2941] 1518402820.170051: Selected etype info: etype aes256-cts, salt
"IDM.EXAMPLE.COMWELLKNOWNANONYMOUS", params ""
[2941] 1518402820.170062: Received cookie: MIT
Password for WELLKNOWN/ANONYMOUS(a)IDM.EXAMPLE.COM:
[2941] 1518402833.34612: Preauth module encrypted_timestamp (2) (real)
returned: -1765328252/Password read interrupted
kinit: Pre-authentication failed: Password read interrupted while
getting initial credentials
Suggestions on what I'm missing?
Thanks.
1 year
