noon
Hallo,
Unfortunately I don't know when this problem occurred first, but it may
have occurred after an update.
The httpd does not start and aborts with the error
[:info] [pid 15383] Using nickname Server-Cert.
[...] [:error] [pid 15383] Certificate not found: 'Server-Cert'
when I want to start FreeIPA via "systemctl start ipa" or "ipactl
start"
or "systemctl start httpd"
If I turn the NSSEngine off it starts of cause.
In contrast to this message "ipa-getcert list -d /etc/httpd/alias/ -n
Server-Cert" does find a certificate, if I get the output [1] right.
ipa-server-upgrade also complained about the HTTPD not starting, so I
tried to run it with "NSSEnigne off" which made the upgrade run through,
but did not fix the problem with the HTTPd
My System:
(After running "ipa-server-upgrade" with out any failures, but with
"NSSEngine off")
# ipa --version
VERSION: 4.4.4, API_VERSION: 2.215
on Fedora Server 26
CA-Server at main IPA-Server (which is failing now)
/etc/hosts has got the fqdn in the first line
and DNS is not installed.
[1] # ipa-getcert list -d /etc/httpd/alias/ -n Server-Cert
Number of certificates and requests being tracked: 8.
Request ID '20160718102648':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=EXAMPLE.COM
subject: CN=ipa_server.example.com,O=EXAMPLE.COM
expires: 2018-03-24 14:33:00 CET
key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
Many thanks in advance,
Julian
9:21 a.m.
Julian Gethmann via FreeIPA-users wrote:
Hallo,
Unfortunately I don't know when this problem occurred first, but it may
have occurred after an update.
The httpd does not start and aborts with the error
[:info] [pid 15383] Using nickname Server-Cert.
[...] [:error] [pid 15383] Certificate not found: 'Server-Cert'
when I want to start FreeIPA via "systemctl start ipa" or "ipactl
start"
or "systemctl start httpd"
If I turn the NSSEngine off it starts of cause.
In contrast to this message "ipa-getcert list -d /etc/httpd/alias/ -n
Server-Cert" does find a certificate, if I get the output [1] right.
ipa-getcert shows certs that are tracked by certmonger but doesn't
guarantee that those certificates actually exist in the filesystem (they
did at the time tracking was started).
You need to look at the Apache NSS database:
# certutil -L -d /etc/httpd/alias
rob
9:47 a.m.
Hallo,
On 08/14/2017 04:21 PM, Rob Crittenden wrote:
Julian Gethmann via FreeIPA-users wrote:
> Hallo,
>
> Unfortunately I don't know when this problem occurred first, but it may
> have occurred after an update.
> The httpd does not start and aborts with the error
>
> [:info] [pid 15383] Using nickname Server-Cert.
> [...] [:error] [pid 15383] Certificate not found: 'Server-Cert'
>
> when I want to start FreeIPA via "systemctl start ipa" or "ipactl
start"
> or "systemctl start httpd"
> If I turn the NSSEngine off it starts of cause.
>
> In contrast to this message "ipa-getcert list -d /etc/httpd/alias/ -n
> Server-Cert" does find a certificate, if I get the output [1] right.
ipa-getcert shows certs that are tracked by certmonger but doesn't
guarantee that those certificates actually exist in the filesystem (they
did at the time tracking was started).
You need to look at the Apache NSS database:
# certutil -L -d /etc/httpd/alias
Ok, I also did this, but it seems to be there
# certutil -L -d /etc/httpd/alias
Certificate Nickname Trust
Attributes
SSL,S/MIME,JAR/XPI
Signing-Cert u,u,u
ipaCert u,u,u
Server-Cert Pu,u,u
EXAMPLE.COM IPA CA CT,C,C
Thanks,
Julian
rob
10:46 a.m.
Julian Gethmann wrote:
Hallo,
On 08/14/2017 04:21 PM, Rob Crittenden wrote:
> Julian Gethmann via FreeIPA-users wrote:
>> Hallo,
>>
>> Unfortunately I don't know when this problem occurred first, but it may
>> have occurred after an update.
>> The httpd does not start and aborts with the error
>>
>> [:info] [pid 15383] Using nickname Server-Cert.
>> [...] [:error] [pid 15383] Certificate not found: 'Server-Cert'
>>
>> when I want to start FreeIPA via "systemctl start ipa" or "ipactl
start"
>> or "systemctl start httpd"
>> If I turn the NSSEngine off it starts of cause.
>>
>> In contrast to this message "ipa-getcert list -d /etc/httpd/alias/ -n
>> Server-Cert" does find a certificate, if I get the output [1] right.
>
> ipa-getcert shows certs that are tracked by certmonger but doesn't
> guarantee that those certificates actually exist in the filesystem (they
> did at the time tracking was started).
>
> You need to look at the Apache NSS database:
>
> # certutil -L -d /etc/httpd/alias
Ok, I also did this, but it seems to be there
# certutil -L -d /etc/httpd/alias
Certificate Nickname Trust
Attributes
SSL,S/MIME,JAR/XPI
Signing-Cert u,u,u
ipaCert u,u,u
Server-Cert Pu,u,u
EXAMPLE.COM IPA CA CT,C,C
I'd check FS permissions. /etc/httpd/alias/*.db should be root:apache 0640
If that checks out, look for SELinux issues by starting httpd then
running: ausearch -m AVC -ts recent
As a last resort perhaps the NSS database is corrupted. You can exercise
it with:
# certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
/etc/httpd/alias/pwdfile.txt
You should get: certutil: certificate is valid
rob
2:06 p.m.
On 08/14/2017 05:46 PM, Rob Crittenden wrote:
Julian Gethmann wrote:
> Hallo,
>
> On 08/14/2017 04:21 PM, Rob Crittenden wrote:
>> Julian Gethmann via FreeIPA-users wrote:
>>> Hallo,
>>>
>>> Unfortunately I don't know when this problem occurred first, but it may
>>> have occurred after an update.
>>> The httpd does not start and aborts with the error
>>>
>>> [:info] [pid 15383] Using nickname Server-Cert.
>>> [...] [:error] [pid 15383] Certificate not found: 'Server-Cert'
>>>
>>> when I want to start FreeIPA via "systemctl start ipa" or
"ipactl start"
>>> or "systemctl start httpd"
>>> If I turn the NSSEngine off it starts of cause.
>>>
>>> In contrast to this message "ipa-getcert list -d /etc/httpd/alias/ -n
>>> Server-Cert" does find a certificate, if I get the output [1] right.
>>
>> ipa-getcert shows certs that are tracked by certmonger but doesn't
>> guarantee that those certificates actually exist in the filesystem (they
>> did at the time tracking was started).
>>
>> You need to look at the Apache NSS database:
>>
>> # certutil -L -d /etc/httpd/alias
> Ok, I also did this, but it seems to be there
> # certutil -L -d /etc/httpd/alias
>
> Certificate Nickname Trust
> Attributes
>
> SSL,S/MIME,JAR/XPI
>
> Signing-Cert u,u,u
> ipaCert u,u,u
> Server-Cert Pu,u,u
> EXAMPLE.COM IPA CA CT,C,C
I'd check FS permissions. /etc/httpd/alias/*.db should be root:apache 0640
ok,
the db were "root:apache 0660", but they were readable at least and
making them 0640 did not help either.
If that checks out, look for SELinux issues by starting httpd then
running: ausearch -m AVC -ts recent
I disabled SELinux for testing it, but that did
not work. Now I also tested:
# ausearch -m AVC -ts recent
<no matches>
As a last resort perhaps the NSS database is corrupted. You can exercise
it with:
# certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
/etc/httpd/alias/pwdfile.txt
You should get: certutil: certificate is valid
I do get it:
# certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
/etc/httpd/alias/pwdfile.txt
certutil: certificate is valid
rob
If I just want to start httpd and not via IPA or with --force I get a
different error, which I think might be because the services started
before httpd in the IPA start-up-phase aren't running since the start of
IPA aborted:
-- Unit httpd.service has begun starting up.
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
: ERROR Unknown error while retrieving setting from ldap
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
Traceback (most recent call last):
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/libexec/ipa/ipa-httpd-kdcproxy", line 84, in _ldap_con
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.con.do_bind(timeout=self.time_limit)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.do_external_bind(pw_name, timeout=timeout)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.__bind_with_wait(self.external_bind, timeout, user_name)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.__wait_for_connection(timeout)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
wait_for_open_socket(lurl.hostport, timeout)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 13
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
raise e
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: error:
[Errno 111] Connection refused
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
: ERROR Unknown error while retrieving setting from ldap
Aug 14 19:05:14 ipa_server.example.com systemd[1]: httpd.service:
Control process exited, code=exited status=1
Aug 14 19:05:14 ipa_server.example.com audit[1]: SERVICE_START pid=1
uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s
Aug 14 19:05:14 ipa_server.example.com systemd[1]: Failed to start The
Apache HTTP Server.
2:51 p.m.
Julian Gethmann wrote:
On 08/14/2017 05:46 PM, Rob Crittenden wrote:
> Julian Gethmann wrote:
>> Hallo,
>>
>> On 08/14/2017 04:21 PM, Rob Crittenden wrote:
>>> Julian Gethmann via FreeIPA-users wrote:
>>>> Hallo,
>>>>
>>>> Unfortunately I don't know when this problem occurred first, but it
>>>> may
>>>> have occurred after an update.
>>>> The httpd does not start and aborts with the error
>>>>
>>>> [:info] [pid 15383] Using nickname Server-Cert.
>>>> [...] [:error] [pid 15383] Certificate not found: 'Server-Cert'
>>>>
>>>> when I want to start FreeIPA via "systemctl start ipa" or
"ipactl
>>>> start"
>>>> or "systemctl start httpd"
>>>> If I turn the NSSEngine off it starts of cause.
>>>>
>>>> In contrast to this message "ipa-getcert list -d /etc/httpd/alias/
-n
>>>> Server-Cert" does find a certificate, if I get the output [1]
right.
>>>
>>> ipa-getcert shows certs that are tracked by certmonger but doesn't
>>> guarantee that those certificates actually exist in the filesystem
>>> (they
>>> did at the time tracking was started).
>>>
>>> You need to look at the Apache NSS database:
>>>
>>> # certutil -L -d /etc/httpd/alias
>> Ok, I also did this, but it seems to be there
>> # certutil -L -d /etc/httpd/alias
>>
>> Certificate Nickname Trust
>> Attributes
>>
>> SSL,S/MIME,JAR/XPI
>>
>> Signing-Cert u,u,u
>> ipaCert u,u,u
>> Server-Cert Pu,u,u
>> EXAMPLE.COM IPA CA CT,C,C
>
>
> I'd check FS permissions. /etc/httpd/alias/*.db should be root:apache
> 0640
ok, the db were "root:apache 0660", but they were readable at least and
making them 0640 did not help either.
>
> If that checks out, look for SELinux issues by starting httpd then
> running: ausearch -m AVC -ts recent
I disabled SELinux for testing it, but that did not work. Now I also
tested:
# ausearch -m AVC -ts recent
<no matches>
>
> As a last resort perhaps the NSS database is corrupted. You can exercise
> it with:
>
> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
> /etc/httpd/alias/pwdfile.txt
>
> You should get: certutil: certificate is valid
>
I do get it:
# certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
/etc/httpd/alias/pwdfile.txt
certutil: certificate is valid
If I just want to start httpd and not via IPA or with --force I get a
different error, which I think might be because the services started
before httpd in the IPA start-up-phase aren't running since the start of
IPA aborted:
-- Unit httpd.service has begun starting up.
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
: ERROR Unknown error while retrieving setting from ldap
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
Traceback (most recent call last):
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/libexec/ipa/ipa-httpd-kdcproxy", line 84, in _ldap_con
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.con.do_bind(timeout=self.time_limit)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.do_external_bind(pw_name, timeout=timeout)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.__bind_with_wait(self.external_bind, timeout, user_name)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
self.__wait_for_connection(timeout)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
wait_for_open_socket(lurl.hostport, timeout)
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
"/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 13
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: raise e
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: error:
[Errno 111] Connection refused
Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
: ERROR Unknown error while retrieving setting from ldap
Aug 14 19:05:14 ipa_server.example.com systemd[1]: httpd.service:
Control process exited, code=exited status=1
Aug 14 19:05:14 ipa_server.example.com audit[1]: SERVICE_START pid=1
uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s
Aug 14 19:05:14 ipa_server.example.com systemd[1]: Failed to start The
Apache HTTP Server.
The KDC proxy needs to talk to LDAP. If you want to continue down this
road you can edit /etc/systemd/system/httpd.service.d/ipa.conf and
comment out the ExecStartPre command, run systemctl daemon-reload and
try to start Apache (you just really need to remember to undo this).
That is a very strange and unexpected error out of mod_nss. What distro
are you running and what version of mod_nss?
Can you share your nss.conf?
rob
3:28 p.m.
On 08/14/2017 09:51 PM, Rob Crittenden wrote:
Julian Gethmann wrote:
> On 08/14/2017 05:46 PM, Rob Crittenden wrote:
>> Julian Gethmann wrote:
>>> Hallo,
>>>
>>> On 08/14/2017 04:21 PM, Rob Crittenden wrote:
>>>> Julian Gethmann via FreeIPA-users wrote:
>>>>> Hallo,
>>>>>
>>>>> Unfortunately I don't know when this problem occurred first, but
it
>>>>> may
>>>>> have occurred after an update.
>>>>> The httpd does not start and aborts with the error
>>>>>
>>>>> [:info] [pid 15383] Using nickname Server-Cert.
>>>>> [...] [:error] [pid 15383] Certificate not found:
'Server-Cert'
>>>>>
>>>>> when I want to start FreeIPA via "systemctl start ipa" or
"ipactl
>>>>> start"
>>>>> or "systemctl start httpd"
>>>>> If I turn the NSSEngine off it starts of cause.
>>>>>
>>>>> In contrast to this message "ipa-getcert list -d
/etc/httpd/alias/ -n
>>>>> Server-Cert" does find a certificate, if I get the output [1]
right.
>>>>
>>>> ipa-getcert shows certs that are tracked by certmonger but doesn't
>>>> guarantee that those certificates actually exist in the filesystem
>>>> (they
>>>> did at the time tracking was started).
>>>>
>>>> You need to look at the Apache NSS database:
>>>>
>>>> # certutil -L -d /etc/httpd/alias
>>> Ok, I also did this, but it seems to be there
>>> # certutil -L -d /etc/httpd/alias
>>>
>>> Certificate Nickname Trust
>>> Attributes
>>>
>>> SSL,S/MIME,JAR/XPI
>>>
>>> Signing-Cert u,u,u
>>> ipaCert u,u,u
>>> Server-Cert Pu,u,u
>>> EXAMPLE.COM IPA CA CT,C,C
>>
>>
>> I'd check FS permissions. /etc/httpd/alias/*.db should be root:apache
>> 0640
> ok, the db were "root:apache 0660", but they were readable at least and
> making them 0640 did not help either.
>>
>> If that checks out, look for SELinux issues by starting httpd then
>> running: ausearch -m AVC -ts recent
> I disabled SELinux for testing it, but that did not work. Now I also
> tested:
> # ausearch -m AVC -ts recent
> <no matches>
>
>>
>> As a last resort perhaps the NSS database is corrupted. You can exercise
>> it with:
>>
>> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
>> /etc/httpd/alias/pwdfile.txt
>>
>> You should get: certutil: certificate is valid
>>
> I do get it:
> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
> /etc/httpd/alias/pwdfile.txt
> certutil: certificate is valid
>
>
> If I just want to start httpd and not via IPA or with --force I get a
> different error, which I think might be because the services started
> before httpd in the IPA start-up-phase aren't running since the start of
> IPA aborted:
>
> -- Unit httpd.service has begun starting up.
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
> : ERROR Unknown error while retrieving setting from ldap
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
> Traceback (most recent call last):
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
> "/usr/libexec/ipa/ipa-httpd-kdcproxy", line 84, in _ldap_con
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
> self.con.do_bind(timeout=self.time_limit)
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
> self.do_external_bind(pw_name, timeout=timeout)
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
> self.__bind_with_wait(self.external_bind, timeout, user_name)
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
> self.__wait_for_connection(timeout)
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
> wait_for_open_socket(lurl.hostport, timeout)
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
> "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 13
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: raise e
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: error:
> [Errno 111] Connection refused
> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
> : ERROR Unknown error while retrieving setting from ldap
> Aug 14 19:05:14 ipa_server.example.com systemd[1]: httpd.service:
> Control process exited, code=exited status=1
> Aug 14 19:05:14 ipa_server.example.com audit[1]: SERVICE_START pid=1
> uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s
> Aug 14 19:05:14 ipa_server.example.com systemd[1]: Failed to start The
> Apache HTTP Server.
>
The KDC proxy needs to talk to LDAP. If you want to continue down this
road you can edit /etc/systemd/system/httpd.service.d/ipa.conf and
comment out the ExecStartPre command, run systemctl daemon-reload and
try to start Apache (you just really need to remember to undo this).
Ok. Now the
error is "Certificate not found: 'Server-Cert'" again.
That is a very strange and unexpected error out of mod_nss. What distro
Fedora
Server 26
are you running and what version of mod_nss?
Version: 1.0.14
Release: 3.fc26
Can you share your nss.conf?
Sure,
https://paste.fedoraproject.org/paste/HAEpFrh3reUlZZoCpARAXA
rob
Thanks.
Julian
8:30 a.m.
Julian Gethmann wrote:
On 08/14/2017 09:51 PM, Rob Crittenden wrote:
> Julian Gethmann wrote:
>> On 08/14/2017 05:46 PM, Rob Crittenden wrote:
>>> Julian Gethmann wrote:
>>>> Hallo,
>>>>
>>>> On 08/14/2017 04:21 PM, Rob Crittenden wrote:
>>>>> Julian Gethmann via FreeIPA-users wrote:
>>>>>> Hallo,
>>>>>>
>>>>>> Unfortunately I don't know when this problem occurred first,
but it
>>>>>> may
>>>>>> have occurred after an update.
>>>>>> The httpd does not start and aborts with the error
>>>>>>
>>>>>> [:info] [pid 15383] Using nickname Server-Cert.
>>>>>> [...] [:error] [pid 15383] Certificate not found:
'Server-Cert'
>>>>>>
>>>>>> when I want to start FreeIPA via "systemctl start ipa"
or "ipactl
>>>>>> start"
>>>>>> or "systemctl start httpd"
>>>>>> If I turn the NSSEngine off it starts of cause.
>>>>>>
>>>>>> In contrast to this message "ipa-getcert list -d
>>>>>> /etc/httpd/alias/ -n
>>>>>> Server-Cert" does find a certificate, if I get the output
[1] right.
>>>>>
>>>>> ipa-getcert shows certs that are tracked by certmonger but
doesn't
>>>>> guarantee that those certificates actually exist in the filesystem
>>>>> (they
>>>>> did at the time tracking was started).
>>>>>
>>>>> You need to look at the Apache NSS database:
>>>>>
>>>>> # certutil -L -d /etc/httpd/alias
>>>> Ok, I also did this, but it seems to be there
>>>> # certutil -L -d /etc/httpd/alias
>>>>
>>>> Certificate Nickname Trust
>>>> Attributes
>>>>
>>>> SSL,S/MIME,JAR/XPI
>>>>
>>>> Signing-Cert u,u,u
>>>> ipaCert u,u,u
>>>> Server-Cert Pu,u,u
>>>> EXAMPLE.COM IPA CA CT,C,C
>>>
>>>
>>> I'd check FS permissions. /etc/httpd/alias/*.db should be root:apache
>>> 0640
>> ok, the db were "root:apache 0660", but they were readable at least
and
>> making them 0640 did not help either.
>>>
>>> If that checks out, look for SELinux issues by starting httpd then
>>> running: ausearch -m AVC -ts recent
>> I disabled SELinux for testing it, but that did not work. Now I also
>> tested:
>> # ausearch -m AVC -ts recent
>> <no matches>
>>
>>>
>>> As a last resort perhaps the NSS database is corrupted. You can
>>> exercise
>>> it with:
>>>
>>> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
>>> /etc/httpd/alias/pwdfile.txt
>>>
>>> You should get: certutil: certificate is valid
>>>
>> I do get it:
>> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
>> /etc/httpd/alias/pwdfile.txt
>> certutil: certificate is valid
>>
>>
>> If I just want to start httpd and not via IPA or with --force I get a
>> different error, which I think might be because the services started
>> before httpd in the IPA start-up-phase aren't running since the start of
>> IPA aborted:
>>
>> -- Unit httpd.service has begun starting up.
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
>> : ERROR Unknown error while retrieving setting from ldap
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> Traceback (most recent call last):
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>> "/usr/libexec/ipa/ipa-httpd-kdcproxy", line 84, in _ldap_con
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> self.con.do_bind(timeout=self.time_limit)
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> self.do_external_bind(pw_name, timeout=timeout)
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> self.__bind_with_wait(self.external_bind, timeout, user_name)
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> self.__wait_for_connection(timeout)
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> wait_for_open_socket(lurl.hostport, timeout)
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>> "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 13
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>> raise e
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: error:
>> [Errno 111] Connection refused
>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
>> : ERROR Unknown error while retrieving setting from ldap
>> Aug 14 19:05:14 ipa_server.example.com systemd[1]: httpd.service:
>> Control process exited, code=exited status=1
>> Aug 14 19:05:14 ipa_server.example.com audit[1]: SERVICE_START pid=1
>> uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s
>> Aug 14 19:05:14 ipa_server.example.com systemd[1]: Failed to start The
>> Apache HTTP Server.
>>
>
> The KDC proxy needs to talk to LDAP. If you want to continue down this
> road you can edit /etc/systemd/system/httpd.service.d/ipa.conf and
> comment out the ExecStartPre command, run systemctl daemon-reload and
> try to start Apache (you just really need to remember to undo this).
Ok. Now the error is "Certificate not found: 'Server-Cert'" again.
>
> That is a very strange and unexpected error out of mod_nss. What distro
Fedora Server 26
> are you running and what version of mod_nss?
Version: 1.0.14 Release: 3.fc26
>
> Can you share your nss.conf?
Sure, https://paste.fedoraproject.org/paste/HAEpFrh3reUlZZoCpARAXA
Ok, my quiver is running out of arrows. I'm a bit stumped here.
I think we can rule out IPA as a problem, this is just mod_nss not being
able to grok your certificate database for some reason.
Can you try starting Apache again and look for SELinux issues: ausearch
-m AVC -ts recent
Can you provide me, privately if you'd like, the error log from Apache?
It might hold some clues but the code that looks up certs by nickname is
dead simple so as I said, I'm a bit baffled.
rob
4:32 a.m.
On 08/15/2017 03:30 PM, Rob Crittenden via FreeIPA-users wrote:
Julian Gethmann wrote:
> On 08/14/2017 09:51 PM, Rob Crittenden wrote:
>> Julian Gethmann wrote:
>>> On 08/14/2017 05:46 PM, Rob Crittenden wrote:
>>>> Julian Gethmann wrote:
>>>>> Hallo,
>>>>>
>>>>> On 08/14/2017 04:21 PM, Rob Crittenden wrote:
>>>>>> Julian Gethmann via FreeIPA-users wrote:
>>>>>>> Hallo,
>>>>>>>
>>>>>>> Unfortunately I don't know when this problem occurred
first, but it
>>>>>>> may
>>>>>>> have occurred after an update.
>>>>>>> The httpd does not start and aborts with the error
>>>>>>>
>>>>>>> [:info] [pid 15383] Using nickname Server-Cert.
>>>>>>> [...] [:error] [pid 15383] Certificate not found:
'Server-Cert'
>>>>>>>
>>>>>>> when I want to start FreeIPA via "systemctl start
ipa" or "ipactl
>>>>>>> start"
>>>>>>> or "systemctl start httpd"
>>>>>>> If I turn the NSSEngine off it starts of cause.
>>>>>>>
>>>>>>> In contrast to this message "ipa-getcert list -d
>>>>>>> /etc/httpd/alias/ -n
>>>>>>> Server-Cert" does find a certificate, if I get the
output [1] right.
>>>>>>
>>>>>> ipa-getcert shows certs that are tracked by certmonger but
doesn't
>>>>>> guarantee that those certificates actually exist in the
filesystem
>>>>>> (they
>>>>>> did at the time tracking was started).
>>>>>>
>>>>>> You need to look at the Apache NSS database:
>>>>>>
>>>>>> # certutil -L -d /etc/httpd/alias
>>>>> Ok, I also did this, but it seems to be there
>>>>> # certutil -L -d /etc/httpd/alias
>>>>>
>>>>> Certificate Nickname Trust
>>>>> Attributes
>>>>>
>>>>> SSL,S/MIME,JAR/XPI
>>>>>
>>>>> Signing-Cert u,u,u
>>>>> ipaCert u,u,u
>>>>> Server-Cert Pu,u,u
>>>>> EXAMPLE.COM IPA CA CT,C,C
>>>>
>>>>
>>>> I'd check FS permissions. /etc/httpd/alias/*.db should be
root:apache
>>>> 0640
>>> ok, the db were "root:apache 0660", but they were readable at least
and
>>> making them 0640 did not help either.
>>>>
>>>> If that checks out, look for SELinux issues by starting httpd then
>>>> running: ausearch -m AVC -ts recent
>>> I disabled SELinux for testing it, but that did not work. Now I also
>>> tested:
>>> # ausearch -m AVC -ts recent
>>> <no matches>
>>>
>>>>
>>>> As a last resort perhaps the NSS database is corrupted. You can
>>>> exercise
>>>> it with:
>>>>
>>>> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
>>>> /etc/httpd/alias/pwdfile.txt
>>>>
>>>> You should get: certutil: certificate is valid
>>>>
>>> I do get it:
>>> # certutil -V -u V -n Server-Cert -d /etc/httpd/alias -e -f
>>> /etc/httpd/alias/pwdfile.txt
>>> certutil: certificate is valid
>>>
>>>
>>> If I just want to start httpd and not via IPA or with --force I get a
>>> different error, which I think might be because the services started
>>> before httpd in the IPA start-up-phase aren't running since the start of
>>> IPA aborted:
>>>
>>> -- Unit httpd.service has begun starting up.
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
>>> : ERROR Unknown error while retrieving setting from ldap
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> Traceback (most recent call last):
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>>> "/usr/libexec/ipa/ipa-httpd-kdcproxy", line 84, in _ldap_con
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> self.con.do_bind(timeout=self.time_limit)
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> self.do_external_bind(pw_name, timeout=timeout)
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> self.__bind_with_wait(self.external_bind, timeout, user_name)
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> self.__wait_for_connection(timeout)
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 16
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> wait_for_open_socket(lurl.hostport, timeout)
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 13
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]:
>>> raise e
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: error:
>>> [Errno 111] Connection refused
>>> Aug 14 19:05:14 ipa_server.example.com ipa-httpd-kdcproxy[22551]: ipa
>>> : ERROR Unknown error while retrieving setting from ldap
>>> Aug 14 19:05:14 ipa_server.example.com systemd[1]: httpd.service:
>>> Control process exited, code=exited status=1
>>> Aug 14 19:05:14 ipa_server.example.com audit[1]: SERVICE_START pid=1
>>> uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s
>>> Aug 14 19:05:14 ipa_server.example.com systemd[1]: Failed to start The
>>> Apache HTTP Server.
>>>
>>
>> The KDC proxy needs to talk to LDAP. If you want to continue down this
>> road you can edit /etc/systemd/system/httpd.service.d/ipa.conf and
>> comment out the ExecStartPre command, run systemctl daemon-reload and
>> try to start Apache (you just really need to remember to undo this).
> Ok. Now the error is "Certificate not found: 'Server-Cert'" again.
>>
>> That is a very strange and unexpected error out of mod_nss. What distro
> Fedora Server 26
>> are you running and what version of mod_nss?
> Version: 1.0.14 Release: 3.fc26
>>
>> Can you share your nss.conf?
> Sure, https://paste.fedoraproject.org/paste/HAEpFrh3reUlZZoCpARAXA
Ok, my quiver is running out of arrows. I'm a bit stumped here.
I think we can rule out IPA as a problem, this is just mod_nss not being
able to grok your certificate database for some reason.
Can you try starting Apache again and look for SELinux issues: ausearch
-m AVC -ts recent
Can you provide me, privately if you'd like, the error log from Apache?
It might hold some clues but the code that looks up certs by nickname is
dead simple so as I said, I'm a bit baffled.
rob
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Hi,
you can also check if Apache is able to access the private key, which is
protected by a password.
Find the file storing the password:
$ sudo grep NSSPassPhraseDialog /etc/httpd/conf.d/nss.conf
NSSPassPhraseDialog "file:/etc/httpd/conf/password.conf"
Find the password:
$ sudo grep internal /etc/httpd/conf/password.conf| cut -d: -f2-
Test the password to access the private key:
$ sudo certutil -K -d /etc/httpd/alias/
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private
Key and Certificate Services"
Enter Password or Pin for "NSS Certificate DB": <<<< enter the
password
< 0> rsa f19a8e6e3b823cb999818e2960fe5225c1f8bab9 NSS Certificate
DB:Server-Cert
If the password is OK, certutil should display a line for Server-Cert.
Flo
2456
days inactive
2465
days old
freeipa-users@lists.fedorahosted.org
8 comments
3 participants
participants (3)
-
Florence Blanc-Renaud -
Julian Gethmann -
Rob Crittenden