Hi wisdom of the list,
I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light.
A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate.
Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051 ). The one I can't work around however is below.
It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error.
Could the below caused by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 ?
Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems?
Thanks in advance,
David
2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net,O=THOMAC.NET" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_server_upgrade.py", line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data, overwrite=False) File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line 1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API'))
2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API 2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
Sorry for the dump size, but not sure if the below from /var/log/pki/pki-tomcat/localhost.date.log helps:
15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
On 15 November 2017 at 13:23, David Harvey davidcharvey@googlemail.com wrote:
Hi wisdom of the list,
I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light.
A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate.
Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs. launchpad.net/ubuntu/+source/freeipa/+bug/1703051 ). The one I can't work around however is below.
It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error.
Could the below caused by https://bugs.launchpad.net/ ubuntu/+source/freeipa/+bug/1716842 ?
Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems?
Thanks in advance,
David
2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net,O= THOMAC.NET" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial, sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white; background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial, sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm. findSecurityConstraints(ProxyRealm.java:138)\n\torg. apache.catalina.authenticator.AuthenticatorBase.invoke( AuthenticatorBase.java:498)\n\torg.apache.catalina.valves. ErrorReportValve.invoke(ErrorReportValve.java:79)\n\ torg.apache.catalina.valves.AbstractAccessLogValve.invoke( AbstractAccessLogValve.java:620)\n\torg.apache.catalina. connector.CoyoteAdapter.service(CoyoteAdapter.java: 502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process( AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$ AbstractConnectionHandler.process(AbstractProtocol.java: 684)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run( JIoEndpoint.java:283)\n\tjava.util.concurrent. ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\ n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$ WrappingRunnable.run(TaskThread.java:61)\n\tjava. lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist- packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_server_upgrade.py", line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data, overwrite=False) File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line 1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API'))
2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API 2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
David Harvey via FreeIPA-users wrote:
Sorry for the dump size, but not sure if the below from /var/log/pki/pki-tomcat/localhost.date.log helps:
Looks like the selftests are failing. I'd check that your CA subsystem certificates are not expired, etc.
rob
15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
On 15 November 2017 at 13:23, David Harvey <davidcharvey@googlemail.com mailto:davidcharvey@googlemail.com> wrote:
Hi wisdom of the list, I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light. A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate. Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051> ). The one I can't work around however is below. It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error. Could the below caused by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842> ? Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems? Thanks in advance, David 2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net <http://ipa1.my.net>,O=THOMAC.NET <http://THOMAC.NET>" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs <http://ws.rs>.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_server_upgrade.py", line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data, overwrite=False) File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line 1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API')) 2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API 2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Thanks Rob, Simon,
Rob, will check, but thought my cert system was healthy before. It's relatively new (6months or less), and no sub-ca's involved.. Any specifics on how to invoke the selftests in some manner that might provide digestible output? Or could it be my dirty hack of cloning and isolation and I should do as Simon suggested :)?
Simon. WRT spinning up a replica. I was under the impression that all running servers had to be of the same version, am I mistaken with that? I had avoided what you were suggesting as I feared the new server might update the schema on the existing ones!
Thanks again, appreciate the steering!
On 15 Nov 2017 14:34, "Rob Crittenden" rcritten@redhat.com wrote:
David Harvey via FreeIPA-users wrote:
Sorry for the dump size, but not sure if the below from /var/log/pki/pki-tomcat/localhost.date.log helps:
Looks like the selftests are failing. I'd check that your CA subsystem certificates are not expired, etc.
rob
15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(
ProxyRealm.java:138)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:498)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(
AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(
AbstractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.
process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.
run(JIoEndpoint.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(
TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(
ProxyRealm.java:138)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:498)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(
AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(
AbstractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.
process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.
run(JIoEndpoint.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(
TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
On 15 November 2017 at 13:23, David Harvey <davidcharvey@googlemail.com mailto:davidcharvey@googlemail.com> wrote:
Hi wisdom of the list, I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light. A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate. Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051> ). The one I can't work around however is below. It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error. Could the below caused by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842> ? Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems? Thanks in advance, David 2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net <http://ipa1.my.net>,O=THOMAC.NET <http://THOMAC.NET>" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;font-size:22px;}
H2 {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;font-size:16px;}
H3 {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;font-size:14px;}
BODY {font-family:Tahoma,Arial,sans-serif;color:black;
background-color:white;}
B {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;}
P {font-family:Tahoma,Arial,sans-serif;background:white;
color:black;font-size:12px;}A
{color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs <http://ws.rs>.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.
findSecurityConstraints(ProxyRealm.java:138)\n\torg. apache.catalina.authenticator.AuthenticatorBase.invoke( AuthenticatorBase.java:498)\n\torg.apache.catalina.valves. ErrorReportValve.invoke(ErrorReportValve.java:79)\n\ torg.apache.catalina.valves.AbstractAccessLogValve.invoke( AbstractAccessLogValve.java:620)\n\torg.apache.catalina. connector.CoyoteAdapter.service(CoyoteAdapter.java: 502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process( AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$ AbstractConnectionHandler.process(AbstractProtocol.java: 684)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run( JIoEndpoint.java:283)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent. ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\ n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java: 748)\n</pre><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/
ipa_server_upgrade.py",
line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/
server/upgrade.py",
line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/
server/upgrade.py",
line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/
server/upgrade.py",
line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data, overwrite=False) File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line 1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API')) 2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API 2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Hi again,
No joy yet with spotting CA anomalies. Any additional tips there Rob?
Gentle bump Simon, are you confident that building a new replica won't fall foul of the below from the upgrade page (the schema part):
Words of caution
- Note that the server is in a *maintenance mode* during upgrade and does not respond to requests! - Schema or Directory Server https://www.freeipa.org/page/Directory_Server database object changes done during the upgrade are replicated to *all FreeIPA masters*
Thanks again for the support,
David
On 15 November 2017 at 16:52, David Harvey davidcharvey@googlemail.com wrote:
Thanks Rob, Simon,
Rob, will check, but thought my cert system was healthy before. It's relatively new (6months or less), and no sub-ca's involved.. Any specifics on how to invoke the selftests in some manner that might provide digestible output? Or could it be my dirty hack of cloning and isolation and I should do as Simon suggested :)?
Simon. WRT spinning up a replica. I was under the impression that all running servers had to be of the same version, am I mistaken with that? I had avoided what you were suggesting as I feared the new server might update the schema on the existing ones!
Thanks again, appreciate the steering!
On 15 Nov 2017 14:34, "Rob Crittenden" rcritten@redhat.com wrote:
David Harvey via FreeIPA-users wrote:
Sorry for the dump size, but not sure if the below from /var/log/pki/pki-tomcat/localhost.date.log helps:
Looks like the selftests are failing. I'd check that your CA subsystem certificates are not expired, etc.
rob
15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log
StandardWrapper.Throwable
java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(
SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(
Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(
SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(
Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(P
roxyRealm.java:138)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(A
uthenticatorBase.java:498)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHo
stValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorRepo
rtValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abs
tractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(Standard
EngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAd
apter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(Abs
tractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler
.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(
JIoEndpoint.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.
run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log
StandardWrapper.Throwable
java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(
SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(
Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(
SelfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(
Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(P
roxyRealm.java:138)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(A
uthenticatorBase.java:498)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHo
stValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorRepo
rtValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abs
tractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(Standard
EngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAd
apter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(Abs
tractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler
.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(
JIoEndpoint.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.
run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
On 15 November 2017 at 13:23, David Harvey <davidcharvey@googlemail.com mailto:davidcharvey@googlemail.com> wrote:
Hi wisdom of the list, I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light. A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate. Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/17030
51
<https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051> ). The one I can't work around however is below. It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error. Could the below caused by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842> ? Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems? Thanks in advance, David 2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net <http://ipa1.my.net>,O=THOMAC.NET <http://THOMAC.NET>" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;font-size:22px;}
H2 {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;font-size:16px;}
H3 {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;font-size:14px;}
BODY {font-family:Tahoma,Arial,sans-serif;color:black;background
-color:white;}
B {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;}
P {font-family:Tahoma,Arial,sans-serif;background:white;color
:black;font-size:12px;}A
{color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs <http://ws.rs>.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecur
ityConstraints(ProxyRealm.java:138)\n\torg.apache. catalina.authenticator.AuthenticatorBase.invoke(Authenticato rBase.java:498)\n\torg.apache.catalina.valves.ErrorReportVal ve.invoke(ErrorReportValve.java:79)\n\torg.apache. catalina.valves.AbstractAccessLogValve.invoke(AbstractAccess LogValve.java:620)\n\torg.apache.catalina.connector.CoyoteAdapter.service( CoyoteAdapter.java:502)\n\torg.apache.coyote.http11.Abst ractHttp11Processor.process(AbstractHttp11Processor.java:113 2)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionH andler.process(AbstractProtocol.java:684)\n\torg.apache.tomcat.util.net. JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283)\n\ tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo lExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExec utor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache. tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\ tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_
server_upgrade.py",
line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/
upgrade.py",
line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/
upgrade.py",
line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/
upgrade.py",
line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data,
overwrite=False)
File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line 1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API')) 2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API 2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedo
rahosted.org
Hoi,
Anyone out there with experience of whether or not adding a replica of more recent version (4.4.4 and 389 dir 1.3.7.5-1 up from 4.4.3 with 389 dir 1.3.5.15-2) would impact the existing servers in terms of schema or similar? I'm still trying to find a safe way to upgrade safely without going past a point of no return...
Kind regards,
David
On 17 November 2017 at 15:10, David Harvey davidcharvey@googlemail.com wrote:
Hi again,
No joy yet with spotting CA anomalies. Any additional tips there Rob?
Gentle bump Simon, are you confident that building a new replica won't fall foul of the below from the upgrade page (the schema part):
Words of caution
- Note that the server is in a *maintenance mode* during upgrade and
does not respond to requests!
- Schema or Directory Server
https://www.freeipa.org/page/Directory_Server database object changes done during the upgrade are replicated to *all FreeIPA masters*
Thanks again for the support,
David
On 15 November 2017 at 16:52, David Harvey davidcharvey@googlemail.com wrote:
Thanks Rob, Simon,
Rob, will check, but thought my cert system was healthy before. It's relatively new (6months or less), and no sub-ca's involved.. Any specifics on how to invoke the selftests in some manner that might provide digestible output? Or could it be my dirty hack of cloning and isolation and I should do as Simon suggested :)?
Simon. WRT spinning up a replica. I was under the impression that all running servers had to be of the same version, am I mistaken with that? I had avoided what you were suggesting as I feared the new server might update the schema on the existing ones!
Thanks again, appreciate the steering!
On 15 Nov 2017 14:34, "Rob Crittenden" rcritten@redhat.com wrote:
David Harvey via FreeIPA-users wrote:
Sorry for the dump size, but not sure if the below from /var/log/pki/pki-tomcat/localhost.date.log helps:
Looks like the selftests are failing. I'd check that your CA subsystem certificates are not expired, etc.
rob
15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log
StandardWrapper.Throwable
java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(Se
lfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:717)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet
[castart]
in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(Se
lfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:717)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(P
roxyRealm.java:138)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(A
uthenticatorBase.java:498)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHo
stValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorRepo
rtValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abs
tractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(Standard
EngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAd
apter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(Abs
tractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler
.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(J
IoEndpoint.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.r
un(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log
StandardWrapper.Throwable
java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(Se
lfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:717)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet
[castart]
in web application [/ca] threw load() exception java.lang.NullPointerException at com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(Se
lfTestSubsystem.java:1886)
at com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEn
gine.java:2118)
at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158) at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1227)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1140)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1027)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5038)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5348)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:729)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:717)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:621)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1835)
at java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] org.apache.catalina.core.StandardHostValve.invoke Exception Processing /ca/rest/account/login javax.ws.rs.ServiceUnavailableException: Subsystem unavailable at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(P
roxyRealm.java:138)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(A
uthenticatorBase.java:498)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHo
stValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorRepo
rtValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abs
tractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(Standard
EngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAd
apter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(Abs
tractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler
.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(J
IoEndpoint.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.r
un(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
On 15 November 2017 at 13:23, David Harvey <davidcharvey@googlemail.com mailto:davidcharvey@googlemail.com> wrote:
Hi wisdom of the list, I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light. A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate. Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs.launchpad.net/ubu
ntu/+source/freeipa/+bug/1703051
<https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051> ). The one I can't work around however is below. It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error. Could the below caused by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842> ? Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems? Thanks in advance, David 2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net <http://ipa1.my.net>,O=THOMAC.NET <http://THOMAC.NET>" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;font-size:22px;}
H2 {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;font-size:16px;}
H3 {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;font-size:14px;}
BODY {font-family:Tahoma,Arial,sans-serif;color:black;background
-color:white;}
B {font-family:Tahoma,Arial,sans-serif;color:white;background
-color:#525D76;}
P {font-family:Tahoma,Arial,sans-serif;background:white;color
:black;font-size:12px;}A
{color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs <http://ws.rs>.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecur
ityConstraints(ProxyRealm.java:138)\n\torg.apache.catalina. authenticator.AuthenticatorBase.invoke(AuthenticatorBase. java:498)\n\torg.apache.catalina.valves.ErrorReportValve. invoke(ErrorReportValve.java:79)\n\torg.apache.catalina. valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve. java:620)\n\torg.apache.catalina.connector.CoyoteAdapt er.service(CoyoteAdapter.java:502)\n\torg.apache.coyote. http11.AbstractHttp11Processor.process(AbstractHttp11Process or.java:1132)\n\torg.apache.coyote.AbstractProtocol$ AbstractConnectionHandler.process(AbstractProtocol.java:684)\n\ torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run( JIoEndpoint.java:283)\n\tjava.util.concurrent.ThreadPoolExec utor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util. concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. java:624)\n\torg.apache.tomcat.util.threads.TaskThread$ WrappingRunnable.run(TaskThread.java:61)\n\tjava. lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line
172,
in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_ser
ver_upgrade.py",
line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/
upgrade.py",
line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/
upgrade.py",
line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/
upgrade.py",
line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data,
overwrite=False)
File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py",
line
1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API')) 2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST
API
2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedo
rahosted.org
David Harvey wrote:
Hoi,
Anyone out there with experience of whether or not adding a replica of more recent version (4.4.4 and 389 dir 1.3.7.5-1 up from 4.4.3 with 389 dir 1.3.5.15-2) would impact the existing servers in terms of schema or similar? I'm still trying to find a safe way to upgrade safely without going past a point of no return...
Yes, creating a replica with a newer version can add schema and modify existing LDAP entries (like ACIs).
rob
Kind regards,
David
On 17 November 2017 at 15:10, David Harvey <davidcharvey@googlemail.com mailto:davidcharvey@googlemail.com> wrote:
Hi again, No joy yet with spotting CA anomalies. Any additional tips there Rob? Gentle bump Simon, are you confident that building a new replica won't fall foul of the below from the upgrade page (the schema part): Words of caution * Note that the server is in a *maintenance mode* during upgrade and does not respond to requests! * Schema or Directory Server <https://www.freeipa.org/page/Directory_Server> database object changes done during the upgrade are replicated to *all FreeIPA masters* * * Thanks again for the support, David On 15 November 2017 at 16:52, David Harvey <davidcharvey@googlemail.com <mailto:davidcharvey@googlemail.com>> wrote: Thanks Rob, Simon, Rob, will check, but thought my cert system was healthy before. It's relatively new (6months or less), and no sub-ca's involved.. Any specifics on how to invoke the selftests in some manner that might provide digestible output? Or could it be my dirty hack of cloning and isolation and I should do as Simon suggested :)? Simon. WRT spinning up a replica. I was under the impression that all running servers had to be of the same version, am I mistaken with that? I had avoided what you were suggesting as I feared the new server might update the schema on the existing ones! Thanks again, appreciate the steering! On 15 Nov 2017 14:34, "Rob Crittenden" <rcritten@redhat.com <mailto:rcritten@redhat.com>> wrote: David Harvey via FreeIPA-users wrote: > Sorry for the dump size, but not sure if the below from > /var/log/pki/pki-tomcat/localhost.date.log helps: Looks like the selftests are failing. I'd check that your CA subsystem certificates are not expired, etc. rob > > 15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] > org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) > at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) > at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) > at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) > at javax.servlet.GenericServlet.init(GenericServlet.java:158) > at > org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) > at > org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) > at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) > at > org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) > at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) > at > org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) > at > org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] > org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] > in web application [/ca] threw load() exception > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) > at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) > at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) > at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) > at javax.servlet.GenericServlet.init(GenericServlet.java:158) > at > org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) > at > org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) > at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) > at > org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) > at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) > at > org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) > at > org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] > org.apache.catalina.core.StandardHostValve.invoke Exception Processing > /ca/rest/account/login > javax.ws.rs <http://javax.ws.rs>.ServiceUnavailableException: Subsystem unavailable > at > com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) > at > org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) > at > org.apache.tomcat.util.net <http://org.apache.tomcat.util.net>.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] > org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) > at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) > at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) > at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) > at javax.servlet.GenericServlet.init(GenericServlet.java:158) > at > org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) > at > org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) > at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) > at > org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) > at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) > at > org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) > at > org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] > org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] > in web application [/ca] threw load() exception > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.shutdown(SelfTestSubsystem.java:1886) > at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(CMSEngine.java:2118) > at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.java:2013) > at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) > at javax.servlet.GenericServlet.init(GenericServlet.java:158) > at > org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227) > at > org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140) > at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027) > at > org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348) > at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) > at > org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:621) > at > org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1835) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] > org.apache.catalina.core.StandardHostValve.invoke Exception Processing > /ca/rest/account/login > javax.ws.rs <http://javax.ws.rs>.ServiceUnavailableException: Subsystem unavailable > at > com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) > at > org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) > at > org.apache.tomcat.util.net <http://org.apache.tomcat.util.net>.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:748) > > > On 15 November 2017 at 13:23, David Harvey <davidcharvey@googlemail.com <mailto:davidcharvey@googlemail.com> > <mailto:davidcharvey@googlemail.com <mailto:davidcharvey@googlemail.com>>> wrote: > > Hi wisdom of the list, > > I know I am an edge case with running on ubuntu, but hoped someone > might be able to shed some light. > > A bit of background. I'm trying to test upgrades without > potentially hosing my existing services, so I have cloned the VM, > given it a new IP address, updated hosts file and pointed DNS > somewhere that doesn't know about the real IPA services (8.8.8.8) so > it won't try and sync or replicate. > > Attempting to upgrade hits a snags or two, some described in bugs > already like the pki version number confusing the apt > scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051> > <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051>> ). > The one I can't work around however is below. > > It seems deeply unhappy, and restarting the services result in the > dogtag-pki web page being available until a login attempt is made > (as occurs during the ipa-server-upgrade) after which point it bombs > with a 500 error. > > Could the below caused > by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842> > <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842>> ? > > Any advice appreciated, as I think even when 18.04 hits with the > proposed updates to rely on to tomcat 8.5, I'll still need to > upgrade via 17.10 which seems currently fraught! If it relates to > my method of cloning the VM, is there a better way of testing > upgrades without potentially hosing the existing live systems? > > > Thanks in advance, > > David > > 2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server > intended_usage = SSL Server > 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net <http://ipa1.my.net> > <http://ipa1.my.net>,O=THOMAC.NET <http://THOMAC.NET> <http://THOMAC.NET>" > 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS > 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 > 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA > 2017-11-15T13:05:59Z DEBUG response status 500 > 2017-11-15T13:05:59Z DEBUG response headers {'content-length': > '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', > 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', > 'content-type': 'text/html;charset=utf-8'} > 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE > html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error > report</title><style type="text/css">H1 > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} > H2 > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} > H3 > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} > BODY > {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} > B > {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} > P > {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A > {color : black;}A.name {color : black;}.line {height: 1px; > background-color: #525D76; border: none;}</style> > </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div > class="line"></div><p><b>type</b> Exception > report</p><p><b>message</b> <u>Subsystem > unavailable</u></p><p><b>description</b> <u>The server encountered > an internal error that prevented it from fulfilling this > request.</u></p><p><b>exception</b></p><pre>javax.ws.rs <http://javax.ws.rs> > <http://ws.rs>.ServiceUnavailableException: Subsystem > unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)\n\torg.apache.tomcat.util.net <http://torg.apache.tomcat.util.net>.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b> > <u>The full stack trace of the root cause is available in the Apache > Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache > Tomcat/8.0.46 (Ubuntu)</h3></body></html>' > 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect > /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. > 2017-11-15T13:05:59Z DEBUG File > "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, > in execute > return_value = self.run() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_server_upgrade.py", > line 46, in run > server.upgrade() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", > line 1878, in upgrade > upgrade_configuration() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", > line 1797, in upgrade_configuration > ca_enable_ldap_profile_subsystem(ca) > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", > line 347, in ca_enable_ldap_profile_subsystem > cainstance.migrate_profiles_to_ldap() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", > line 1981, in migrate_profiles_to_ldap > _create_dogtag_profile(profile_id, profile_data, overwrite=False) > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", > line 1987, in _create_dogtag_profile > with api.Backend.ra_certprofile as profile_api: > File > "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line > 1294, in __enter__ > raise errors.RemoteRetrieveError(reason=_('Failed to > authenticate to CA REST API')) > > 2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, > exception: RemoteRetrieveError: Failed to authenticate to CA REST API > 2017-11-15T13:05:59Z ERROR Unexpected error - see > /var/log/ipaupgrade.log for details: > RemoteRetrieveError: Failed to authenticate to CA REST API > > > > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> >
For anyone interested, I think I have it working properly after the following:
Edit /etc/pki/pki.version to remove +12 (confused the postinstall script).
Ensure you have kinit admin from the root session you're using to upgrade.
If like me you find the rest API on 8443 dies when being hit and gives a 501 or internal server error (in the IPA server install log) Install libtomcat8.0-java (which removes libtomcat8-java). Then the really weird bit. Kill the process you find with (ps aux | grep tomcat). Launch it again using the full command line ps aux gave you. Then running ipa-server-upgrade continues..
Not sure why tomcat is more resilient when launched as root, but the pki seems to work ok at issuing certs after the above and a reboot for good measure.
Hope this is of some help to someone! Typed with thumbs so excuse typos and memory fails.
David
On 21 Nov 2017 13:10, "Rob Crittenden" rcritten@redhat.com wrote:
David Harvey wrote:
Hoi,
Anyone out there with experience of whether or not adding a replica of more recent version (4.4.4 and 389 dir 1.3.7.5-1 up from 4.4.3 with 389 dir 1.3.5.15-2) would impact the existing servers in terms of schema or similar? I'm still trying to find a safe way to upgrade safely without going past a point of no return...
Yes, creating a replica with a newer version can add schema and modify existing LDAP entries (like ACIs).
rob
Kind regards,
David
On 17 November 2017 at 15:10, David Harvey <davidcharvey@googlemail.com mailto:davidcharvey@googlemail.com> wrote:
Hi again, No joy yet with spotting CA anomalies. Any additional tips there Rob? Gentle bump Simon, are you confident that building a new replica won't fall foul of the below from the upgrade page (the schema part): Words of caution * Note that the server is in a *maintenance mode* during upgrade and does not respond to requests! * Schema or Directory Server <https://www.freeipa.org/page/Directory_Server> database object changes done during the upgrade are replicated to *all FreeIPA masters* * * Thanks again for the support, David On 15 November 2017 at 16:52, David Harvey <davidcharvey@googlemail.com <mailto:davidcharvey@googlemail.com>> wrote: Thanks Rob, Simon, Rob, will check, but thought my cert system was healthy before. It's relatively new (6months or less), and no sub-ca's involved.. Any specifics on how to invoke the selftests in some manner that might provide digestible output? Or could it be my dirty hack of cloning and isolation and I should do as Simon suggested :)? Simon. WRT spinning up a replica. I was under the impression that all running servers had to be of the same version, am I mistaken with that? I had avoided what you were suggesting as I feared the new server might update the schema on the existing ones! Thanks again, appreciate the steering! On 15 Nov 2017 14:34, "Rob Crittenden" <rcritten@redhat.com <mailto:rcritten@redhat.com>> wrote: David Harvey via FreeIPA-users wrote: > Sorry for the dump size, but not sure if the below from > /var/log/pki/pki-tomcat/localhost.date.log helps: Looks like the selftests are failing. I'd check that your CA subsystem certificates are not expired, etc. rob > > 15-Nov-2017 12:14:50.557 SEVERE [localhost-startStop-1] > org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
> at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
> at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.
java:2013)
> at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
> at javax.servlet.GenericServlet.
init(GenericServlet.java:158)
> at > org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
> at > org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
> at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
> at > org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
> at > org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
> at org.apache.catalina.util.LifecycleBase.start(
LifecycleBase.java:145)
> at > org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
> at org.apache.catalina.core.ContainerBase.addChild(
ContainerBase.java:729)
> at org.apache.catalina.core.StandardHost.addChild(
StandardHost.java:717)
> at > org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
> at > org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
> at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.
java:266)
> at > java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
> at > java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 12:14:50.558 SEVERE [localhost-startStop-1] > org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] > in web application [/ca] threw load() exception > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
> at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
> at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.
java:2013)
> at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
> at javax.servlet.GenericServlet.
init(GenericServlet.java:158)
> at > org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
> at > org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
> at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
> at > org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
> at > org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
> at org.apache.catalina.util.LifecycleBase.start(
LifecycleBase.java:145)
> at > org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
> at org.apache.catalina.core.ContainerBase.addChild(
ContainerBase.java:729)
> at org.apache.catalina.core.StandardHost.addChild(
StandardHost.java:717)
> at > org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
> at > org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
> at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.
java:266)
> at > java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
> at > java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 12:14:54.509 SEVERE [http-bio-8443-exec-1] > org.apache.catalina.core.StandardHostValve.invoke Exception Processing > /ca/rest/account/login > javax.ws.rs <http://javax.ws.rs>.ServiceUnavailableException: Subsystem unavailable > at > com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(
ProxyRealm.java:138)
> at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:498)
> at > org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:141)
> at > org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:79)
> at > org.apache.catalina.valves.AbstractAccessLogValve.invoke(
AbstractAccessLogValve.java:620)
> at > org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:88)
> at > org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:502)
> at > org.apache.coyote.http11.AbstractHttp11Processor.process(
AbstractHttp11Processor.java:1132)
> at > org.apache.coyote.AbstractProtocol$
AbstractConnectionHandler.process(AbstractProtocol.java:684)
> at > org.apache.tomcat.util.net <http://org.apache.tomcat.util.net>.JIoEndpoint$
SocketProcessor.run(JIoEndpoint.java:283)
> at > java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
> at > java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
> at > org.apache.tomcat.util.threads.TaskThread$
WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 13:05:55.874 SEVERE [localhost-startStop-1] > org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
> at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
> at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.
java:2013)
> at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
> at javax.servlet.GenericServlet.
init(GenericServlet.java:158)
> at > org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
> at > org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
> at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
> at > org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
> at > org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
> at org.apache.catalina.util.LifecycleBase.start(
LifecycleBase.java:145)
> at > org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
> at org.apache.catalina.core.ContainerBase.addChild(
ContainerBase.java:729)
> at org.apache.catalina.core.StandardHost.addChild(
StandardHost.java:717)
> at > org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
> at > org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
> at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.
java:266)
> at > java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
> at > java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 13:05:55.875 SEVERE [localhost-startStop-1] > org.apache.catalina.core.StandardContext.loadOnStartup Servlet [castart] > in web application [/ca] threw load() exception > java.lang.NullPointerException > at > com.netscape.cmscore.selftests.SelfTestSubsystem.
shutdown(SelfTestSubsystem.java:1886)
> at > com.netscape.cmscore.apps.CMSEngine.shutdownSubsystems(
CMSEngine.java:2118)
> at com.netscape.cmscore.apps.CMSEngine.shutdown(CMSEngine.
java:2013)
> at com.netscape.certsrv.apps.CMS.shutdown(CMS.java:234) > at com.netscape.certsrv.apps.CMS.start(CMS.java:1630) > at > com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
> at javax.servlet.GenericServlet.
init(GenericServlet.java:158)
> at > org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1227)
> at > org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1140)
> at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1027)
> at > org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5038)
> at > org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5348)
> at org.apache.catalina.util.LifecycleBase.start(
LifecycleBase.java:145)
> at > org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:753)
> at org.apache.catalina.core.ContainerBase.addChild(
ContainerBase.java:729)
> at org.apache.catalina.core.StandardHost.addChild(
StandardHost.java:717)
> at > org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:621)
> at > org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1835)
> at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.
java:266)
> at > java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
> at > java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) > > 15-Nov-2017 13:05:59.706 SEVERE [http-bio-8443-exec-1] > org.apache.catalina.core.StandardHostValve.invoke Exception Processing > /ca/rest/account/login > javax.ws.rs <http://javax.ws.rs>.ServiceUnavailableException: Subsystem unavailable > at > com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(
ProxyRealm.java:138)
> at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:498)
> at > org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:141)
> at > org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:79)
> at > org.apache.catalina.valves.AbstractAccessLogValve.invoke(
AbstractAccessLogValve.java:620)
> at > org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:88)
> at > org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:502)
> at > org.apache.coyote.http11.AbstractHttp11Processor.process(
AbstractHttp11Processor.java:1132)
> at > org.apache.coyote.AbstractProtocol$
AbstractConnectionHandler.process(AbstractProtocol.java:684)
> at > org.apache.tomcat.util.net <http://org.apache.tomcat.util.net>.JIoEndpoint$
SocketProcessor.run(JIoEndpoint.java:283)
> at > java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1149)
> at > java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:624)
> at > org.apache.tomcat.util.threads.TaskThread$
WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748) > > > On 15 November 2017 at 13:23, David Harvey <davidcharvey@googlemail.com <mailto:davidcharvey@googlemail.com> > <mailto:davidcharvey@googlemail.com <mailto:davidcharvey@googlemail.com>>> wrote: > > Hi wisdom of the list, > > I know I am an edge case with running on ubuntu, but hoped someone > might be able to shed some light. > > A bit of background. I'm trying to test upgrades
without
> potentially hosing my existing services, so I have cloned the VM, > given it a new IP address, updated hosts file and pointed DNS > somewhere that doesn't know about the real IPA services (8.8.8.8) so > it won't try and sync or replicate. > > Attempting to upgrade hits a snags or two, some described in bugs > already like the pki version number confusing the apt > scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1703051
<https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1703051>
> <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1703051
<https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1703051>>
). > The one I can't work around however is below. > > It seems deeply unhappy, and restarting the services result in the > dogtag-pki web page being available until a login attempt is made > (as occurs during the ipa-server-upgrade) after which point it bombs > with a 500 error. > > Could the below caused > by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1716842
<https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1716842>
> <https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1716842
<https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/
1716842>>
? > > Any advice appreciated, as I think even when 18.04 hits with the > proposed updates to rely on to tomcat 8.5, I'll still need to > upgrade via 17.10 which seems currently fraught! If it relates to > my method of cloning the VM, is there a better way of testing > upgrades without potentially hosing the existing live systems? > > > Thanks in advance, > > David > > 2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server > intended_usage = SSL Server > 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net <http://ipa1.my.net> > <http://ipa1.my.net>,O=THOMAC.NET <http://THOMAC.NET> <http://THOMAC.NET>" > 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS > 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 > 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA > 2017-11-15T13:05:59Z DEBUG response status 500 > 2017-11-15T13:05:59Z DEBUG response headers {'content-length': > '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', > 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', > 'content-type': 'text/html;charset=utf-8'} > 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE > html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error > report</title><style type="text/css">H1 > {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;font-size:22px;}
> H2 > {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;font-size:16px;}
> H3 > {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;font-size:14px;}
> BODY > {font-family:Tahoma,Arial,sans-serif;color:black;
background-color:white;}
> B > {font-family:Tahoma,Arial,sans-serif;color:white;
background-color:#525D76;}
> P > {font-family:Tahoma,Arial,sans-serif;background:white;
color:black;font-size:12px;}A
> {color : black;}A.name {color : black;}.line {height:
1px;
> background-color: #525D76; border: none;}</style> > </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div > class="line"></div><p><b>type</b> Exception > report</p><p><b>message</b> <u>Subsystem > unavailable</u></p><p><b>description</b> <u>The server encountered > an internal error that prevented it from fulfilling
this
> request.</u></p><p><b>exception</b></p><pre>javax.ws.rs <http://javax.ws.rs> > <http://ws.rs>.ServiceUnavailableException: Subsystem > unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.
findSecurityConstraints(ProxyRealm.java:138)\n\torg. apache.catalina.authenticator.AuthenticatorBase.invoke( AuthenticatorBase.java:498)\n\torg.apache.catalina.valves. ErrorReportValve.invoke(ErrorReportValve.java:79)\n\ torg.apache.catalina.valves.AbstractAccessLogValve.invoke( AbstractAccessLogValve.java:620)\n\torg.apache.catalina. connector.CoyoteAdapter.service(CoyoteAdapter.java: 502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process( AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$ AbstractConnectionHandler.process(AbstractProtocol.java:684)\n\ torg.apache.tomcat.util.net
<http://torg.apache.tomcat.util.net>.JIoEndpoint$
SocketProcessor.run(JIoEndpoint.java:283)\n\tjava.util.concurrent. ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\ n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$ WrappingRunnable.run(TaskThread.java:61)\n\tjava. lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b>
> <u>The full stack trace of the root cause is available in the Apache > Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache > Tomcat/8.0.46 (Ubuntu)</h3></body></html>' > 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect > /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. > 2017-11-15T13:05:59Z DEBUG File > "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, > in execute > return_value = self.run() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/
ipa_server_upgrade.py",
> line 46, in run > server.upgrade() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/
server/upgrade.py",
> line 1878, in upgrade > upgrade_configuration() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/
server/upgrade.py",
> line 1797, in upgrade_configuration > ca_enable_ldap_profile_subsystem(ca) > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/
server/upgrade.py",
> line 347, in ca_enable_ldap_profile_subsystem > cainstance.migrate_profiles_to_ldap() > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/
cainstance.py",
> line 1981, in migrate_profiles_to_ldap > _create_dogtag_profile(profile_id, profile_data, overwrite=False) > File > "/usr/lib/python2.7/dist-packages/ipaserver/install/
cainstance.py",
> line 1987, in _create_dogtag_profile > with api.Backend.ra_certprofile as profile_api: > File > "/usr/lib/python2.7/dist-packages/ipaserver/plugins/
dogtag.py",
line > 1294, in __enter__ > raise errors.RemoteRetrieveError(reason=_('Failed
to
> authenticate to CA REST API')) > > 2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, > exception: RemoteRetrieveError: Failed to authenticate to CA REST API > 2017-11-15T13:05:59Z ERROR Unexpected error - see > /var/log/ipaupgrade.log for details: > RemoteRetrieveError: Failed to authenticate to CA REST
API
> > > > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> >
On 23/11/17 05:34, David Harvey via FreeIPA-users wrote:
Not sure why tomcat is more resilient when launched as root, but the pki seems to work ok at issuing certs after the above and a reboot for good measure.
This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things.
Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken?
hi Peter,
Not a full answer to your questions but from my experience:
Xenial: Worked, except OTP functionality Zesty: Worked except for DNS Artful: Seems fully functional and stable on the fresh installed replica, my upgraded from Zesty rig (with the workarounds noted earlier in thread) Still has pki-tomcat bombing fairly frequently. Bionic: I have high hopes for given LTS.. Currently showing same package versions https://packages.ubuntu.com/search?keywords=freeipa&searchon=names&suite=bionic§ion=all 4.4.4 as Artful
Most of them required some cajoling during install or upgrade due to broken installer components (like directories not being created in one case, /etc/pki/pki.version confusing postinstall in another), but most of these behaviours were captured as bugs too. It feels very close to being something that can be reliably deployed, so I don't think it needs a huge amount more TLC to make it more of a pleasure to install ;)
Cheers,
David
On 28 November 2017 at 20:58, Peter Fern via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
On 23/11/17 05:34, David Harvey via FreeIPA-users wrote:
Not sure why tomcat is more resilient when launched as root, but the pki seems to work ok at issuing certs after the above and a reboot for good measure.
This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things.
Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken? _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Without installing a system to check, it appears to me that nss-pem is still not packaged for Debian/Ubuntu, which means that certmonger will break on you when it comes time to auto-renew your CAs.
I found this out the hard way early this year while running FreeIPA with CA on Ubuntu, and recovery is very painful once your CA certs have expired (actually impossible without compiling nss-pem, which requires some source hacking and compiling of libnss to obtain static libs).
Since nss-pem is unlikely to be packaged on Debian/-derivs, it looks to me like until FreeIPA 4.5+ is packaged (where the conversion to OpenSSL has been completed), it is still not safe to run a CA on Ubuntu.
On 01/12/17 23:27, David Harvey via FreeIPA-users wrote:
hi Peter,
Not a full answer to your questions but from my experience:
Xenial: Worked, except OTP functionality Zesty: Worked except for DNS Artful: Seems fully functional and stable on the fresh installed replica, my upgraded from Zesty rig (with the workarounds noted earlier in thread) Still has pki-tomcat bombing fairly frequently. Bionic: I have high hopes for given LTS.. Currently showing same package versions https://packages.ubuntu.com/search?keywords=freeipa&searchon=names&suite=bionic§ion=all 4.4.4 as Artful
Most of them required some cajoling during install or upgrade due to broken installer components (like directories not being created in one case, /etc/pki/pki.version confusing postinstall in another), but most of these behaviours were captured as bugs too. It feels very close to being something that can be reliably deployed, so I don't think it needs a huge amount more TLC to make it more of a pleasure to install ;)
Cheers,
David
On 28 November 2017 at 20:58, Peter Fern via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
On 23/11/17 05:34, David Harvey via FreeIPA-users wrote: > Not sure why tomcat is more resilient when launched as root, but the > pki seems to work ok at issuing certs after the above and a reboot for > good measure. This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things. Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken? _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org>
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Well that sounds fun :) I'm hesistent to crosspost to pkg-freeipa-devel@lists.alioth.debian.org to ask after likelihood of seeing 4.5 in 18.04/Bionic but hope someone here might be able to comment?
WRT the exploding CA situation. I guess I'll need to get to a more sane build, or switch over to a better supported rpm based distro if that's not on the cards.. I should be safe in the short term given the standard lifetime of an IPA cert I hope!?
I'll continue to try and dig into why pki-tomcat dies on one but not all VMs (ca enabled on 2 of them)
On 1 December 2017 at 13:53, Peter Fern via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Without installing a system to check, it appears to me that nss-pem is still not packaged for Debian/Ubuntu, which means that certmonger will break on you when it comes time to auto-renew your CAs.
I found this out the hard way early this year while running FreeIPA with CA on Ubuntu, and recovery is very painful once your CA certs have expired (actually impossible without compiling nss-pem, which requires some source hacking and compiling of libnss to obtain static libs).
Since nss-pem is unlikely to be packaged on Debian/-derivs, it looks to me like until FreeIPA 4.5+ is packaged (where the conversion to OpenSSL has been completed), it is still not safe to run a CA on Ubuntu.
On 01/12/17 23:27, David Harvey via FreeIPA-users wrote:
hi Peter,
Not a full answer to your questions but from my experience:
Xenial: Worked, except OTP functionality Zesty: Worked except for DNS Artful: Seems fully functional and stable on the fresh installed replica, my upgraded from Zesty rig (with the workarounds noted earlier in thread) Still has pki-tomcat bombing fairly frequently. Bionic: I have high hopes for given LTS.. Currently showing same package versions https://packages.ubuntu.com/search?keywords=freeipa&searchon=names&suite=bionic§ion=all 4.4.4 as Artful
Most of them required some cajoling during install or upgrade due to broken installer components (like directories not being created in one case, /etc/pki/pki.version confusing postinstall in another), but most of these behaviours were captured as bugs too. It feels very close to being something that can be reliably deployed, so I don't think it needs a huge amount more TLC to make it more of a pleasure to install ;)
Cheers,
David
On 28 November 2017 at 20:58, Peter Fern via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
On 23/11/17 05:34, David Harvey via FreeIPA-users wrote:
Not sure why tomcat is more resilient when launched as root, but the pki seems to work ok at issuing certs after the above and a reboot for good measure.
This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things.
Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken? _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedo rahosted.org
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
David Harvey via FreeIPA-users wrote:
Well that sounds fun :) I'm hesistent to crosspost to pkg-freeipa-devel@lists.alioth.debian.org mailto:pkg-freeipa-devel@lists.alioth.debian.org to ask after likelihood of seeing 4.5 in 18.04/Bionic but hope someone here might be able to comment?
WRT the exploding CA situation. I guess I'll need to get to a more sane build, or switch over to a better supported rpm based distro if that's not on the cards.. I should be safe in the short term given the standard lifetime of an IPA cert I hope!?
I'll continue to try and dig into why pki-tomcat dies on one but not all VMs (ca enabled on 2 of them)
The risk you have isn't with the CA itself expiring but with the support certificates (OCSP, audit, subsystem, etc). Those have a 2-year validity period.
rob
On 1 December 2017 at 13:53, Peter Fern via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
Without installing a system to check, it appears to me that nss-pem is still not packaged for Debian/Ubuntu, which means that certmonger will break on you when it comes time to auto-renew your CAs. I found this out the hard way early this year while running FreeIPA with CA on Ubuntu, and recovery is very painful once your CA certs have expired (actually impossible without compiling nss-pem, which requires some source hacking and compiling of libnss to obtain static libs). Since nss-pem is unlikely to be packaged on Debian/-derivs, it looks to me like until FreeIPA 4.5+ is packaged (where the conversion to OpenSSL has been completed), it is still not safe to run a CA on Ubuntu. On 01/12/17 23:27, David Harvey via FreeIPA-users wrote:
hi Peter, Not a full answer to your questions but from my experience: Xenial: Worked, except OTP functionality Zesty: Worked except for DNS Artful: Seems fully functional and stable on the fresh installed replica, my upgraded from Zesty rig (with the workarounds noted earlier in thread) Still has pki-tomcat bombing fairly frequently. Bionic: I have high hopes for given LTS.. Currently showing same package versions <https://packages.ubuntu.com/search?keywords=freeipa&searchon=names&suite=bionic§ion=all> 4.4.4 as Artful Most of them required some cajoling during install or upgrade due to broken installer components (like directories not being created in one case, /etc/pki/pki.version confusing postinstall in another), but most of these behaviours were captured as bugs too. It feels very close to being something that can be reliably deployed, so I don't think it needs a huge amount more TLC to make it more of a pleasure to install ;) Cheers, David On 28 November 2017 at 20:58, Peter Fern via FreeIPA-users <freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org>> wrote: On 23/11/17 05:34, David Harvey via FreeIPA-users wrote: > Not sure why tomcat is more resilient when launched as root, but the > pki seems to work ok at issuing certs after the above and a reboot for > good measure. This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things. Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken? _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeopendnssecipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org>
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org>
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Ok, thanks for the clarification. Hopefully can still mitigate by changing platform or waiting for a better supported Ubuntu release!
On 1 Dec 2017 18:40, "Rob Crittenden" rcritten@redhat.com wrote:
David Harvey via FreeIPA-users wrote:
Well that sounds fun :) I'm hesistent to crosspost to pkg-freeipa-devel@lists.alioth.debian.org mailto:pkg-freeipa-devel@lists.alioth.debian.org to ask after likelihood of seeing 4.5 in 18.04/Bionic but hope someone here might be able to comment?
WRT the exploding CA situation. I guess I'll need to get to a more sane build, or switch over to a better supported rpm based distro if that's not on the cards.. I should be safe in the short term given the standard lifetime of an IPA cert I hope!?
I'll continue to try and dig into why pki-tomcat dies on one but not all VMs (ca enabled on 2 of them)
The risk you have isn't with the CA itself expiring but with the support certificates (OCSP, audit, subsystem, etc). Those have a 2-year validity period.
rob
On 1 December 2017 at 13:53, Peter Fern via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
Without installing a system to check, it appears to me that nss-pem is still not packaged for Debian/Ubuntu, which means that certmonger will break on you when it comes time to auto-renew your CAs. I found this out the hard way early this year while running FreeIPA with CA on Ubuntu, and recovery is very painful once your CA certs have expired (actually impossible without compiling nss-pem, which requires some source hacking and compiling of libnss to obtain static libs). Since nss-pem is unlikely to be packaged on Debian/-derivs, it looks to me like until FreeIPA 4.5+ is packaged (where the conversion to OpenSSL has been completed), it is still not safe to run a CA on
Ubuntu.
On 01/12/17 23:27, David Harvey via FreeIPA-users wrote:
hi Peter, Not a full answer to your questions but from my experience: Xenial: Worked, except OTP functionality Zesty: Worked except for DNS Artful: Seems fully functional and stable on the fresh installed replica, my upgraded from Zesty rig (with the workarounds noted earlier in thread) Still has pki-tomcat bombing fairly frequently. Bionic: I have high hopes for given LTS.. Currently showing same package versions <https://packages.ubuntu.com/search?keywords=freeipa&
searchon=names&suite=bionic§ion=all>
4.4.4 as Artful Most of them required some cajoling during install or upgrade due to broken installer components (like directories not being created in one case, /etc/pki/pki.version confusing postinstall in another), but most of these behaviours were captured as bugs too. It feels very close to being something that can be reliably deployed, so I don't think it needs a huge amount more TLC to make it more of a pleasure to install ;) Cheers, David On 28 November 2017 at 20:58, Peter Fern via FreeIPA-users <freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org>> wrote: On 23/11/17 05:34, David Harvey via FreeIPA-users wrote: > Not sure why tomcat is more resilient when launched as root, but the > pki seems to work ok at issuing certs after the above and a reboot for > good measure. This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS
libs
missing PEM support, which will stop your CA from renewing, amongst other things. Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken? _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeopendnssecipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.
fedorahosted.org
<mailto:freeipa-users-leave@lists.fedorahosted.org>
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org>
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.
fedorahosted.org
On 28.11.2017 22:58, Peter Fern via FreeIPA-users wrote:
On 23/11/17 05:34, David Harvey via FreeIPA-users wrote:
Not sure why tomcat is more resilient when launched as root, but the pki seems to work ok at issuing certs after the above and a reboot for good measure.
This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things.
I'd like to get a bug filed for each issue you find. For instance that upgrade thing should already be fixed but sounds like it isn't?
And yes, not being able to package nss-pem does mean the CA is less than useful. Maybe I should try to gently force the libnss maintainer to ship the needed (static) libs to be able to finish packaging nss-pem..
Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken?
Freeipa is/was stuck at 4.4 because getting bind9 9.11 in the archive took a year. That's now fixed, and I'm working on 4.6.x. But I need to update the whole stack, so right now I'm stuck with Dogtag 10.5.3 not building because it needed a newer (and patched) ldapjdk. Uploaded it today but it won't build before the (Debian) archive is otherwise untangled.
Anyway, for Ubuntu 18.04 I might be forced to drop support for the CA altogether, as it looks like Dogtag won't get fixed to support Tomcat 8.5 and RESTEasy 3.1 (and maybe others I haven't found out about yet) in time. Oh and I need to package the JBOSS version of jaxrs-api too, since the current alternative broke things when it got updated.. fun times ahead, as always.
t
On 13 December 2017 at 23:29, Timo Aaltonen via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
On 28.11.2017 22:58, Peter Fern via FreeIPA-users wrote:
On 23/11/17 05:34, David Harvey via FreeIPA-users wrote:
Not sure why tomcat is more resilient when launched as root, but the pki seems to work ok at issuing certs after the above and a reboot for good measure.
This sounds like there are broken permissions in the current Ubuntu packages. You should be aware that last time I checked, FreeIPA on Ubuntu was subtly yet severely broken, mostly due to the NSS libs missing PEM support, which will stop your CA from renewing, amongst other things.
I'd like to get a bug filed for each issue you find. For instance that upgrade thing should already be fixed but sounds like it isn't?
It's absolutely possible that the state of my upgrade didn't take in or countered your fixes due to my hacking around issues that reared their heads during the initial 17.04 install i upgraded from. Now that I'm upgraded it's a little harder to find out, but will see if I have any backups hanging around from the before upgrade state.
And yes, not being able to package nss-pem does mean the CA is less than useful. Maybe I should try to gently force the libnss maintainer to ship the needed (static) libs to be able to finish packaging nss-pem..
Does anyone know what the state of packaging for deb distros is currently? Now that the OpenSSL migration is complete(?), the barriers to functional packages should be removed, but it looks like that only happened in 4.5, and it appears only 4.4 is packaged, which is likely still broken?
Freeipa is/was stuck at 4.4 because getting bind9 9.11 in the archive took a year. That's now fixed, and I'm working on 4.6.x. But I need to update the whole stack, so right now I'm stuck with Dogtag 10.5.3 not building because it needed a newer (and patched) ldapjdk. Uploaded it today but it won't build before the (Debian) archive is otherwise untangled.
Anyway, for Ubuntu 18.04 I might be forced to drop support for the CA altogether, as it looks like Dogtag won't get fixed to support Tomcat 8.5 and RESTEasy 3.1 (and maybe others I haven't found out about yet) in time. Oh and I need to package the JBOSS version of jaxrs-api too, since the current alternative broke things when it got updated.. fun times ahead, as always.
Oh crikey, that sounds like as much fun as pulling teeth.
I can hold out a bit longer on the (as far as I can tell), very functional 17.10 install. Will make a call on it nearer the 18.04 time, but might make the jump to Fedora or the Docker based installs if things aren't looking good for the state of Ubuntu by then..
Thanks for the taking the time to explain the state of affairs. Appreciate your work as ever.
David
t
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
We successfully ran on Centos 7.3 with 4.4.4 and 4.5, the 4.5 having been installed later. The first step in installing the replica was that it automatically upgraded itself to the newest release, so it happened without giving us any choice. We later upgraded everything to 4.5.
4.5 have generally been OK, though the gssproxy that came with it seems to have a serious memory leak. We have to watch it and restart it when it gets too big.
On Nov 21, 2017, at 6:15 AM, David Harvey via FreeIPA-users <freeipa-users@lists.fedorahosted.orgmailto:freeipa-users@lists.fedorahosted.org> wrote:
Anyone out there with experience of whether or not adding a replica of more recent version (4.4.4 and 389 dir 1.3.7.5-1 up from 4.4.3 with 389 dir 1.3.5.15-2) would impact the existing servers in terms of schema or similar? I'm still trying to find a safe way to upgrade safely without going past a point of no return...
There may be a million and one reasons not to do it this way, but have you considered building a new VM on 17.10 and replicating from the existing server? I have just tried to upgrade a development environment (IPA client) to 17.10 and had endless issues. I ended up creating a new machine and copying across my files which was considerably quicker.
The upgrade to 17.10, particularly for machines that started out life on 16.04, appears to be fraut with problems even without having to deal with FreeIPA updates!
On Wed, 15 Nov 2017, 13:24 David Harvey via FreeIPA-users, < freeipa-users@lists.fedorahosted.org> wrote:
Hi wisdom of the list,
I know I am an edge case with running on ubuntu, but hoped someone might be able to shed some light.
A bit of background. I'm trying to test upgrades without potentially hosing my existing services, so I have cloned the VM, given it a new IP address, updated hosts file and pointed DNS somewhere that doesn't know about the real IPA services (8.8.8.8) so it won't try and sync or replicate.
Attempting to upgrade hits a snags or two, some described in bugs already like the pki version number confusing the apt scripts https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1703051 ). The one I can't work around however is below.
It seems deeply unhappy, and restarting the services result in the dogtag-pki web page being available until a login attempt is made (as occurs during the ipa-server-upgrade) after which point it bombs with a 500 error.
Could the below caused by https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1716842 ?
Any advice appreciated, as I think even when 18.04 hits with the proposed updates to rely on to tomcat 8.5, I'll still need to upgrade via 17.10 which seems currently fraught! If it relates to my method of cloning the VM, is there a better way of testing upgrades without potentially hosing the existing live systems?
Thanks in advance,
David
2017-11-15T13:05:59Z DEBUG approved_usage = SSL Server intended_usage = SSL Server 2017-11-15T13:05:59Z DEBUG cert valid True for "CN=ipa1.my.net,O= THOMAC.NET" 2017-11-15T13:05:59Z DEBUG handshake complete, peer = IPADDRESS 2017-11-15T13:05:59Z DEBUG Protocol: TLS1.2 2017-11-15T13:05:59Z DEBUG Cipher: TLS_RSA_WITH_AES_128_CBC_SHA 2017-11-15T13:05:59Z DEBUG response status 500 2017-11-15T13:05:59Z DEBUG response headers {'content-length': '2292', 'content-language': 'en', 'server': 'Apache-Coyote/1.1', 'connection': 'close', 'date': 'Wed, 15 Nov 2017 13:05:59 GMT', 'content-type': 'text/html;charset=utf-8'} 2017-11-15T13:05:59Z DEBUG response body '<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.46 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Subsystem unavailable</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Subsystem unavailable</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.ws.rs.ServiceUnavailableException: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:138)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:283)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.46 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.46 (Ubuntu)</h3></body></html>' 2017-11-15T13:05:59Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2017-11-15T13:05:59Z DEBUG File "/usr/lib/python2.7/dist-packages/ipapython/admintool.py", line 172, in execute return_value = self.run() File "/usr/lib/python2.7/dist-packages/ipaserver/install/ipa_server_upgrade.py", line 46, in run server.upgrade() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1878, in upgrade upgrade_configuration() File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 1797, in upgrade_configuration ca_enable_ldap_profile_subsystem(ca) File "/usr/lib/python2.7/dist-packages/ipaserver/install/server/upgrade.py", line 347, in ca_enable_ldap_profile_subsystem cainstance.migrate_profiles_to_ldap() File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1981, in migrate_profiles_to_ldap _create_dogtag_profile(profile_id, profile_data, overwrite=False) File "/usr/lib/python2.7/dist-packages/ipaserver/install/cainstance.py", line 1987, in _create_dogtag_profile with api.Backend.ra_certprofile as profile_api: File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/dogtag.py", line 1294, in __enter__ raise errors.RemoteRetrieveError(reason=_('Failed to authenticate to CA REST API'))
2017-11-15T13:05:59Z DEBUG The ipa-server-upgrade command failed, exception: RemoteRetrieveError: Failed to authenticate to CA REST API 2017-11-15T13:05:59Z ERROR Unexpected error - see /var/log/ipaupgrade.log for details: RemoteRetrieveError: Failed to authenticate to CA REST API
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
freeipa-users@lists.fedorahosted.org