Re: [candlepin] [foreman-users] gutterball and candlepin not working after upgrade

[Cliff Perry]

On 16/06/15 09:48, Jorick Astrego wrote:
> Hi,
>
> A week ago I upgraded from 2.1 to 2.2.1 and although everything seemed
> ok, I didn't have time to check everything. Yesterday, I noticed the
> sync task was not running and I am unable to run any task now.
>
> Checking various logs, I can see gutterball and candelpin are giving
> connection erros to QPID (longer log excerpt at the bottom):
> 2015-06-15 13:45:12,289 [gutterball-qpid-connect] ERROR
> org.candlepin.gutterball.receiver.EventReceiver - Unable to
> initialize connection to QPID.
> java.lang.RuntimeException: javax.jms.JMSException:*Error creating
> connection: Keystore was tampered with, or password was incorrect*
>
> 2015-06-16 10:02:30,946 [=, org=] ERROR
> org.candlepin.guice.AMQPBusPubProvider - Unable to instantiate
> AMQPBusProvider:
> javax.jms.JMSException: Error creating connection: Connection refused
>
> Then I checked the installer log and I can see errors with the certificates:
> [DEBUG 2015-06-10 16:37:05 main]  Exec[[import client certificate
> into gutterball keystore] signal import if pair has not been
> imported](provider=posix): Executing check 'keytool -list -keystore
> /etc/gutterball/certs/amqp//gutterball.jks -storepass
> eS8hZrxxf6tcZfHEHkN2hYY2BXN9NnXc -alias gutterball'
> [DEBUG 2015-06-10 16:37:05 main]  Executing 'keytool -list -keystore
> /etc/gutterball/certs/amqp//gutterball.jks -storepass
> eS8hZrxxf6tcZfHEHkN2hYY2BXN9NnXc -alias gutterball'
> [DEBUG 2015-06-10 16:37:05 main]
> /Stage[main]/Certs::Gutterball/Certs::Ssltools::Keytool::Import_keypair[import
> client certificate into gutterball keystore]/Exec[[import client
> certificate into gutterball keystore] signal import if pair has not
> been imported]/unless: keytool error: java.io.IOException: Keystore
> was tampered with, or password was incorrect
> [DEBUG 2015-06-10 16:37:05 main]  Exec[[import client certificate
> into gutterball keystore] signal import if pair has not been
> imported](provider=posix): Executing 'echo importing keypair'
> [DEBUG 2015-06-10 16:37:05 main]  Executing 'echo importing keypair'
>
>
> [DEBUG 2015-06-10 16:37:07 main]  Exec[import client certificate
> into Candlepin keystore](provider=posix): Executing check 'keytool
> -list -keystore /etc/candlepin/certs/amqp/candlepin.jks -storepass
> gttJyejWhbMVgcBkVwqsX6g9nSAb76WN -alias katello-default-ca'
> [DEBUG 2015-06-10 16:37:07 main]  Executing 'keytool -list -keystore
> /etc/candlepin/certs/amqp/candlepin.jks -storepass
> gttJyejWhbMVgcBkVwqsX6g9nSAb76WN -alias katello-default-ca'
> [DEBUG 2015-06-10 16:37:07 main]
> /Stage[main]/Certs::Candlepin/Exec[import client certificate into
> Candlepin keystore]/unless: keytool error: java.lang.Exception:
> Alias <katello-default-ca> does not exist
> [DEBUG 2015-06-10 16:37:07 main]  Exec[import client certificate
> into Candlepin keystore](provider=posix): Executing 'openssl pkcs12
> -export -name amqp-client -in /etc/pki/katello/certs/java-client.crt
> -inkey /etc/pki/katello/private/java-client.key -out
> /tmp/keystore.p12 -passout
> file:/etc/pki/katello/keystore_password-file && keytool
> -importkeystore -destkeystore
> /etc/candlepin/certs/amqp/candlepin.jks -srckeystore
> /tmp/keystore.p12 -srcstoretype pkcs12 -alias amqp-client -storepass
> gttJyejWhbMVgcBkVwqsX6g9nSAb76WN -srcstorepass
> gttJyejWhbMVgcBkVwqsX6g9nSAb76WN -noprompt && rm /tmp/keystore.p12'
> [DEBUG 2015-06-10 16:37:07 main]  Executing 'openssl pkcs12 -export
> -name amqp-client -in /etc/pki/katello/certs/java-client.crt -inkey
> /etc/pki/katello/private/java-client.key -out /tmp/keystore.p12
> -passout file:/etc/pki/katello/keystore_password-file && keytool
> -importkeystore -destkeystore
> /etc/candlepin/certs/amqp/candlepin.jks -srckeystore
> /tmp/keystore.p12 -srcstoretype pkcs12 -alias amqp-client -storepass
> gttJyejWhbMVgcBkVwqsX6g9nSAb76WN -srcstorepass
> gttJyejWhbMVgcBkVwqsX6g9nSAb76WN -noprompt && rm /tmp/keystore.p12'
> [ WARN 2015-06-10 16:37:08 main]
> /Stage[main]/Certs::Candlepin/Exec[import client certificate into
> Candlepin keystore]/returns: executed successfully
>
>
> When I try it manually, I also get an error:
> keytool -list -keystore /etc/candlepin/certs/amqp/candlepin.jks
> -storepass ********* -alias katello-default-ca
>
> keytool error: java.lang.Exception: Alias <katello-default-ca> does
> not exist
>
>
> Anything I can try to fix this? Any idea on how this could have happened?
>
> From the logs:
> /var/log/gutterball/gutterball.log
>
> 2015-06-15 13:42:42,223 [main] WARN
> org.hibernate.service.jdbc.connections.internal.ConnectionProviderInitiator
> - HHH000208: org.hibernate.connection.C3P0ConnectionProvider has
> been deprecated in favor of
> org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider;
> that provider will be used instead.
> 2015-06-15 13:42:42,304 [main] WARN
> com.mchange.v2.c3p0.management.ActiveManagementCoordinator - A
> C3P0Registry mbean is already registered. This probably means that
> an application using c3p0 was undeployed, but not all
> PooledDataSources were closed prior to undeployment. This may lead
> to resource leaks over time. Please take care to close all
> PooledDataSources.
> 2015-06-15 13:42:43,022 [main] WARN
> org.hibernate.id.UUIDHexGenerator - HHH000409: Using
> org.hibernate.id.UUIDHexGenerator which does not generate IETF RFC
> 4122 compliant UUID values; consider using
> org.hibernate.id.UUIDGenerator instead
> 2015-06-15 13:45:12,289 [gutterball-qpid-connect] ERROR
> org.candlepin.gutterball.receiver.EventReceiver - Unable to
> initialize connection to QPID.
> java.lang.RuntimeException: javax.jms.JMSException:*Error creating
> connection: Keystore was tampered with, or password was incorrect*
>      at
> org.candlepin.gutterball.receiver.EventReceiver$QpidConnectionThread.run(EventReceiver.java:179)
> ~[EventReceiver$QpidConnectionThread.class:na]
> javax.jms.JMSException: Error creating connection: Keystore was
> tampered with, or password was incorrect
>      at
> org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:128)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.candlepin.gutterball.receiver.EventReceiver.init(EventReceiver.java:82)
> ~[EventReceiver.class:na]
>      at
> org.candlepin.gutterball.receiver.EventReceiver$QpidConnectionThread.run(EventReceiver.java:176)
> ~[EventReceiver$QpidConnectionThread.class:na]
> Caused by: org.apache.qpid.AMQConnectionFailureException: Keystore
> was tampered with, or password was incorrect
>      at
> org.apache.qpid.client.AMQConnection.<init>(AMQConnection.java:503)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:124)
> ~[qpid-client-0.30.jar:0.30]
>      ... 2 common frames omitted
> Caused by: org.apache.qpid.transport.TransportException: Error
> creating SSL Context
>      at
> org.apache.qpid.transport.network.security.SecurityLayerFactory$SSLSecurityLayer.<init>(SecurityLayerFactory.java:91)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.transport.network.security.SecurityLayerFactory.newInstance(SecurityLayerFactory.java:51)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.transport.Connection.connect(Connection.java:232)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnectionDelegate_0_10.makeBrokerConnection(AMQConnectionDelegate_0_10.java:221)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnection.makeBrokerConnection(AMQConnection.java:647)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnection.<init>(AMQConnection.java:429)
> ~[qpid-client-0.30.jar:0.30]
>      ... 3 common frames omitted
> Caused by: java.io.IOException: Keystore was tampered with, or
> password was incorrect
>      at
> sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
> ~[na:1.7.0_79]
>      at
> sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
> ~[na:1.7.0_79]
>      at java.security.KeyStore.load(KeyStore.java:1214) ~[na:1.7.0_79]
>      at
> org.apache.qpid.transport.network.security.ssl.SSLUtil.getInitializedKeyStore(SSLUtil.java:172)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.ssl.SSLContextFactory.buildContext(SSLContextFactory.java:92)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.ssl.SSLContextFactory.buildClientContext(SSLContextFactory.java:62)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.transport.network.security.SecurityLayerFactory$SSLSecurityLayer.<init>(SecurityLayerFactory.java:78)
> ~[qpid-common-0.30.jar:0.30]
>      ... 8 common frames omitted
> Caused by: java.security.UnrecoverableKeyException: Password
> verification failed
>      at
> sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770)
> ~[na:1.7.0_79]
>      ... 14 common frames omitted
> 2015-06-16 09:59:49,416 [main] INFO
> org.candlepin.gutterball.servlet.GutterballContextListener -
> Destroying gutterball context
> 2015-06-16 09:59:49,416 [main] INFO
> org.candlepin.gutterball.receiver.EventReceiver - Closing QPID
> connection
>
>
> /var/log/candlepin/error.log
>
> 2015-06-16 10:02:30,946 [=, org=] ERROR
> org.candlepin.guice.AMQPBusPubProvider - Unable to instantiate
> AMQPBusProvider:
> javax.jms.JMSException: Error creating connection: Connection refused
>      at
> org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:128)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.candlepin.guice.AMQPBusPubProvider.<init>(AMQPBusPubProvider.java:81)
> ~[AMQPBusPubProvider.class:na]
>      at
> org.candlepin.guice.AMQPBusPubProvider$$FastClassByGuice$$ef75a35a.newInstance(<generated>)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:53)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
> [guice-3.0.jar:na]
>      at com.google.inject.Scopes$1$1.get(Scopes.java:65)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:204)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:198)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:198)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:179)
> [guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
> [guice-3.0.jar:na]
>      at com.google.inject.Guice.createInjector(Guice.java:95)
> [guice-3.0.jar:na]
>      at
> org.candlepin.guice.CandlepinContextListener.getInjector(CandlepinContextListener.java:146)
> [CandlepinContextListener.class:na]
>      at
> org.candlepin.guice.CandlepinGuiceResteasyBootstrap.contextInitialized(CandlepinGuiceResteasyBootstrap.java:57)
> [CandlepinGuiceResteasyBootstrap.class:na]
>      at
> org.candlepin.guice.CandlepinContextListener.contextInitialized(CandlepinContextListener.java:114)
> [CandlepinContextListener.class:na]
>      at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:637)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:563)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:498)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> [catalina-6.0.24.jar:na]
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.7.0_79]
>      at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ~[na:1.7.0_79]
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.7.0_79]
>      at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
>      at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> [bootstrap-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> [bootstrap-6.0.24.jar:na]
> Caused by: org.apache.qpid.AMQConnectionFailureException: Connection
> refused
>      at
> org.apache.qpid.client.AMQConnection.<init>(AMQConnection.java:503)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:124)
> ~[qpid-client-0.30.jar:0.30]
>      ... 46 common frames omitted
> Caused by: org.apache.qpid.transport.TransportException: Error
> connecting to broker
>      at
> org.apache.qpid.transport.network.io.IoNetworkTransport.connect(IoNetworkTransport.java:89)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.transport.Connection.connect(Connection.java:243)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnectionDelegate_0_10.makeBrokerConnection(AMQConnectionDelegate_0_10.java:221)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnection.makeBrokerConnection(AMQConnection.java:647)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnection.<init>(AMQConnection.java:429)
> ~[qpid-client-0.30.jar:0.30]
>      ... 47 common frames omitted
> Caused by: java.net.ConnectException: Connection refused
>      at java.net.PlainSocketImpl.socketConnect(Native Method)
> ~[na:1.7.0_79]
>      at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> ~[na:1.7.0_79]
>      at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> ~[na:1.7.0_79]
>      at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> ~[na:1.7.0_79]
>      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.7.0_79]
>      at java.net.Socket.connect(Socket.java:579) ~[na:1.7.0_79]
>      at
> org.apache.qpid.transport.network.io.IoNetworkTransport.connect(IoNetworkTransport.java:85)
> ~[qpid-common-0.30.jar:0.30]
>      ... 51 common frames omitted
> 2015-06-16 10:02:34,595 [=, org=] WARN
> org.hibernate.id.UUIDHexGenerator - HHH000409: Using
> org.hibernate.id.UUIDHexGenerator which does not generate IETF RFC
> 4122 compliant UUID values; consider using
> org.hibernate.id.UUIDGenerator instead
> 2015-06-16 10:02:36,405 [=, org=] ERROR
> org.candlepin.guice.CandlepinGuiceResteasyBootstrap - Could not
> create Guice injector.
> com.google.inject.CreationException: Guice creation errors:
>
> 1) Error injecting constructor, java.lang.RuntimeException:
> javax.jms.JMSException: Error creating connection: Connection refused
>    at
> org.candlepin.guice.AMQPBusPubProvider.<init>(AMQPBusPubProvider.java:67)
>    while locating org.candlepin.guice.AMQPBusPubProvider
>    at
> org.candlepin.guice.CandlepinModule.configureAmqp(CandlepinModule.java:351)
>    while locating org.candlepin.audit.AMQPBusPublisher
>
> 1 error
>      at
> com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:183)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
> ~[guice-3.0.jar:na]
>      at com.google.inject.Guice.createInjector(Guice.java:95)
> ~[guice-3.0.jar:na]
>      at
> org.candlepin.guice.CandlepinContextListener.getInjector(CandlepinContextListener.java:146)
> [CandlepinContextListener.class:na]
>      at
> org.candlepin.guice.CandlepinGuiceResteasyBootstrap.contextInitialized(CandlepinGuiceResteasyBootstrap.java:57)
> ~[CandlepinGuiceResteasyBootstrap.class:na]
>      at
> org.candlepin.guice.CandlepinContextListener.contextInitialized(CandlepinContextListener.java:114)
> [CandlepinContextListener.class:na]
>      at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:637)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:563)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:498)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> [catalina-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> [catalina-6.0.24.jar:na]
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.7.0_79]
>      at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ~[na:1.7.0_79]
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.7.0_79]
>      at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
>      at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> [bootstrap-6.0.24.jar:na]
>      at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> [bootstrap-6.0.24.jar:na]
> Caused by: java.lang.RuntimeException: javax.jms.JMSException: Error
> creating connection: Connection refused
>      at
> org.candlepin.guice.AMQPBusPubProvider.<init>(AMQPBusPubProvider.java:88)
> ~[AMQPBusPubProvider.class:na]
>      at
> org.candlepin.guice.AMQPBusPubProvider$$FastClassByGuice$$ef75a35a.newInstance(<generated>)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:53)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
> ~[guice-3.0.jar:na]
>      at com.google.inject.Scopes$1$1.get(Scopes.java:65)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:204)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:198)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:198)
> ~[guice-3.0.jar:na]
>      at
> com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:179)
> ~[guice-3.0.jar:na]
>      ... 29 common frames omitted
> Caused by: javax.jms.JMSException: Error creating connection:
> Connection refused
>      at
> org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:128)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.candlepin.guice.AMQPBusPubProvider.<init>(AMQPBusPubProvider.java:81)
> ~[AMQPBusPubProvider.class:na]
>      ... 45 common frames omitted
> Caused by: org.apache.qpid.AMQConnectionFailureException: Connection
> refused
>      at
> org.apache.qpid.client.AMQConnection.<init>(AMQConnection.java:503)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnectionFactory.createConnection(AMQConnectionFactory.java:124)
> ~[qpid-client-0.30.jar:0.30]
>      ... 46 common frames omitted
> Caused by: org.apache.qpid.transport.TransportException: Error
> connecting to broker
>      at
> org.apache.qpid.transport.network.io.IoNetworkTransport.connect(IoNetworkTransport.java:89)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.transport.Connection.connect(Connection.java:243)
> ~[qpid-common-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnectionDelegate_0_10.makeBrokerConnection(AMQConnectionDelegate_0_10.java:221)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnection.makeBrokerConnection(AMQConnection.java:647)
> ~[qpid-client-0.30.jar:0.30]
>      at
> org.apache.qpid.client.AMQConnection.<init>(AMQConnection.java:429)
> ~[qpid-client-0.30.jar:0.30]
>      ... 47 common frames omitted
> Caused by: java.net.ConnectException: Connection refused
>      at java.net.PlainSocketImpl.socketConnect(Native Method)
> ~[na:1.7.0_79]
>      at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> ~[na:1.7.0_79]
>      at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> ~[na:1.7.0_79]
>      at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> ~[na:1.7.0_79]
>      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.7.0_79]
>      at java.net.Socket.connect(Socket.java:579) ~[na:1.7.0_79]
>      at
> org.apache.qpid.transport.network.io.IoNetworkTransport.connect(IoNetworkTransport.java:85)
> ~[qpid-common-0.30.jar:0.30]
>      ... 51 common frames omitted
>
>
>
> **
> Met vriendelijke groet, With kind regards,
>
> Jorick Astrego*
> Netbulae Virtualization Experts *
>
> Tel: [phone] 	[e-mail] 	Staalsteden 4-3A 	KvK 08198180
> Fax: [phone] 	www.netbulae.eu 	7547 TA Enschede 	BTW NL821234584B01
>
> --
> You received this message because you are subscribed to the Google
> Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [e-mail]
> .
> To post to this group, send email to [e-mail]
> .
> Visit this group at http://groups.google.com/group/foreman-users.
> For more options, visit https://groups.google.com/d/optout.

Cross posting to the Candlepin community list for visibility there.

Regards,
Cliff