Azure test failures
by Rob Crittenden
Looks a lot of IPA PRs have been failing the last few days due to one or
more tests in Azure failing. I cherry-picked a few and they all look
like CA startup failures during server installation, replication
creation, ipa-ca-install, etc.
In the one test I did a deeper dive into the CA debug log the start up
looks quite benign with no errors reported and it finishes up at 19:33:27
The IPA installer kicked off the restart at 19:32:11 and gave up at
19:35:23.
The journal logs a suspect (why this isn't in the CA debug log I don't
know):
Apr 30 19:32:35 master1.ipa.test server[3589]: SEVERE: Failed to
initialize component [Connector[AJP/1.3-8009]]
Apr 30 19:32:35 master1.ipa.test server[3589]:
org.apache.catalina.LifecycleException: Protocol handler initialization
failed
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.connector.Connector.initInternal(Connector.java:1042)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.core.StandardService.initInternal(StandardService.java:533)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:1057)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.startup.Catalina.load(Catalina.java:724)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.startup.Catalina.load(Catalina.java:746)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
java.lang.reflect.Method.invoke(Method.java:498)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:302)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:472)
Apr 30 19:32:35 master1.ipa.test server[3589]: Caused by:
java.net.BindException: Address already in use
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.nio.ch.Net.bind0(Native Method)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.nio.ch.Net.bind(Net.java:461)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.nio.ch.Net.bind(Net.java:453)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.tomcat.util.net.NioEndpoint.initServerSocket(NioEndpoint.java:228)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:211)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1141)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1154)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:581)
Apr 30 19:32:35 master1.ipa.test server[3589]: at
org.apache.catalina.connector.Connector.initInternal(Connector.java:1039)
Apr 30 19:32:35 master1.ipa.test server[3589]: ... 13 more
and the journal is full of the expected HTTP connect issues trying to
verify that the CA is fully up.
The full logs are available at
https://dev.azure.com/freeipa/freeipa/_build/results?buildId=8758&view=ar...
Right click the three-dot thinger at the end of
logs-GATING-sudo_1_to_5_8758-1-1-3-Linux-X86 and select Download artifacts.
rob
2 years, 11 months
[freeipa PR#5750][opened] azure: Re-group test envs
by stanislavlevin
URL: https://github.com/freeipa/freeipa/pull/5750
Author: stanislavlevin
Title: #5750: azure: Re-group test envs
Action: opened
PR body:
"""
This is the experimental fix to workaround the issue with
PKI component, which is sensitive to slow systems(at least,
appropriate delays and timeouts should be adjusted for such).
Somehow Azure's test envs became slower then they were earlier
(for example, CA subsystem start changed
~(20-30)sec -> ~(45-60)sec). This triggered various issues with
subsystems of PKI.
To release CPU resources(Azure VM provides 2 cores, 1 thread per
core) the extreme usage of swap was reduced to the limits provided
by Azure(7Gb + 4Gb swap). Server/replica container consumes ~2Gb,
client one ~0.5Gb.
Also this increases the number of Azure's Gating jobs from 4 to 6.
Related: https://lists.fedorahosted.org/archives/list/freeipa-devel@lists.fedoraho...
"""
To pull the PR as Git branch:
git remote add ghfreeipa https://github.com/freeipa/freeipa
git fetch ghfreeipa pull/5750/head:pr5750
git checkout pr5750
2 years, 11 months
[freeipa PR#5740][opened] Constrain pylint to supported versions
by tiran
URL: https://github.com/freeipa/freeipa/pull/5740
Author: tiran
Title: #5740: Constrain pylint to supported versions
Action: opened
PR body:
"""
Two, three times a year PR-CI starts failing because tox tests pull in a
newer version of pylint with new warnings. To reduce breakage this
change constraints pylint (and indirectly astroid) to latest tested
minor version. The constraint should be updated when FreeIPA starts to
support a new Fedora version with more recent pylint.
Related: pagure.io/freeipa/issue/8818
Signed-off-by: Christian Heimes <cheimes(a)redhat.com>
"""
To pull the PR as Git branch:
git remote add ghfreeipa https://github.com/freeipa/freeipa
git fetch ghfreeipa pull/5740/head:pr5740
git checkout pr5740
2 years, 11 months