I'm having some fun with the Jenkins instance that's supposed to be running rpminspect and hope that someone here might have some suggestions.
The instance is hosted in the CentOS CI OpenShift cluster and the configuration is here:
https://github.com/tflink/fedora-ci-generic-checks/
At this point, every Jenkins master pod that attempts to start up gets an error like the following:
2020-03-23 21:21:31.509+0000 [id=28] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all extensions 2020-03-23 21:21:31.535+0000 [id=28] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading global config java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 16 at com.thoughtworks.xstream.core.util.FastStack.pop(FastStack.java:42) at com.thoughtworks.xstream.io.xml.AbstractPullReader.move(AbstractPullReader.java:125) at com.thoughtworks.xstream.io.xml.AbstractPullReader.moveDown(AbstractPullReader.java:103) at com.thoughtworks.xstream.io.xml.XppReader.<init>(XppReader.java:63) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:54) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65) at hudson.XmlFile.unmarshal(XmlFile.java:180) Caused: java.io.IOException: Unable to read /var/lib/jenkins/config.xml at hudson.XmlFile.unmarshal(XmlFile.java:183) at hudson.XmlFile.unmarshal(XmlFile.java:163) at jenkins.model.Jenkins.loadConfig(Jenkins.java:3081) at jenkins.model.Jenkins.access$1200(Jenkins.java:315) at jenkins.model.Jenkins$13.run(Jenkins.java:3182) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1121) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2020-03-23 21:21:31.536+0000 [id=21] SEVERE hudson.util.BootFailure#publish: Failed to initialize Jenkins java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 16 at com.thoughtworks.xstream.core.util.FastStack.pop(FastStack.java:42) at com.thoughtworks.xstream.io.xml.AbstractPullReader.move(AbstractPullReader.java:125) at com.thoughtworks.xstream.io.xml.AbstractPullReader.moveDown(AbstractPullReader.java:103) at com.thoughtworks.xstream.io.xml.XppReader.<init>(XppReader.java:63) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:54) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65) at hudson.XmlFile.unmarshal(XmlFile.java:180) Caused: java.io.IOException: Unable to read /var/lib/jenkins/config.xml at hudson.XmlFile.unmarshal(XmlFile.java:183) at hudson.XmlFile.unmarshal(XmlFile.java:163) at jenkins.model.Jenkins.loadConfig(Jenkins.java:3081) at jenkins.model.Jenkins.access$1200(Jenkins.java:315) at jenkins.model.Jenkins$13.run(Jenkins.java:3182) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1121) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused: org.jvnet.hudson.reactor.ReactorException at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282) at jenkins.InitReactorRunner.run(InitReactorRunner.java:48) at jenkins.model.Jenkins.executeReactor(Jenkins.java:1154) at jenkins.model.Jenkins.<init>(Jenkins.java:965) at hudson.model.Hudson.<init>(Hudson.java:85) at hudson.model.Hudson.<init>(Hudson.java:81) at hudson.WebAppMain$3.run(WebAppMain.java:233) Caused: hudson.util.HudsonFailedToLoad at hudson.WebAppMain$3.run(WebAppMain.java:250) 2020-03-23 21:21:31.550+0000 [id=21] INFO jenkins.model.Jenkins#cleanUp: Stopping Jenkins 2020-03-23 21:21:31.564+0000 [id=27] WARNING hudson.model.UpdateCenter#save: Failed to save /var/lib/jenkins/hudson.model.UpdateCenter.xml java.nio.channels.ClosedByInterruptException at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:199) at java.base/sun.nio.ch.FileChannelImpl.endBlocking(FileChannelImpl.java:162) at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:285) at hudson.util.FileChannelWriter.write(FileChannelWriter.java:73) at java.base/java.io.Writer.write(Writer.java:290) at hudson.util.AtomicFileWriter.write(AtomicFileWriter.java:162) at java.base/java.io.Writer.write(Writer.java:249) at hudson.XmlFile.write(XmlFile.java:191) at hudson.model.UpdateCenter.save(UpdateCenter.java:893) at hudson.util.PersistedList.onModified(PersistedList.java:173) at hudson.util.PersistedList.replaceBy(PersistedList.java:85) at hudson.model.UpdateCenter.load(UpdateCenter.java:907) at hudson.model.UpdateCenter.init(UpdateCenter.java:2465) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104) at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1121) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2020-03-23 21:21:31.714+0000 [id=21] INFO jenkins.model.Jenkins$18#onAttained: Started termination
Has anyone seen this before? Any suggestions on things I could try? At this point, the only idea I have to fix the thing is to delete the persistent storage and rebuild the thing. If I do that, all the result links in rpminspect results would be broken and the logs from those runs lost.
Thanks,
Tim