Jenkins in the Fedora infrastructure

[David Nalley]

On Sun, Oct 7, 2012 at 6:58 PM, Garrett Holmstrom
<gholms at fedoraproject.org> wrote:
> On 2012-10-05 10:00, Pierre-Yves Chibon wrote:
>>
>> This week Seth, Toshio and I have been thinking about and playing with
>> Jenkins.
>>
>> The current jenkins we used is administrted by Luke at:
>> http://jenkins.turbogears.org/
>> and runs on hardware which is not within the Fedora infrastructure.
>> This machine is:
>>    Processor: Dual Xeon @ 2.50GHz (on a dual quad-core Xen dom0)
>>    Memory: 1G allocated; 12G on dom0
>>    OS: Red Hat Enterprise Linux Server 5.8
>>    Python: python-2.4, 2.5, 2.6 and 2.7
>>
>> This week had two co-occurring events:
>> - fedora-review did not build on this instance of jenkins due to missing
>> dependencies on the system
>> - Toshio started to port Kitchen to python3 and had no place to run his
>> unit-tests in an automated way.
>>
>> So we thought about using our new cloud system for setting up jenkins
>> build nodes.
>>
>> We now have two build nodes within our cloud, one running Fedora 17 and
>> one running EL6 (down right now as it is being rebuilt).
>> [http://jenkins.turbogears.org/computer/]
>>
>> Where do we stand from this:
>> - We can create nodes on our cloud
>> - Seth created an Ansible routine to configure the nodes directly after
>> their creation [http://fpaste.org/jRX1/raw/]
>>
>> So adding new nodes to a Jenkins instance becomes really easy and rather
>> fast.
>>
>>
>> If we want to run our own jenkins master:
>>
>> This is the system I can think of:
>> * Configure the Jenkins master in a machine within the Fedora
>> infrastructure
>> * This master is not allowed to do build
>> * The master can send emails (current jenkins can not due to mail server
>> restrictions)
>> * All the builds ran in nodes in the cloud
>> * Nodes are reinstalled every 6 month, when there is a new version of
>> Fedora or when needed (via Ansible)
>> * Nodes can be thrown away at any time
>
>
> Jenkins has an EC2 plugin that works with Eucalyptus (or at least version
> 1.14 does; I haven't tested anything newer at $dayjob).  You add entries to
> the master's config that point it toward the right images in the cloud,
> label those config entries, and add matching labels to your Jenkins jobs.
> Then when it goes to run a job it automatically spins up new instances with
> the right labels if there are no open slots.  It will also kill off
> instances that have been idle for a while.
>
> Combine that with the Copy to Slave plugin and you've got yourself a nice
> little build pipeline.

There's also a jclouds plugin that works with well nigh every cloud
platform known to man, and plenty of things that aren't really clouds.
The jclouds folks are almost maniacal in their efforts to keep things
working with the latest and greatest versions.

--David