Changing Cobbler's Build Process

[Devan Goodwin]

Wanted to pitch an idea, we wrote build tool for Spacewalk called Tito
(http://rm-rf.ca/tito) for projects using Git and Rpm, and I was
wondering if there would be any interest in using it to build Cobbler
packages.

In effect it would allow things like:

- Build test packages off the latest git head. (this is just a little
cleaner for developers to remove off your system and replace with a
stable version than doing a raw install)
- Auto-bump the version and tag a new release with auto-generated
changelog. (helpful if we wanted to do more frequent package rebuilds)
- Build a .tar.gz, srpm, or rpms off any past tag.

The impact would be a yum install tito for anyone who wanted to build
themselves rpms off source, and a new top level rel-eng directory in
git that tracks some metadata. There should be no impact for users
running from source, make webtest can remain. (make rpms would
probably die)

Not positive how much effort it would be to do the changeover but
looking at the build process and Makefile today, I don't think it's
much. Let ne know if you think it would be useful. The tito readme is
here: http://github.com/dgoodwin/tito

Cheers,

Devan

[brett lentz]

On Mon, Jan 11, 2010 at 11:21 AM, Devan Goodwin  wrote:
> Wanted to pitch an idea, we wrote build tool for Spacewalk called Tito
> (http://rm-rf.ca/tito) for projects using Git and Rpm, and I was
> wondering if there would be any interest in using it to build Cobbler
> packages.
>
> Cheers,
>
> Devan

This sounds an awful lot like reinventing the wheel to me.

Why Tito and not Koji/Mock and the existing Red Hat/Fedora build
tools?  Does Tito support Koji and/or vice versa?

---Brett.

[Devan Goodwin]

> This sounds an awful lot like reinventing the wheel to me.
>
> Why Tito and not Koji/Mock and the existing Red Hat/Fedora build
> tools?  Does Tito support Koji and/or vice versa?

The tool is intended to solve different problems than these, tito was
more about answering questions like:

- How do I tag a new release (in cobbler.git, not Fedora CVS), where
do I need to bump version numbers, what's going into the changelog,
etc.
- As a developer, how can I install Cobbler on my system using the
latest source. (something I prefer to do with RPMs as I feel it's more
controlled to later remove)
- How do I actually generate a tarball or source rpm for a particular
tagged release?
- How do I generate that tarball for this release with a consistent
checksum in the future?

So this is more relevant to an upstream project's release process,
independent of the specific tools around Fedora. Even mock only comes
into play once you've got a source rpm, tito is involved in getting
you to that point. (although now that you mention it an option to do
the build in mock would be quite useful)

We did however write some code in Tito for integrating with CVS and
Koji because we needed it to handle the volume of package builds
involved with Spacewalk. This command checks out the module in CVS,
imports the latest spec file and patches from git, builds the tarball,
uploads it, and submits builds in Koji. This isn't documented yet as
I've only run it against Red Hat's internal build system, but it
should work fine in Fedora CVS and Koji, just need to test it.

Cheers,

Devan

[Bill Peck]

On 01/11/2010 03:45 PM, Devan Goodwin wrote:
> > This sounds an awful lot like reinventing the wheel to me.
> >
> > Why Tito and not Koji/Mock and the existing Red Hat/Fedora build
> > tools?  Does Tito support Koji and/or vice versa?
>
> The tool is intended to solve different problems than these, tito was
> more about answering questions like:
> How do I tag a new release (in cobbler.git, not Fedora CVS), where
>
> do I need to bump version numbers, what's going into the changelog,
> etc.
> As a developer, how can I install Cobbler on my system using the
>
> latest source. (something I prefer to do with RPMs as I feel it's more
> controlled to later remove)
> How do I actually generate a tarball or source rpm for a particular
>
> tagged release?
> How do I generate that tarball for this release with a consistent
>
> checksum in the future?
>
> So this is more relevant to an upstream project's release process,
> independent of the specific tools around Fedora. Even mock only comes
> into play once you've got a source rpm, tito is involved in getting
> you to that point. (although now that you mention it an option to do
> the build in mock would be quite useful)
>
> We did however write some code in Tito for integrating with CVS and
> Koji because we needed it to handle the volume of package builds
> involved with Spacewalk. This command checks out the module in CVS,
> imports the latest spec file and patches from git, builds the tarball,
> uploads it, and submits builds in Koji. This isn't documented yet as
> I've only run it against Red Hat's internal build system, but it
> should work fine in Fedora CVS and Koji, just need to test it.
>
> Cheers,
>
> Devan

Sounds interesting.  But I was able to do most of the above by stealing 
someone else's Makefile code and doing minimal tweaks.

I use make snaparchive and srpm/rpm to build a local snapshot, and I use 
make archive for release.

I have some magic in my rpm spec file which relies on git archive export 
substition:

%{!?branch: %define branch %(echo "$Format:%d$"| awk -F/ '{print $NF}'| 
sed -e 's/[()$]//g' | awk '{pr
int "." $NF}' | grep -v master)}
%if "0%{branch}" != "0"
%{!?timestamp: %define timestamp $Format:.%at$}
%endif

Name:           beaker
Version:        0.4.76
Release:        0%{?timestamp}%{?branch}%{?dist}

Whenever I make a new release I simply bump the version in the spec.  
But when I'm doing development and I build a test release off of a 
branch it creates an srpm/rpm with the last commits timestamp and the 
branch name.

After testing I merge the devel branch into master and make a release.

to make the above $Format$ work I added this:
cat .gitattributes
beaker.spec export-subst

I have a common file which includes the version:
more Common/beaker/__init__.py.in
# See http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages
try:
     __import__('pkg_resources').declare_namespace(__name__)
except ImportError:
     from pkgutil import extend_path
     __path__ = extend_path(__path__, __name__)

__version__ = '@VERSION@'

Wouldn't be too hard to add the release info here as well if needed.

The Makefile for common has the logic to replace @VERSION@ with the 
version specified in the spec file...

beaker/__init__.py: beaker/__init__.py.in
     sed -e 's/@VERSION@/$(PKGVERSION)/g' < $< > $@

You can see everything in my git repo..
http://git.fedorahosted.org/git/beaker.git?p=beaker.git;a=tree

The main difference is that I don't use setup.py to create the archive.  
I found git-archive to be much handier and easier to maintain.

[brett lentz]

On Mon, Jan 11, 2010 at 12:45 PM, Devan Goodwin  wrote:
> > This sounds an awful lot like reinventing the wheel to me.
> >
> > Why Tito and not Koji/Mock and the existing Red Hat/Fedora build
> > tools?  Does Tito support Koji and/or vice versa?
>
> The tool is intended to solve different problems than these, tito was
> more about answering questions like:
>
> Cheers,
>
> Devan
>
> --
> Devan Goodwin 
> http://rm-rf.ca

Thanks for the clarification.  Sounds like a worthwhile change to me.

---Brett.