build system glue scripts and requirements
by seth vidal
hi folks,
So right now I think things with mach and yum are working for building
fedora extras. The packages _seem_ like they're coming out right and
things seem functional enough. The second part that I need help and
input on is the glue scripts and requirements for having automatic
triggering of builds for packagers.
The questions I have:
1. if this is meant to run on the red hat boxes in the PHX coloc, what
does that infrastructure look like? What features does it have? Can we
assume all the build boxes have access to the cvs tree? Do we need to
worry about pushing srpms around?
2. How do folks want packagers to send notices about builds? Just a
cvs tag? A webpage? A gpg-signed email with specific content? A custom
xmpp/jabber-client to send a custom message to a listening build client
across an xmpp infrastructure? :)
3. What things am I missing or not understanding about what is needed
from the build system? The requirements I've been working under
are/were:
- self hosting on Fedora Core
- not crazy
What else do I need to think about?
4. Who else is interested in working on this and getting things
progressing more? The yum changes to mach are just a hackjob to get a
problem solved for the short term. However, I'd like to continue down
this general line of development. so <buffy>Where do we go from
here?</buffy>
Thanks,
-sv
18 years, 4 months
new mach+yum pkgs
by seth vidal
Hi Folks,
some new mach+yum pkgs are up:
http://linux.duke.edu/~skvidal/mach/pkgs/
Changes:
- fix for traceback when rebuilding srpm when spec file in srpm is mode
600
- add in:
lrwxrwxrwx 1 root root 15 Apr 17 2003 fd -> ../proc/self/fd
crw-rw-rw- 1 root root 1, 7 Dec 11 15:49 full
crw-rw-rw- 1 root root 1, 3 Dec 11 15:49 null
crw-rw-rw- 1 root root 5, 2 Dec 11 15:49 ptmx
crw-r--r-- 1 root root 1, 8 Dec 11 15:49 random
crw-rw-rw- 1 root root 5, 0 Dec 11 15:49 tty
crw-r--r-- 1 root root 1, 9 Dec 11 15:49 urandom
crw-rw-rw- 1 root root 1, 5 Dec 11 15:49 zero
to /dev in chroots to make sure builds work :)
- make the command 'mach yum' actually work
At this point I'll probably check the code in as 'extras-buildsys-temp'
or something equally as obviously named. Can anyone here tell me if I
can even make new modules in /cvs/fedora?
-sv
18 years, 5 months
RPATH and build root traces
by Warren Togami
/usr/lib/rpm/check-buildroot
/usr/lib/rpm/check-rpaths
%__arch_install_post /usr/lib/rpm/check-rpaths /usr/lib/rpm/check-buildroot
fedora-rpmdevtools contains these two scripts that can be run at the end
of rpmbuild automatically with this above rpmmacro. Can we consider
adding this as a standard to mach buildroots?
Enrico's scripts above have worked very well for us in detecting and
forcing us to correct RPATH problems for a long time now. I am not
aware of any false positives discovered during all this time.
Warren Togami
wtogami(a)redhat.com
18 years, 6 months
code checked in
by seth vidal
I checked the mach+yum stuff into:
extras-buildsys-temp in /cvs/fedora
-sv
18 years, 6 months
more mach+yum stuff
by seth vidal
Hi folks,
I added the libselinux-devel buildreq into the spec file and I fixed a
couple of problems in the default config and dist.d files.
Right now I'm using mach to build the packages coming out for fedora
extras 3 and fedora extras development.
I'm building them all on an fc3 x86_64 machine using 'setarch i686' for
the i386 builds.
Things appear pretty normal, so far :)
pkgs are here:
http://linux.duke.edu/~skvidal/mach/pkgs/
I've noticed one bug that i'm going to see about fixing - if the .spec
file is set with mode 600 or 400, then mach will traceback b/c it won't
be able to read the extracted spec from /tmp of the chroot. The easy-fix
to this is to just chmod the spec file so it's readable by everyone, as
soon as I figure out where this is happening I'll fix it and put a new
pkg up.
Also at the above url, I've included the incredibly simple shell scripts
I use for starting builds and maintaining the extras trees. They're
really not complex but maybe worth looking at, I think. Modifying those
to add an arch like ppc/ppc64 should be extremely trivial :)
If things work like we hope to expect then I'm probably going to
recommend that packagers for fedora extras start running their package
though mach before they request a build, just to verify that the package
will build at all.
Thanks!
-sv
18 years, 6 months
Build system ideas/requirements
by seth vidal
Items for thoughts:
1. build system using comps.xml for chroot install definitions (base,
build, minimal) - it would make sense and we could leverage the
groupinstall/update/remove mechanism in yum.
2. I talked to Jeremy some about this and I think if we do all rpmdb
transactions from OUTSIDE of the rpmdb and only build in the chroot then
we should be able to safely juggle multiple rpmdb versions from host to
chroot systems.
3. there's no reason to not develop a specialized script that uses the
yum modules that can be run by something like mach-helper for making
chroots reasonably correctly.
4. we're going to run into problems with contention for the rpm
transaction lock on the host system b/c rpm likes to lock the rpmdb on
the host system even when operating on the chroot system. A queuing
mechanism for access to the that lock so we know what else is left in
the process is not a bad idea.
>From what I can think breaking up the build system into:
- something that watches cvs for things to be built
- something that makes/handles/cleans up the chroots
- something that spawns the builds
- something that deals with the results
Is it reasonable to focus on these as modules to be developed?
-sv
18 years, 6 months
mach+yum update and yum 2.2.1
by seth vidal
Hi Everyone,
I've got a couple of items for you here:
new mach packages here:
http://linux.duke.edu/~skvidal/mach/pkgs/
- just a tar ball and src.rpm - you'll need to build it for whatever
platform you're on.
Changes:
- use yum to clean out buildroots
- minor tidying efforts - output some more data about what happens
in the 'removing packages' section.
I've also uploaded a yum 2.2.1 package to:
http://linux.duke.edu/yum/download/2.2/
I've not announced that to the yum mailing list yet but I will do so
tomorrow when I'm not so tired. This is useful for you folks running
fc3, rhel3 or rhel4 and wanting to use these packages.
Changes useful for mach:
- lock for yum run only in installroot, not in host root
- install-by-dependency
I know these work on x86 and x86_64. They _should_ work on ppc and
sparc, but I can't test those platforms.
I know these work on fc3, rawhide and rhel4, but I've done no testing on
rhel3.
I can also build rawhide/fc4t1 chroots and packages on fc3 w/o any rpmdb
problems, provided that I do not run rpm or yum from INSIDE the chroot,
only from the outside.
Let me know if these work for you or, more importantly, if they don't.
Thanks,
-sv
18 years, 6 months
buildroots.xml and packages have to be in same location?
by Dan Williams
Seth,
Using your mach SRPM and buildroots file, I had to actually pull down all the
packages locally and place the buildroots.xml file in the same dir as the
packages, then run createrepo on it before it would take (obviously pointing
mach to the local files as the yumsource).
It seems that (unless I'm mistaken) the buildroots.xml file needs to be in any
repo that I'd point mach to? Is that correct? Is there no way of keeping the
buildroots.xml file locally, or at some other site, separate from the actual
packages?
Anyway, once everything was in the same place it seemed to work fine.
Note: This FC3 (well, Aurora "corona" on Sparc, which is == FC3).
mach: mach-0.4.6.1-0.fdr.0.20050314.000733
yum: yum-2.2.0-0.fc3
Dan
18 years, 6 months
new mach+yum packages
by seth vidal
Hey folks,
I hacked some more on mach2+yum today.
Changes:
- made mach use yum/comps groups for 'minimal', 'base', 'build'
groupinstalls. (this killed apt-get compat so I pulled the apt stuff
out) The groups file used for these installs is here:
http://linux.duke.edu/~skvidal/mach/i386/buildroots.xml
- made new config files for the groups functionality and for fedora
development
- misc changes I cannot recall right now :)
Things that I've tested
- building on fedora core 4 test1 and rawhide on x86_64
- building in a 'setarch i686' shell on x86_64 on fedora core 4 test1
and rawhide.
You can get things here:
http://linux.duke.edu/~skvidal/mach/pkgs/
I'm also adding a small shell script I use to build packages from fedora
extras cvs.
I think these pkgs should let people build/test pkgs for extras in a
known environment.
This will 'mostly' work on fc3 but not quite. I need to backport a
couple of items from yum 2.3.X to yum 2.2.X, then it will work.
-sv
18 years, 6 months
minimum build environment
by seth vidal
Hi everyone,
I've been building some of fedora extras for fc4t1 with mach. Something
I'm encountering are A LOT of unlisted buildreqs. A lot of items needing
things like tclConfig and intltool.
I'm curious - what do you want to be the minimum build environment? What
things should we assume.
-sv
18 years, 6 months
