SCM Root
by Moray Henderson
Ah! SCM checkout and "make sources" are run in /tmp/scmroot, not
/builddir/build/SOURCES. That might help people writing a custom "make
sources".
Moray.
"To err is human; to purr, feline."
11 years, 5 months
RE: Dist tag question
by Moray Henderson
> From: Moray Henderson [mailto:Moray.Henderson@ict-software.org]
> Sent: 25 April 2012 15:27
> I've got Koji to build rpms - but the task is still failing with the
> message
>
> GenericError: Unable to complete build: release mismatch (build: 2,
> rpm: 2.el6)
>
> The different build steps are giving different results. My .spec file
> contains
>
> Release: %{release}%{?dist}
>
> and my build and srpm-build groups both contain redhat-release, which
> resolves to centos-release-6-2.el6.centos.7.x86_64 and contains
> /etc/rpm/macros.dist to set "%dist .el6".
>
> However buildSRPMFromSCM results in
>
> Building target platforms: x86_64
> Building for target x86_64
> Wrote: /builddir/build/SRPMS/mypackage-1.0-2.src.rpm
>
> while buildArch also builds a srpm before the binary rpm:
>
> Building target platforms: noarch
> Building for target noarch
> Wrote: /builddir/build/SRPMS/mypackage-1.0-2.el6.src.rpm
>
> Building target platforms: noarch
> Building for target noarch
> ...
> Wrote: /builddir/build/RPMS/slspkg-1.0-2.el6.noarch.rpm
>
> My Koji hub lists the build as "mypackage-1.0-2". If I get
> buildSRPMFromSCM to use the dist macro, will the Koji hub pick up the
> right name? How do I get buildSRPMFromSCM to do that?
(There was a typo in the above: slspkg should read mypackage. Sorry.)
Never mind - got it. Yum needs the correct package name, it doesn't match
on what the package provides.
Still getting to grips with the Koji and Koji hub interfaces. I found the
root.log of the task and that gave me the clue:
DEBUG util.py:257: Installing:
DEBUG util.py:257: bash x86_64 4.1.2-8.el6.centos
build 902 k
DEBUG util.py:257: curl x86_64 7.19.7-26.el6_2.4
build 192 k
DEBUG util.py:257: make x86_64 1:3.81-19.el6
build 389 k
DEBUG util.py:257: redhat-rpm-config noarch 9.0.3-34.el6
build 57 k
DEBUG util.py:257: rpm-build x86_64 4.8.0-19.el6_2.1
build 124 k
DEBUG util.py:257: shadow-utils x86_64 2:4.1.4.2-13.el6
build 896 k
Some packages from srpm-build were missing from the install, because they
are in the repository under different names. They got pulled in to the
buildArch buildroot as dependencies.
Moray.
"To err is human; to purr, feline."
11 years, 5 months
Dist tag question
by Moray Henderson
Hi again,
I've got Koji to build rpms - but the task is still failing with the message
GenericError: Unable to complete build: release mismatch (build: 2, rpm:
2.el6)
The different build steps are giving different results. My .spec file
contains
Release: %{release}%{?dist}
and my build and srpm-build groups both contain redhat-release, which
resolves to centos-release-6-2.el6.centos.7.x86_64 and contains
/etc/rpm/macros.dist to set "%dist .el6".
However buildSRPMFromSCM results in
Building target platforms: x86_64
Building for target x86_64
Wrote: /builddir/build/SRPMS/mypackage-1.0-2.src.rpm
while buildArch also builds a srpm before the binary rpm:
Building target platforms: noarch
Building for target noarch
Wrote: /builddir/build/SRPMS/mypackage-1.0-2.el6.src.rpm
Building target platforms: noarch
Building for target noarch
...
Wrote: /builddir/build/RPMS/slspkg-1.0-2.el6.noarch.rpm
My Koji hub lists the build as "mypackage-1.0-2". If I get buildSRPMFromSCM
to use the dist macro, will the Koji hub pick up the right name? How do I
get buildSRPMFromSCM to do that?
Moray.
"To err is human; to purr, feline."
11 years, 5 months
How does...?
by Moray Henderson
Hello again,
I'm trying to understand how Koji does things in order to migrate from our
previous home-made build system with svn repositories to Koji. Most of our
stuff is in svn as source files, not tarballs, so I'll have to implement a
"make sources". I was looking in the Fedora git repositories, but couldn't
see how this is actually done. I can see how it _used_ to be done: with a
common package containing a Makefile.common. However that doesn't seem to
be available any more.
I've found fedora-packager and fedpkg; there was a thread here in 2010
(http://www.mail-archive.com/buildsys@lists.fedoraproject.org/msg00619.html)
which suggested that they configured Koji to issue the command to get the
sources. However, that's "allowed_scms" in kojid.conf, and I can't see
anything in either package now which modifies that.
So basically I'd really like to know the steps that Fedora's Koji goes
through to build packages like anaconda - which has an old Makefile, and has
no URL to say where to find the sources.
Moray.
"To err is human; to purr, feline."
11 years, 5 months
koji: small patch to fix warnings
by John Morris
After the koji.conf 'pkgurl' attribute was enabled (sigul wants this;
I'll submit a patch for that too, later), one of my scripts broke
because the "Warning: the pkgurl option is obsolete" message is printed
to stdout instead of stderr. Here's a little patch to print the warning
with the warn() function instead.
I realize some warnings from the cli might be best printed to stdout,
but for ones that might appear when using a computer-readable type of
command like 'list-* --quiet', stderr seems like the better choice.
Thanks-
John
11 years, 5 months
Re: How does...?
by Andreas Mack
I still use the old "make sources" way. Pretty much copied from the Fedora
set up, with the "lookaside" repository for the files.
Here's how I understood it, please correct me if I'm wrong:
Koji checks out the package subdir and executes "make sources" there. The
Makefile within the subdir of the package in svn references a
"Makefile.common" that lives in "trunk/common". The Makefile retrieves the
"Makefile.common" from svn and loads it. Makefile.common then retrieves the
sources file and the sources.
To download them it uses the REPOSITORY variable, i.e. the svn host. It
fetches the sources like this:
http://REPOSTITORY_HOST/repo/pkgs/<packagename>/<source file name>/<md5sum
of source file>/<source file name>. If the md5sum matches, "make sources"
returns and koji proceeds with building.
As Jesse wrote back then, if you don't want to do it that way, you're free
to specify any other command that will download the sources. For me it was
a good start without getting too much into fedpkg stuff.
Andreas.
On Fri, Apr 20, 2012 at 18:34, Moray Henderson <
Moray.Henderson(a)ict-software.org> wrote:
> Hello again,
>
> I'm trying to understand how Koji does things in order to migrate from our
> previous home-made build system with svn repositories to Koji. Most of our
> stuff is in svn as source files, not tarballs, so I'll have to implement a
> "make sources". I was looking in the Fedora git repositories, but couldn't
> see how this is actually done. I can see how it _used_ to be done: with a
> common package containing a Makefile.common. However that doesn't seem to
> be available any more.
>
> I've found fedora-packager and fedpkg; there was a thread here in 2010
> (
> http://www.mail-archive.com/buildsys@lists.fedoraproject.org/msg00619.html
> )
> which suggested that they configured Koji to issue the command to get the
> sources. However, that's "allowed_scms" in kojid.conf, and I can't see
> anything in either package now which modifies that.
>
> So basically I'd really like to know the steps that Fedora's Koji goes
> through to build packages like anaconda - which has an old Makefile, and
> has
> no URL to say where to find the sources.
>
>
> Moray.
> "To err is human; to purr, feline."
>
>
>
>
>
>
> --
> buildsys mailing list
> buildsys(a)lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/buildsys
11 years, 5 months
mockchain.py
by Seth Vidal
I worked on this a bit last week. I thought I'd mention it here:
http://fedorapeople.org/gitweb?p=skvidal/public_git/scripts.git;a=tree;f=...
mockchain.py
mockchain.py [options] chrootname file.src.rpm [file1.src.rpm]
[file2.src.rpm] ...
Builds a series of srpms in mock one at a time. After each successful
build
of a package it adds the resulting packages to a local repo which
are available to the next package to satisfy buildreqs.
options:
-c - continue on package build failure - by default it will exit if
a package fails to build. set this if you wish it to try and
continue
for the rest of the packages.
-l path - set the path to put the results/repo in. This path needs to be
somewhere accessible to users other than you for reading as the
mock process doesn't run as you.
-a url - add this repo url to the yumconfig for the buildroot. This can
be specified multiple times. Let's you point to multiple
paths beyond the default to pull build deps from.
--recurse - build all pkgs, record the failures and try to rebuild them
again and again until everything gets built (or until the
set of pkgs failing to build are the same over)
This does not try to sort the packages by build order b/c that is too much
effort and not obviously doable with the buildreq information we have.
The build process when you use -l is idempotent so a package which has
already been successfully built will not be built again.
If you want to force the rebuild of a package which has been built
successfully simply remove the 'success' file from the dir with the
package
results in it. Package results are written out into
a dir named:
chrootname/pkgname-ver-rel/
for example:
fedora-16-x86_64/yum-3.2.29-0/
-sv
11 years, 5 months
Documentation discussion
by Moray Henderson (ICT)
I spotted a couple of points where I think the Koji documentation could be
improved:
In Koji/ServerHowTo#Koji_filesystem_skeleton you give an example
kojiadmin@localhost$ koji add-host kojibuilder1 x86_64 i386
which looks as if a simple user name style is sufficient. Near the end of
the same document, in
Koji/ServerHowTo#Add_the_host_entry_for_the_koji_builder_to_the_database,
you say it like this:
kojiadmin@localhost$ koji add-host kojibuilder1.example.com i386 x86_64
Now it looks as if you're supposed to use the full host name, not a user
name. It would be good to make those entries consistent and, if you do need
the FQDN, to explain why.
In
Koji/ExternalRepoServerBootstrap#Bootstrapping_a_new_External_Repo_Koji_buil
d_environment you have a note
* NOTE: If you are adding multiple external repos, koji assigns a priority
to each repo in FIFO order. This may cause updated packages to not be
visible if a repo with older packages is ranked at a higher priority (lower
numeric value). Use the -p flag to set specific repo priorities.
That suggests that repos with newer packages should be added first:
testing/\$arch
updates/\$arch
os/\$arch
However in
Koji/ExternalRepoServerBootstrap#Examples_of_urls_to_use_for_external_Reposi
tories your examples list the os release repo before the updates repo,
which would give the repo with the older packages the lower numeric value
and therefore the higher priority.
Moray.
"To err is human; to purr, feline."
OM International Limited - Unit B Clifford Court, Cooper Way - Carlisle CA3 0JG - United Kingdom
Charity reg no: 1112655 - Company reg no: 5649412 (England and Wales)
11 years, 5 months
rescue mode and lvm
by Sérgio Basto
Hi,
from http://fedorasolved.org/Members/zcat/shrink-lvm-for-new-partition
"Boot into rescue mode and on the last step make sure to SKIP mounting
your filesystems under /mnt/sysimage
Activate LVM
* lvm vgchange -a y"
If I boot into rescue mode at the not skip mounting, I got a message
that can't find any (lvm) partitions.
but could be good mounting my filesystems even if a lvm ?
if yes, rescue mode should active LVM, before scan for partitions with
Linux .
I check with F17 boot.iso and also the same with F16 boot.iso .
Thanks,
--
Sérgio M. B.
11 years, 5 months
Re: newRepo Permission denied: '/mnt/koji/repos'
by Andreas Mack
Hey,
just my two cents: It's possible to run the builders completely without
NFS. I use "topurl=" instead of "topdir=" in kojid.conf. The URL is
http://kojihub.domain.tld/kojimnt where kojimnt is apache aliased to the
/mnt/koji on kojihub. Regen repos has to be run on kojihub in thatcase.
It's the only one in the createrepo channel.
As for the NFS problem: Smells so much like Selinux. Maybe on the
server-side? Do you use NFS4 ?
HTH
Andreas
On Thu, Apr 12, 2012 at 17:40, Moray Henderson <
Moray.Henderson(a)ict-software.org> wrote:
> Is this the right place for questions on local koji installations?
>
> Fresh setup of koji on CentOS 6.2. I've got hub, web and builder all
> talking to each other, external repositories defined for the build tag and
> build groups set up.
>
> /mnt/koji is an nfs mount with root squashed to uid 48 (apache). I've
> tested that I can write to the subdirectories as root and the owner comes
> out as apache. The directory looks like
>
> # ll -R koji
> koji:
> total 16
> drwxr-xr-x 2 apache apache 4096 Apr 12 11:13 packages
> drwxr-xr-x 3 apache apache 4096 Apr 12 15:20 repos
> drwxr-xr-x 2 apache apache 4096 Apr 12 11:13 scratch
> drwxr-xr-x 2 apache apache 4096 Apr 12 11:13 work
>
> koji/packages:
> total 0
>
> koji/repos:
> total 0
>
> koji/scratch:
> total 0
>
> koji/work:
> total 0
>
> The Koji/ExternalRepoServerBootstrap document says "Wait for the repo to
> regenerate, and you should now be able to run a build successfully."
> However, Koji-web lists the newRepo task as failed with result "<type
> 'exceptions.OSError'>: [Errno 13] Permission denied: '/mnt/koji/repos'".
> On
> the builder, kojid.log reports:
>
> 2012-04-12 14:20:31,067 [INFO] koji.build: Starting up
> 2012-04-12 14:20:34,363 [INFO] koji.TaskManager: Attempting to take task
> 176
> 2012-04-12 14:20:36,275 [INFO] koji.TaskManager: pids: {176: 17925}
> 2012-04-12 14:20:36,855 [WARNING] koji.TaskManager: FAULT:
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/koji/daemon.py", line 1114, in
> runTask
> response = (handler.run(),)
> File "/usr/lib/python2.6/site-packages/koji/tasks.py", line 146, in run
> return self.handler(*self.params,**self.opts)
> File "/usr/sbin/kojid", line 2491, in handler
> repo_id, event_id = self.session.host.repoInit(tinfo['id'], **kwargs)
> File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1510, in
> __call__
> return self.__func(self.__name,args,opts)
> File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1760, in
> _callMethod
> raise err
> Fault: <Fault 1: "<type 'exceptions.OSError'>: [Errno 13] Permission
> denied: '/mnt/koji/repos'">
>
> 2012-04-12 14:20:37,110 [INFO] koji.TaskManager: open task: {'waiting':
> None, 'id': 176, 'weight': 0.10000000000000001}
>
>
> I've looked into the code, but my python is not up to debugging that. It's
> not an SELinux problem (I tried permissive mode) and /mnt/koji is mounted
> read-write on the builder even though the documentation says that's not
> necessary. Can someone point me in the right direction?
>
>
> Moray.
> "To err is human; to purr, feline."
>
>
> --
> buildsys mailing list
> buildsys(a)lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/buildsys
11 years, 5 months
