buildsys January 2011

buildsys@lists.fedoraproject.org

22 participants
22 discussions

[BUG] Koji and Lustre
by Thomas Guthmann 09 Sep '11

09 Sep '11

Hey guys, We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py) when used with a Lustre filesystem. This function parses all attributes of every file of a build and is used when you want to display build.log/root.log through the web interface. Everything returned by listTaskOutput is returned through XML-RPC and as a result we had this : An error has occurred while processing your request. Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC limits'> Traceback (most recent call last): File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in publish_object return old_publish_object(req, object) File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py", line 412, in publish_object return publish_object(req,util.apply_fs_data(object, req.form, req=req)) File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line 439, in apply_fs_data return object(**args) File "/usr/share/koji-web/scripts/index.py", line 649, in getfile output = server.listTaskOutput(taskID, stat=True) File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468, in __call__ return self.__func(self.__name,args,opts) File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718, in _callMethod raise err Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC limits'> The issue comes from the st_dev value gathered by getStat (stat). In Lustre this value can be very high and that's why it complains. To fix that, we used the same condition than st_size, we cast the value as a string. See attached patch. Example (see 'Device:' the value after the '/'): # stat build.log File: `build.log' Size: 106021 Blocks: 208 IO Block: 2097152 regular file Device: e04ae70eh/3763005198d Inode: 721664 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache) Access: 2010-07-13 10:44:40.000000000 +1000 Modify: 2010-07-12 12:54:15.000000000 +1000 Change: 2010-07-12 12:54:52.000000000 +1000 # That's what is read by listTaskOutput relfilename=build.log ATTR=st_atime GETATTR=1278981616 relfilename=build.log ATTR=st_blksize GETATTR=2097152 relfilename=build.log ATTR=st_blocks GETATTR=208 relfilename=build.log ATTR=st_ctime GETATTR=1278903292 relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\ relfilename=build.log ATTR=st_gid GETATTR=48 relfilename=build.log ATTR=st_ino GETATTR=721664 relfilename=build.log ATTR=st_mode GETATTR=33188 relfilename=build.log ATTR=st_mtime GETATTR=1278903255 relfilename=build.log ATTR=st_nlink GETATTR=1 relfilename=build.log ATTR=st_rdev GETATTR=0 relfilename=build.log ATTR=st_size GETATTR=106021 relfilename=build.log ATTR=st_uid GETATTR=48 Hope it helps, lost a good amount of time on that one :) Cheers, Thomas

2 3

Cleaning scratch & work directories
by Anthony Joseph Messina 07 Feb '11

07 Feb '11

I'm continuing work a on a private Koji instance (Kerberos) and am now working on automating cleanup, etc. It is mentioned here http://lists.fedoraproject.org/pipermail/buildsys/2010-December/003415.html that "Koji leaves it to the administrator to clean out the scratch build directory." Is there anything special about cleaning up scratch builds or the work/{cli-build,cli-import,tasks} directories for that matter? Do I need to interact with the Koji database in any way to let koji know these files are gone? What I mean is, can I simply remove the files and directories without any Koji integration? Thanks. -A -- Anthony - http://messinet.com - http://messinet.com/~amessina/gallery 8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

4 6

How do I remove a package from koji?
by steve.webb＠beatport.com 03 Feb '11

03 Feb '11

How do I remove or clean out a build repo/tag from koji to start over with a new build tag? - Steve -- Steve Webb | System Administrator Beatport | Play With Music ------------------------------------------ 2399 Blake Street, Suite 170 Denver, Colorado USA 80205 tel: +1.720.932.9103 fax: +1.720.932.9104 noc: +1.303.565.2710 mobile: +1.303.564.4269

6 10

I've got admin privs, but I can't remove packages through the web interface
by steve.webb＠beatport.com 28 Jan '11

28 Jan '11

I've got admin privs, but I don't seem to be able to remove packages or builds through the web interface. Is the web interface missing something? [swebb@bpbuild001 ~]$ koji grant-permission admin swebb GenericError: user swebb already has permission: admin - Steve -- Steve Webb | System Administrator Beatport | Play With Music ------------------------------------------ 2399 Blake Street, Suite 170 Denver, Colorado USA 80205 tel: +1.720.932.9103 fax: +1.720.932.9104 noc: +1.303.565.2710 mobile: +1.303.564.4269

2 1

pungi vs anaconda buildinstall: TMPDIR works, local cache ignored...
by Martin Langhoff 28 Jan '11

28 Jan '11

Working with Pungi 2.1.4 to build the School Server OLPC spin (which fir test purposes is being done on F14, though that's not the final target). - Seems like Pungi has for a long time avoided setting a smart TMPDIR when calling anaconda's buildinstall. This uses a sizable chunk of /tmp . I've pached pypungi/__init__.py to set it, and it works fairly well (for me at least). Was going to post a patch but v2.5.0 has been completely reworked. - The buildinstall process is dog slow for me -- because buildinstall is ignoring the pungi cache and grabbing it all itself, again. Is this expected? Should pungi add a "repo" of it's own local cache? Looking at pungi's git repo, I ended up looking at Lorax code -- so I gather patches to scratch my own itch won't be merged. Keen on hearing when new pungi plus new lorax can build a F14 image. cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

3 4

GenericError: Unable to complete build: release mismatch (build: 4.P1.el5_4.2, rpm: 4.P1.el5.2)
by steve.webb＠beatport.com 26 Jan '11

26 Jan '11

What does koji mean when I get this error? GenericError: Unable to complete build: release mismatch (build: 4.P1.el5_4.2, rpm: 4.P1.el5.2) The individual builds completed successfully, but the "Result" of the build ended up with that error. - Steve -- Steve Webb | System Administrator Beatport | Play With Music ------------------------------------------ 2399 Blake Street, Suite 170 Denver, Colorado USA 80205 tel: +1.720.932.9103 fax: +1.720.932.9104 noc: +1.303.565.2710 mobile: +1.303.564.4269

3 5

koji client on another machine
by steve.webb＠beatport.com 26 Jan '11

26 Jan '11

Hey there. I think that I've got a working koji server authenticating using kerberos (yea!). My latest issue is that I'm trying to call koji from another host using the koji command-line client. I can get read-only commands, but build submissions are failing: [swebb(a)bplwebdev001.co0 ~]$ koji -s http://bpbuild001.co0.nar.beatportcorp.net/kojihub list-external-repos External repo name URL ------------------------- ---------------------------------------- dist-EL5-beatport-external-repo-centos-updates-x86_64 http://repo.beatportcorp.net/centos/5/updates/x86_64/ dist-EL5-beatport-external-repo-centos-epel-x86_64 http://repo.beatportcorp.net/epel/5/x86_64/ dist-EL5-beatport-external-repo-centos-php_new53-x86_64 http://repo.beatportcorp.net/php_new5.3/ dist-EL5-beatport-external-repo-centos-updates-i386 http://repo.beatportcorp.net/centos/5/updates/i386/ dist-EL5-beatport-external-repo-centos-epel-i386 http://repo.beatportcorp.net/epel/5/i386/ dist-EL5-beatport-official-x86_64 http://mirrors.kernel.org/centos/5/os/x86_64/ dist-EL5-beatport-official-i386 http://mirrors.kernel.org/centos/5/os/i386/ dist-EL5-beatport-external-repo-buildsys http://buildsys.fedoraproject.org/buildgroups/rhel5/x86_64/ [swebb(a)bplwebdev001.co0 ~]$ koji -s http://bpbuild001.co0.nar.beatportcorp.net/kojihub build dist-EL5-beatport cdrtools-2.01-10.7.el5.src.rpm Kerberos authentication failed: Server not found in Kerberos database (-1765328377) Here's the contents of my psql koji DB: ji=> select * from users; id | name | password | status | usertype | krb_principal ----+-------------------------------------+----------+--------+----------+------------------------------------------------------------------- 2 | swebb | | 0 | 0 | swebb(a)AUTH.BEATPORTCORP.NET 1 | koji | | 0 | 0 | koji/bpbuild001.co0.nar.beatportcorp.net(a)AUTH.BEATPORTCORP.NET 3 | kojira | | 0 | 0 | 4 | bpbuild001.co0.nar.beatportcorp.net | | 0 | 1 | compile/bpbuild001.co0.nar.beatportcorp.net(a)AUTH.BEATPORTCORP.NET 5 | dharris | | 0 | 0 | dharris(a)AUTH.BEATPORTCORP.NET 6 | jsteffan | | 0 | 0 | jsteffan(a)AUTH.BEATPORTCORP.NET 7 | talexander | | 0 | 0 | talexander(a)AUTH.BEATPORTCORP.NET 8 | jhiza | | 0 | 0 | jhiza(a)AUTH.BEATPORTCORP.NET 9 | taitken | | 0 | 0 | taitken(a)AUTH.BEATPORTCORP.NET 10 | ccardone | | 0 | 0 | ccardone(a)AUTH.BEATPORTCORP.NET (10 rows) - Steve -- Steve Webb | System Administrator Beatport | Play With Music ------------------------------------------ 2399 Blake Street, Suite 170 Denver, Colorado USA 80205 tel: +1.720.932.9103 fax: +1.720.932.9104 noc: +1.303.565.2710 mobile: +1.303.564.4269

2 2

[PATCH] fedpkg: call sources before running mockbuild
by Josh Boyer 25 Jan '11

25 Jan '11

Currently a 'fedpkg mockbuild' in a fresh clone of a package will result in something like: [jwboyer@r2d2 kernel]$ fedpkg mockbuild error: File /home/jwboyer/src/fedora/kernel/linux-2.6.34.tar.bz2: No such file or directory Could not run mockbuild: Command 'rpmbuild --define '_sourcedir /home/jwboyer/src/fedora/kernel' --define '_specdir /home/jwboyer/src/fedora/kernel' --define '_builddir /home/jwboyer/src/fedora/kernel' --define '_srcrpmdir /home/jwboyer/src/fedora/kernel' --define '_rpmdir /home/jwboyer/src/fedora/kernel' --define 'dist .fc13' --define 'fedora 13' --define 'fc13 1' --nodeps -bs /home/jwboyer/src/fedora/kernel/kernel.spec' returned non-zero exit status 1 [jwboyer@r2d2 kernel]$ because the sources have not been downloaded yet. Under the old dist-cvs system, a mockbuild had a dependency on srpm, which had a dependency on sources. The following patch accomplishes the a similar dependency in fedpkg by making it fetch the sources prior to doing the mock setup. Tested locally on F14. josh --- diff --git a/src/fedpkg.py b/src/fedpkg.py index dbae1a8..31c18dd 100755 --- a/src/fedpkg.py +++ b/src/fedpkg.py @@ -576,6 +576,12 @@ def local(args): sys.exit(1) def mockbuild(args): + try: + pyfedpkg.sources(args.path) + except pyfedpkg.FedpkgError, e: + log.error('Could not download sources: %s' % e) + sys.exit(1) + # Pick up any mockargs from the env mockargs = [] try:

2 1

Strange mock build failure due to typo
by Paul Howarth 24 Jan '11

24 Jan '11

Hi, today I have been preparing an update to perl-Math-Pari and came across a very strange build failure whilst doing a local mock build on a Fedora 13 x86_64 host. My package built successfully on x86_64 but when I tried to build for i386, the build failed during %setup but without any diagnostics. The SRPM was installed but no attempt to install its build requirements was made. The root.log showed an exit status of 0 for all commands that had been run. After much experimentation bisecting the changes I had made, I discovered that a typo in the changelog entry was the culprit: I had set the year to 2100 instead of 2010. So it would appear that somewhere in the mock/yum/rpm stack there may be a year 2038 problem waiting to bite us (though I suspect there may not be too many 32-bit builds happening by then). Seriously though, it would be nice to have better diagnostics for this and perhaps an rpmlint check for changelog entries in the future? Paul.

2 1

koji mock build trying to use package from another build
by steve.webb＠beatport.com 24 Jan '11

24 Jan '11

Hey there. Another issue (I'm getting close to getting everything working, I swear!) I've done several good builds of packages and I switched around some external repos to use local repos instead. Not sure if that's related, but I thought that I'd throw that in there in case it's a variable. When doing the mock part of the buildroot, I'm getting: [snip] termcap noarch 1:5.5-1.20060701.1 build 265 k tzdata x86_64 2010l-1.el5 build 796 k udev x86_64 095-14.21.el5_5.1 build 2.4 M util-linux x86_64 2.13-0.52.el5_4.1 build 1.8 M zlib x86_64 1.2.3-3 build 51 k Transaction Summary ================================================================================ Install 106 Package(s) Total download size: 109 M Installed size: 109 M http://bpbuild001.co0.nar.beatportcorp.net/packages/less/436/2.el5/x86_64/l…: [Errno 14] HTTP Error 404 : http://bpbuild001.co0.nar.beatportcorp.net/packages/less/436/2.el5/x86_64/l… Trying other mirror. Error Downloading Packages: less-436-2.el5.x86_64: failed to retrieve less-436-2.el5.x86_64.rpm from build error was [Errno 14] HTTP Error 404 : http://bpbuild001.co0.nar.beatportcorp.net/packages/less/436/2.el5/x86_64/l… [/snip] The bpbuild001.co0.nar.beatportcorp.net machine is my build server, so I'm guessing that the new package is expecting to find the rpm for 'less' on my build server instead of on the repo, right? I'm expecting that I just need to configure the .../packages/ URL to hit the right packages directory on my server. I changed the kojidir from /mnt/koji to /data/koji (a larger disk). Which config file do I need to change to get .../packages/ to point to the /data/koji/... ?? I checked the dir, and the files are in there, so I just need to change that config var somewhere. Thanks in advance. - Steve -- Steve Webb | System Administrator Beatport | Play With Music ------------------------------------------ 2399 Blake Street, Suite 170 Denver, Colorado USA 80205 tel: +1.720.932.9103 fax: +1.720.932.9104 noc: +1.303.565.2710 mobile: +1.303.564.4269

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys January 2011