buildsys June 2013

buildsys@lists.fedoraproject.org

5 participants
6 discussions

[koji PATCH] allow setting build priority
by Dan Horák 09 Dec '13

09 Dec '13

When building for multiple tags it could be useful to set different priorities for the tasks, eg. rawhide builds will be lower prio than branched builds. --- util/koji-shadow | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/util/koji-shadow b/util/koji-shadow index 25492d5..8790cf8 100755 --- a/util/koji-shadow +++ b/util/koji-shadow @@ -140,6 +140,8 @@ def get_options(): help=_("tag sucessful builds into the tag we are building, default is to not tag")) parser.add_option("--arches", help=_("arches to use when creating tags")) + parser.add_option("--priority", type="int", default=5, + help=_("priority to set for submitted builds")) #parse once to get the config file (options, args) = parser.parse_args() @@ -1080,7 +1082,7 @@ class BuildTracker(object): #TODO ...and verify repo repo_id, event_id = session.getTaskResult(task_id) #kick off build - task_id = session.build(src, None, opts={'repo_id': repo_id}, priority=5 ) + task_id = session.build(src, None, opts={'repo_id': repo_id}, priority=options.priority ) return task_id def report(self): -- 1.8.1.4

2 1

[koji PATCH] allow setting the timeout for builds from config file
by Dan Horák 09 Dec '13

09 Dec '13

--- builder/kojid | 5 +++-- builder/kojid.conf | 3 +++ koji/__init__.py | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/builder/kojid b/builder/kojid index f906d0b..18c1fa4 100755 --- a/builder/kojid +++ b/builder/kojid @@ -204,7 +204,7 @@ class BuildRoot(object): for k in ('repoid', 'tag_name'): if hasattr(self, k): opts[k] = getattr(self, k) - for k in ('mockdir', 'topdir', 'topurl', 'topurls', 'packager', 'vendor', 'distribution', 'mockhost', 'yum_proxy'): + for k in ('mockdir', 'topdir', 'topurl', 'topurls', 'packager', 'vendor', 'distribution', 'mockhost', 'yum_proxy', 'rpmbuild_timeout'): if hasattr(self.options, k): opts[k] = getattr(self.options, k) opts['buildroot_id'] = self.id @@ -3060,6 +3060,7 @@ def get_options(): 'maven_repo_ignore': '*.md5 *.sha1 maven-metadata*.xml _maven.repositories ' 'resolver-status.properties *.lastUpdated', 'failed_buildroot_lifetime' : 3600 * 4, + 'rpmbuild_timeout' : 3600 * 24, 'cert': '/etc/kojid/client.crt', 'ca': '/etc/kojid/clientca.crt', 'serverca': '/etc/kojid/serverca.crt'} @@ -3067,7 +3068,7 @@ def get_options(): for name, value in config.items('kojid'): if name in ['sleeptime', 'maxjobs', 'minspace', 'retry_interval', 'max_retries', 'offline_retry_interval', 'failed_buildroot_lifetime', - 'timeout',]: + 'timeout', 'rpmbuild_timeout',]: try: defaults[name] = int(value) except ValueError: diff --git a/builder/kojid.conf b/builder/kojid.conf index 1fa9ad4..05fe3b0 100644 --- a/builder/kojid.conf +++ b/builder/kojid.conf @@ -32,6 +32,9 @@ ; The _host string to use in mock ; mockhost=koji-linux-gnu +; Timeout for build duration (24 hours) +; rpmbuild_timeout=86400 + ; The URL for the xmlrpc server server=http://hub.example.com/kojihub diff --git a/koji/__init__.py b/koji/__init__.py index f42226b..8ab81a8 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -1266,7 +1266,7 @@ def genMockConfig(name, arch, managed=False, repoid=None, tag_name=None, **opts) # don't encourage network access from the chroot 'use_host_resolv': opts.get('use_host_resolv', False), # Don't let a build last more than 24 hours - 'rpmbuild_timeout': 86400 + 'rpmbuild_timeout': opts.get('rpmbuild_timeout', 86400) } # bind_opts are used to mount parts (or all of) /dev if needed. -- 1.7.7.6

2 1

Koji-1.7.1-1.fc17 repository toplink failure
by Anthony Joseph Messina 03 Jul '13

03 Jul '13

After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log: error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/... The repos are created successfully: $ /usr/bin/createrepo -vd -o /tmp/koji/tasks/9129/19129/repo -i /mnt/koji/repos/f17-build/3627/x86_64/pkglist -g /mnt/koji/repos/f17- build/3627/groups/comps.xml --update --skip-stat /mnt/koji/repos/f17- build/3627/x86_64 Scanning old repo data Spawning worker 0 with 36 pkgs Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-devel-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-libs-2.6.1-3.fc17.1.x86_64.rpm ... ... I believe the issue is related to the following commit: https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaced… I'm not sure why when building packages, the builders are inserting '/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink' instead of topdir or topurl, such as '/mnt/koji' or 'http://example.com/kojifiles' Right now I'm working on selectively reverting that commit to see if I can build packages against other local packages. Any assistance is appreciated. Thanks. -Anthony -- Anthony - http://messinet.com - http://messinet.com/~amessina/gallery 8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

4 13

New Koji Install, createrepo never completes.
by Stephen Brasher 18 Jun '13

18 Jun '13

I’ve installed all the pieces of koji-1.8.0 (hub, web, kojira kojid), configured for ssl and have everything working (I think). Following the Server Bootstrap and I import all the SRPM’s and RPM’s for fedora fc17 (approximately 35K rpms). Can see all the packages imported on the web in the Packages and the Builds tabs. When I execute the “koji add-group f17-build build” and the “koji add-group-pkg f17-build build pkg1”, kojira starts two tasks, a newRepo task and a createrepo task. Looking at the kojira.log, I see the following repeated about every 15 seconds: 2013-06-10 14:07:53,790 [DEBUG] koji.repo.manager: Task 17 is waiting 2013-06-10 14:07:53,876 [DEBUG] koji.repo.manager: Untracked task 19 is waiting 2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Current tasks: {2: 17} 2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Found 1 untracked newRepo tasks 2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Updating repos 2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Reading current repo data 2013-06-10 14:07:53,977 [DEBUG] koji.repo.manager: Repo data: [{'create_event': 34851, 'state': 2, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370716632.03544, 'id': 1}, {'create_event': 34888, 'state': 2, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370717056.06374, 'id': 2}, {'create_event': 34891, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370882529.75254, 'id': 3}, {'create_event': 34892, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370890996.61094, 'id': 4}, {'create_event': 34893, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370891441.92783, 'id': 5}, {'create_event': 34894, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370891585.94049, 'id': 6}, {'create_event': 34895, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370892392.69252, 'id': 7}, {'create_event': 34896, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370892439.9626, 'id': 8}, {'create_event': 34897, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370892487.90491, 'id': 9}, {'create_event': 34898, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370892548.62247, 'id': 10}, {'create_event': 34899, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370892581.09955, 'id': 11}, {'create_event': 34900, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370895682.22075, 'id': 12}, {'create_event': 34901, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370895683.00858, 'id': 13}, {'create_event': 34905, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370895884.96146, 'id': 14}, {'create_event': 34931, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370896102.89165, 'id': 15}] 2013-06-10 14:07:54,061 [DEBUG] koji.repo.manager: Needed tags: [2] 2013-06-10 14:07:54,061 [DEBUG] koji.repo.manager: Current tags: [2] 2013-06-10 14:07:54,061 [DEBUG] koji.repo.manager: Skipped check for repos: set([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]) 2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: order: [] 2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: Tracking 15 repos, 0 child processes 2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: Tracking task 17 for tag 2 2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: Scanning filesystem for repos When the above starts, I see the following the the kojid.log: 2013-06-10 14:10:39,768 [INFO] koji.TaskManager: pids: {17: 23421, 18: 23422} 2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting': None, 'id': 18, 'weight': 1.5} 2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting': True, 'id': 17, 'weight': 0.1} When I enter the “koji regen-repo f17-build” command, an additional task is created (newRepo with a child task of createrepo) and the kojid.log changes to: 2013-06-10 14:10:39,768 [INFO] koji.TaskManager: pids: {17: 23421, 18: 23422, 19: 23489, 20: 23492} 2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting': None, 'id': 18, 'weight': 1.5} 2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting': True, 'id': 17, 'weight': 0.1} 2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting': True, 'id': 19, 'weight': 0.1} 2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting': None, 'id': 20, 'weight': 1.5} And the system stays in this mode, never completing any of the tasks. I’ve let them run for as long and 72 hours and they just don’t complete. The httpd/error_log doesn't show any errors, it is stuck in a loop also and attached as a .gz file. I think everything looks good in all three logs, just can’t figure out why it doesn’t complete. If anyone has any suggestions, they would be greatly appreciated. Thanks in advance, Steve

2 1

Odd koji 1.8.0 error - krbLogin being called on my SSL auth system
by Pat Riehecky 18 Jun '13

18 Jun '13

I've configured my system for SSL auth (at least everywhere I saw to set SSL auth) but my apache error log is full of entries like this one: [Sat Jun 15 03:17:01 2013] [error] 2013-06-15 03:17:01,623 [WARNING] m=krbLogin u=None p=19135 r=<MY IP ADDRESS>:45156 koji.xmlrpc: Traceback (most recent call last): [Sat Jun 15 03:17:01 2013] [error] File "/usr/share/koji-hub/kojixmlrpc.py", line 212, in _wrap_handler [Sat Jun 15 03:17:01 2013] [error] response = handler(environ) [Sat Jun 15 03:17:01 2013] [error] File "/usr/share/koji-hub/kojixmlrpc.py", line 255, in handle_rpc [Sat Jun 15 03:17:01 2013] [error] return self._dispatch(method, params) [Sat Jun 15 03:17:01 2013] [error] File "/usr/share/koji-hub/kojixmlrpc.py", line 292, in _dispatch [Sat Jun 15 03:17:01 2013] [error] ret = koji.util.call_with_argcheck(func, params, opts) [Sat Jun 15 03:17:01 2013] [error] File "/usr/lib/python2.6/site-packages/koji/util.py", line 139, in call_with_argcheck [Sat Jun 15 03:17:01 2013] [error] return func(*args, **kwargs) [Sat Jun 15 03:17:01 2013] [error] File "/usr/lib/python2.6/site-packages/koji/auth.py", line 669, in krbLogin [Sat Jun 15 03:17:01 2013] [error] return context.session.krbLogin(*args, **opts) [Sat Jun 15 03:17:01 2013] [error] File "/usr/lib/python2.6/site-packages/koji/auth.py", line 296, in krbLogin [Sat Jun 15 03:17:01 2013] [error] raise koji.AuthError, 'not configured for Kerberos authentication' [Sat Jun 15 03:17:01 2013] [error] AuthError: not configured for Kerberos authentication Any idea what is wrong? -- Pat Riehecky Scientific Linux developer http://www.scientificlinux.org/

2 1

task createrepo do nothing
by Fabrice Cantos 06 Jun '13

06 Jun '13

Hi, I have setup my own koji build system . I have the koji hub on a separate server that the koji builder. However every time that I tried to build a package, the task createrepo never completed. -bash-3.2$ koji list-tasks 2259 20 fabrice OPEN noarch build [sat-1.fabrice.co.nz] 2260 19 fabrice FREE noarch +waitrepo 2257 15 kojira OPEN noarch newRepo [sat-1.fabrice.co.nz] 2258 14 kojira OPEN noarch +createrepo [sat-1.fabrice.co.nz] -bash-3.2$ koji list-hosts --channel=createrepo Hostname Enb Rdy Load/Cap Arches Last Update fc-test.fabrice.co.nz Y N 0.0/4.0 x86_64,i386,i686 2013-05-20 22:57:17 sat-1.fabrice.co.nz Y Y 1.5/8.0 i386,x86_64,i686 2013-05-21 02:38:23 I don't know if it is relevant but on the kojid log I have this warning : " [INFO] koji: Try #5 for call 2 (rawUpload) failed: global name 'handler' is not defined" File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1886, in _callMethod return self._sendCall(handler, headers, request) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1797, in _sendCall return self._sendOneCall(handler, headers, request) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1819, in _sendOneCall ret = self._read_xmlrpc_response(response) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1850, in _read_xmlrpc_response raise xmlrpclib.ProtocolError(self._host + handler, NameError: global name 'handler' is not defined Thanks in advance -- Please consider the environment before printing this email. NIWA is the trading name of the National Institute of Water & Atmospheric Research Ltd.

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys June 2013