[koji PATCH] allow setting build priority
by Dan Horák
When building for multiple tags it could be useful to set different priorities
for the tasks, eg. rawhide builds will be lower prio than branched builds.
---
util/koji-shadow | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/util/koji-shadow b/util/koji-shadow
index 25492d5..8790cf8 100755
--- a/util/koji-shadow
+++ b/util/koji-shadow
@@ -140,6 +140,8 @@ def get_options():
help=_("tag sucessful builds into the tag we are building, default is to not tag"))
parser.add_option("--arches",
help=_("arches to use when creating tags"))
+ parser.add_option("--priority", type="int", default=5,
+ help=_("priority to set for submitted builds"))
#parse once to get the config file
(options, args) = parser.parse_args()
@@ -1080,7 +1082,7 @@ class BuildTracker(object):
#TODO ...and verify repo
repo_id, event_id = session.getTaskResult(task_id)
#kick off build
- task_id = session.build(src, None, opts={'repo_id': repo_id}, priority=5 )
+ task_id = session.build(src, None, opts={'repo_id': repo_id}, priority=options.priority )
return task_id
def report(self):
--
1.8.1.4
9 years, 9 months
[koji PATCH] allow setting the timeout for builds from config file
by Dan Horák
---
builder/kojid | 5 +++--
builder/kojid.conf | 3 +++
koji/__init__.py | 2 +-
3 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/builder/kojid b/builder/kojid
index f906d0b..18c1fa4 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -204,7 +204,7 @@ class BuildRoot(object):
for k in ('repoid', 'tag_name'):
if hasattr(self, k):
opts[k] = getattr(self, k)
- for k in ('mockdir', 'topdir', 'topurl', 'topurls', 'packager', 'vendor', 'distribution', 'mockhost', 'yum_proxy'):
+ for k in ('mockdir', 'topdir', 'topurl', 'topurls', 'packager', 'vendor', 'distribution', 'mockhost', 'yum_proxy', 'rpmbuild_timeout'):
if hasattr(self.options, k):
opts[k] = getattr(self.options, k)
opts['buildroot_id'] = self.id
@@ -3060,6 +3060,7 @@ def get_options():
'maven_repo_ignore': '*.md5 *.sha1 maven-metadata*.xml _maven.repositories '
'resolver-status.properties *.lastUpdated',
'failed_buildroot_lifetime' : 3600 * 4,
+ 'rpmbuild_timeout' : 3600 * 24,
'cert': '/etc/kojid/client.crt',
'ca': '/etc/kojid/clientca.crt',
'serverca': '/etc/kojid/serverca.crt'}
@@ -3067,7 +3068,7 @@ def get_options():
for name, value in config.items('kojid'):
if name in ['sleeptime', 'maxjobs', 'minspace', 'retry_interval',
'max_retries', 'offline_retry_interval', 'failed_buildroot_lifetime',
- 'timeout',]:
+ 'timeout', 'rpmbuild_timeout',]:
try:
defaults[name] = int(value)
except ValueError:
diff --git a/builder/kojid.conf b/builder/kojid.conf
index 1fa9ad4..05fe3b0 100644
--- a/builder/kojid.conf
+++ b/builder/kojid.conf
@@ -32,6 +32,9 @@
; The _host string to use in mock
; mockhost=koji-linux-gnu
+; Timeout for build duration (24 hours)
+; rpmbuild_timeout=86400
+
; The URL for the xmlrpc server
server=http://hub.example.com/kojihub
diff --git a/koji/__init__.py b/koji/__init__.py
index f42226b..8ab81a8 100644
--- a/koji/__init__.py
+++ b/koji/__init__.py
@@ -1266,7 +1266,7 @@ def genMockConfig(name, arch, managed=False, repoid=None, tag_name=None, **opts)
# don't encourage network access from the chroot
'use_host_resolv': opts.get('use_host_resolv', False),
# Don't let a build last more than 24 hours
- 'rpmbuild_timeout': 86400
+ 'rpmbuild_timeout': opts.get('rpmbuild_timeout', 86400)
}
# bind_opts are used to mount parts (or all of) /dev if needed.
--
1.7.7.6
9 years, 9 months
Koji-1.7.1-1.fc17 repository toplink failure
by Anthony Joseph Messina
After upgrading to koji-1.7.1 on my private hub and builders, my builders are
no longer able to use the repositories they create when building subsequent
dependent packages. When attempting to install locally built packages, the
following error is issued in the root.log:
error was [Errno 2] Local file does not exist:
/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
The repos are created successfully:
$ /usr/bin/createrepo -vd -o /tmp/koji/tasks/9129/19129/repo -i
/mnt/koji/repos/f17-build/3627/x86_64/pkglist -g /mnt/koji/repos/f17-
build/3627/groups/comps.xml --update --skip-stat /mnt/koji/repos/f17-
build/3627/x86_64
Scanning old repo data
Spawning worker 0 with 36 pkgs
Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi-
tools-2.6.1-3.fc17.1.x86_64.rpm
Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi-
tools-devel-2.6.1-3.fc17.1.x86_64.rpm
Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi-
tools-libs-2.6.1-3.fc17.1.x86_64.rpm
...
...
I believe the issue is related to the following commit:
https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aac...
I'm not sure why when building packages, the builders are inserting
'/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink'
instead of topdir or topurl, such as
'/mnt/koji' or 'http://example.com/kojifiles'
Right now I'm working on selectively reverting that commit to see if I can
build packages against other local packages.
Any assistance is appreciated. Thanks. -Anthony
--
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E
10 years, 3 months
New Koji Install, createrepo never completes.
by Stephen Brasher
I’ve installed all the pieces of koji-1.8.0 (hub, web, kojira kojid),
configured for ssl and have everything working (I think). Following the
Server Bootstrap and I import all the SRPM’s and RPM’s for fedora fc17
(approximately 35K rpms). Can see all the packages imported on the web in
the Packages and the Builds tabs.
When I execute the “koji add-group f17-build build” and the “koji
add-group-pkg f17-build build pkg1”, kojira starts two tasks, a newRepo
task and a createrepo task. Looking at the kojira.log, I see the following
repeated about every 15 seconds:
2013-06-10 14:07:53,790 [DEBUG] koji.repo.manager: Task 17 is waiting
2013-06-10 14:07:53,876 [DEBUG] koji.repo.manager: Untracked task 19 is
waiting
2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Current tasks: {2: 17}
2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Found 1 untracked
newRepo tasks
2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Updating repos
2013-06-10 14:07:53,877 [DEBUG] koji.repo.manager: Reading current repo data
2013-06-10 14:07:53,977 [DEBUG] koji.repo.manager: Repo data:
[{'create_event': 34851, 'state': 2, 'tag_id': 2, 'tag_name': 'f17-build',
'create_ts': 1370716632.03544, 'id': 1}, {'create_event': 34888, 'state':
2, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370717056.06374,
'id': 2}, {'create_event': 34891, 'state': 0, 'tag_id': 2, 'tag_name':
'f17-build', 'create_ts': 1370882529.75254, 'id': 3}, {'create_event':
34892, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts':
1370890996.61094, 'id': 4}, {'create_event': 34893, 'state': 0, 'tag_id':
2, 'tag_name': 'f17-build', 'create_ts': 1370891441.92783, 'id': 5},
{'create_event': 34894, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build',
'create_ts': 1370891585.94049, 'id': 6}, {'create_event': 34895, 'state':
0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370892392.69252,
'id': 7}, {'create_event': 34896, 'state': 0, 'tag_id': 2, 'tag_name':
'f17-build', 'create_ts': 1370892439.9626, 'id': 8}, {'create_event':
34897, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts':
1370892487.90491, 'id': 9}, {'create_event': 34898, 'state': 0, 'tag_id':
2, 'tag_name': 'f17-build', 'create_ts': 1370892548.62247, 'id': 10},
{'create_event': 34899, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build',
'create_ts': 1370892581.09955, 'id': 11}, {'create_event': 34900, 'state':
0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts': 1370895682.22075,
'id': 12}, {'create_event': 34901, 'state': 0, 'tag_id': 2, 'tag_name':
'f17-build', 'create_ts': 1370895683.00858, 'id': 13}, {'create_event':
34905, 'state': 0, 'tag_id': 2, 'tag_name': 'f17-build', 'create_ts':
1370895884.96146, 'id': 14}, {'create_event': 34931, 'state': 0, 'tag_id':
2, 'tag_name': 'f17-build', 'create_ts': 1370896102.89165, 'id': 15}]
2013-06-10 14:07:54,061 [DEBUG] koji.repo.manager: Needed tags: [2]
2013-06-10 14:07:54,061 [DEBUG] koji.repo.manager: Current tags: [2]
2013-06-10 14:07:54,061 [DEBUG] koji.repo.manager: Skipped check for repos:
set([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: order: []
2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: Tracking 15 repos, 0
child processes
2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: Tracking task 17 for tag
2
2013-06-10 14:07:54,157 [DEBUG] koji.repo.manager: Scanning filesystem for
repos
When the above starts, I see the following the the kojid.log:
2013-06-10 14:10:39,768 [INFO] koji.TaskManager: pids: {17: 23421, 18:
23422}
2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting':
None, 'id': 18, 'weight': 1.5}
2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting':
True, 'id': 17, 'weight': 0.1}
When I enter the “koji regen-repo f17-build” command, an additional task is
created (newRepo with a child task of createrepo) and the kojid.log changes
to:
2013-06-10 14:10:39,768 [INFO] koji.TaskManager: pids: {17: 23421, 18:
23422, 19: 23489, 20: 23492}
2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting':
None, 'id': 18, 'weight': 1.5}
2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting':
True, 'id': 17, 'weight': 0.1}
2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting':
True, 'id': 19, 'weight': 0.1}
2013-06-10 14:10:39,854 [INFO] koji.TaskManager: open task: {'waiting':
None, 'id': 20, 'weight': 1.5}
And the system stays in this mode, never completing any of the tasks. I’ve
let them run for as long and 72 hours and they just don’t complete.
The httpd/error_log doesn't show any errors, it is stuck in a loop also and
attached as a .gz file.
I think everything looks good in all three logs, just can’t figure out why
it doesn’t complete. If anyone has any suggestions, they would be greatly
appreciated.
Thanks in advance,
Steve
10 years, 3 months
Odd koji 1.8.0 error - krbLogin being called on my SSL auth system
by Pat Riehecky
I've configured my system for SSL auth (at least everywhere I saw to set SSL
auth) but my apache error log is full of entries like this one:
[Sat Jun 15 03:17:01 2013] [error] 2013-06-15 03:17:01,623 [WARNING]
m=krbLogin u=None p=19135 r=<MY IP ADDRESS>:45156 koji.xmlrpc: Traceback (most
recent call last):
[Sat Jun 15 03:17:01 2013] [error] File "/usr/share/koji-hub/kojixmlrpc.py",
line 212, in _wrap_handler
[Sat Jun 15 03:17:01 2013] [error] response = handler(environ)
[Sat Jun 15 03:17:01 2013] [error] File "/usr/share/koji-hub/kojixmlrpc.py",
line 255, in handle_rpc
[Sat Jun 15 03:17:01 2013] [error] return self._dispatch(method, params)
[Sat Jun 15 03:17:01 2013] [error] File "/usr/share/koji-hub/kojixmlrpc.py",
line 292, in _dispatch
[Sat Jun 15 03:17:01 2013] [error] ret =
koji.util.call_with_argcheck(func, params, opts)
[Sat Jun 15 03:17:01 2013] [error] File
"/usr/lib/python2.6/site-packages/koji/util.py", line 139, in call_with_argcheck
[Sat Jun 15 03:17:01 2013] [error] return func(*args, **kwargs)
[Sat Jun 15 03:17:01 2013] [error] File
"/usr/lib/python2.6/site-packages/koji/auth.py", line 669, in krbLogin
[Sat Jun 15 03:17:01 2013] [error] return context.session.krbLogin(*args,
**opts)
[Sat Jun 15 03:17:01 2013] [error] File
"/usr/lib/python2.6/site-packages/koji/auth.py", line 296, in krbLogin
[Sat Jun 15 03:17:01 2013] [error] raise koji.AuthError, 'not configured
for Kerberos authentication'
[Sat Jun 15 03:17:01 2013] [error] AuthError: not configured for Kerberos
authentication
Any idea what is wrong?
--
Pat Riehecky
Scientific Linux developer
http://www.scientificlinux.org/
10 years, 3 months
task createrepo do nothing
by Fabrice Cantos
Hi,
I have setup my own koji build system .
I have the koji hub on a separate server that the koji builder.
However every time that I tried to build a package, the task createrepo
never completed.
-bash-3.2$ koji list-tasks
2259 20 fabrice OPEN noarch build [sat-1.fabrice.co.nz]
2260 19 fabrice FREE noarch +waitrepo
2257 15 kojira OPEN noarch newRepo [sat-1.fabrice.co.nz]
2258 14 kojira OPEN noarch +createrepo
[sat-1.fabrice.co.nz]
-bash-3.2$ koji list-hosts --channel=createrepo
Hostname Enb Rdy Load/Cap Arches Last Update
fc-test.fabrice.co.nz Y N 0.0/4.0 x86_64,i386,i686 2013-05-20
22:57:17
sat-1.fabrice.co.nz Y Y 1.5/8.0 i386,x86_64,i686 2013-05-21
02:38:23
I don't know if it is relevant but on the kojid log I have this warning :
" [INFO] koji: Try #5 for call 2 (rawUpload) failed: global name
'handler' is not defined"
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1886,
in _callMethod
return self._sendCall(handler, headers, request)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1797,
in _sendCall
return self._sendOneCall(handler, headers, request)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1819,
in _sendOneCall
ret = self._read_xmlrpc_response(response)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1850,
in _read_xmlrpc_response
raise xmlrpclib.ProtocolError(self._host + handler,
NameError: global name 'handler' is not defined
Thanks in advance
--
Please consider the environment before printing this email.
NIWA is the trading name of the National Institute of Water & Atmospheric Research Ltd.
10 years, 3 months