Koji client tools python3 compatible
by Adam Miller
It was recently brought up to me that with the impending "Python3 as
Default"[0] change coming to Fedora it would be nice if we had Python3
client-side compatibility. The main motivation is from a developer
(Fedora Workstation User) perspective, it would be nice if the didn't
need to pull in all of the python2 stack just to be a Fedora Packager.
I don't know how big of an overhaul this would be or what it would
potentially break on the back end for the ability to still run server
side bits on EL5 machines, but I wanted to at least initiate the
discussion.
Thank you,
-AdamM
[0] - https://fedoraproject.org/wiki/Changes/Python_3_as_Default
8 years, 5 months
Extra options for tags and dnf support
by Mike McLean
With mock switching to dnf, we needed to conditionally adjust mock's
package_manager config on a per-tag basis. I didn't want to add yet another
specialized field in the tag_config table, so I extended the schema to
support more flexible extra options for tags.
This set of patches adds the extra options functionality and uses it to control
set mock's package_manager option.
Each tag can now be assigned arbitrary extra options. These are stored as json
in the database, so they can be of any type that json supports. The
getBuildConfig call follows inheritance when determining these.
To tell mock to use dnf in a tag, you would run a command like:
# koji edit-tag TAG -x mock.package_manager=dnf
This setting is inherited.
8 years, 5 months
[PATCH] download-logs: print a message when there are no logs to download
by Mike McLean
see: https://bugzilla.redhat.com/show_bug.cgi?id=1232916
---
cli/koji | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/cli/koji b/cli/koji
index 908d829..9ec098d 100755
--- a/cli/koji
+++ b/cli/koji
@@ -6229,15 +6229,21 @@ def anon_handle_download_logs(options, session, args):
task_log_dir = os.path.join(parent_dir,
"%s-%s" % (task_info["arch"], task_id))
+ count = 0
state = koji.TASK_STATES[task_info['state']]
if state == 'FAILED':
if not match or koji.util.multi_fnmatch(FAIL_LOG, match):
write_fail_log(task_log_dir, task_id)
+ count += 1
elif state not in ['CLOSED', 'CANCELED']:
sys.stderr.write(_("Warning: task %s is %s\n") % (task_id, state))
for log_filename in logs:
download_log(task_log_dir, task_id, log_filename)
+ count += 1
+
+ if count == 0 and not recurse:
+ sys.stderr.write(_("No logs found for task %i. Perhaps try --recurse?\n") % task_id)
if recurse:
child_tasks = session.getTaskChildren(task_id)
--
1.9.3
8 years, 5 months
[PATCH] include lib64 dir in default pluginpath
by Mike McLean
---
builder/kojid | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/builder/kojid b/builder/kojid
index b6b79c1..8243434 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -4460,7 +4460,7 @@ def get_options():
'topdir': '/mnt/koji',
'topurl': None,
'workdir': '/var/tmp/koji',
- 'pluginpath': '/usr/lib/koji-builder-plugins',
+ 'pluginpath': '/usr/lib/koji-builder-plugins:/usr/lib64/koji-builder-plugins',
'mockdir': '/var/lib/mock',
'mockuser': 'kojibuilder',
'packager': 'Koji',
--
1.9.3
8 years, 5 months
[PATCH] capture mock output even if it exits immediately
by Mike McLean
---
builder/kojid | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/builder/kojid b/builder/kojid
index 8243434..2e1add3 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -374,7 +374,7 @@ class BuildRoot(object):
results = os.listdir(resultdir)
except OSError:
# will happen when mock hasn't created the resultdir yet
- continue
+ results = []
for fname in results:
if fname.endswith('.log') and not logs.has_key(fname):
--
1.9.3
8 years, 5 months
[PATCH] inject the url of the install tree into the kickstart used for generating images
by Brian Stinson
From: Brian Stinson <bstinson(a)redhat.com>
Date: Sat, 6 Jun 2015 16:43:40 -0500
---
builder/kojid | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/builder/kojid b/builder/kojid
index ad2cf1f..b6b79c1 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -2841,7 +2841,7 @@ class OzImageTask(BaseTaskHandler):
"'%s' : %s" % (kspath, e))
return ks
- def prepareKickstart(self, kspath):
+ def prepareKickstart(self, kspath, install_tree):
"""
Process the ks file to be used for controlled image generation. This
method also uploads the modified kickstart file to the task output
@@ -2878,6 +2878,8 @@ class OzImageTask(BaseTaskHandler):
self.logger.debug('BASEURL: %s' % baseurl)
ks.handler.repo.repoList.append(repo_class(
baseurl=baseurl, name='koji-override-0'))
+ # inject the URL of the install tree into the kickstart
+ ks.handler.url.url = install_tree
return ks
def writeKickstart(self, ksobj, ksname):
@@ -3327,7 +3329,7 @@ class BaseImageTask(OzImageTask):
# First, prepare the kickstart to use the repos we tell it
kspath = self.fetchKickstart()
- ks = self.prepareKickstart(kspath)
+ ks = self.prepareKickstart(kspath, inst_tree)
kskoji = self.writeKickstart(ks,
os.path.join(self.workdir, 'koji-%s-%i-base.ks' %
(self.target_info['build_tag_name'], self.id)))
--
1.8.3.1
8 years, 6 months
[PATCH] only use PlgHTTPSConnection for ssl_login
by Mike McLean
---
koji/__init__.py | 25 ++++++++++---------------
1 file changed, 10 insertions(+), 15 deletions(-)
diff --git a/koji/__init__.py b/koji/__init__.py
index 58971da..36772bb 100644
--- a/koji/__init__.py
+++ b/koji/__init__.py
@@ -1599,12 +1599,7 @@ class ClientSession(object):
self.explicit_port = bool(_port)
self._path = uri[2]
default_port = 80
- if self.opts.get('certs'):
- ctx = ssl.SSLCommon.CreateSSLContext(self.opts['certs'])
- cnxOpts = {'ssl_context' : ctx}
- cnxClass = ssl.SSLCommon.PlgHTTPSConnection
- default_port = 443
- elif scheme == 'https':
+ if scheme == 'https':
cnxOpts = {}
cnxClass = httplib.HTTPSConnection
default_port = 443
@@ -1737,21 +1732,21 @@ class ClientSession(object):
certs['ca_cert'] = ca
certs['peer_ca_cert'] = serverca
- ctx = ssl.SSLCommon.CreateSSLContext(certs)
- self._cnxOpts = {'ssl_context' : ctx}
- # 60 second timeout during login
- old_timeout = self._cnxOpts.get('timeout')
- self._cnxOpts['timeout'] = 60
+ # for ssl authentication we use custom a connection class so
+ # that we can pass our certs
try:
+ old_cnx = (self._cnxClass, self._cnxOpts)
+ ctx = ssl.SSLCommon.CreateSSLContext(certs)
+ self._cnxOpts = {'ssl_context' : ctx}
+ # 60 second timeout during login
+ self._cnxOpts['timeout'] = 60
self._cnxClass = ssl.SSLCommon.PlgHTTPSConnection
if self._port == 80 and not self.explicit_port:
self._port = 443
sinfo = self.callMethod('sslLogin', proxyuser)
finally:
- if old_timeout is None:
- del self._cnxOpts['timeout']
- else:
- self._cnxOpts['timeout'] = old_timeout
+ # restore original connection type
+ self._cnxClass, self._cnxOpts = old_cnx
if not sinfo:
raise AuthError, 'unable to obtain a session'
--
1.9.3
8 years, 6 months
[PATCH 1/2] logfile config option in kojid/kojivmd/kojira
by Mike McLean
---
builder/kojid | 8 +++++++-
util/kojira | 2 +-
vm/kojivmd | 8 +++++++-
3 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/builder/kojid b/builder/kojid
index dcd0840..49c1510 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -4411,6 +4411,7 @@ def get_options():
parser.add_option("--debug-mock", action="store_true", default=False,
#obsolete option
help=SUPPRESS_HELP)
+ parser.add_option("--logfile", help="specify log file")
parser.add_option("--skip-main", action="store_true", default=False,
help="don't actually run main")
parser.add_option("--maxjobs", type='int', help="Specify maxjobs")
@@ -4445,6 +4446,7 @@ def get_options():
'minspace': 8192,
'admin_emails': None,
'log_level': None,
+ 'logfile': '/var/log/kojid.log',
'topdir': '/mnt/koji',
'topurl': None,
'workdir': '/var/tmp/koji',
@@ -4509,6 +4511,9 @@ def get_options():
if getattr(options, name, None) is None:
setattr(options, name, value)
+ if options.logfile in ('','None','none'):
+ options.logfile = None
+
#honor topdir
if options.topdir:
koji.BASEDIR = options.topdir
@@ -4549,7 +4554,8 @@ def quit(msg=None, code=1):
sys.exit(code)
if __name__ == "__main__":
- koji.add_file_logger("koji", "/var/log/kojid.log")
+ if options.logfile:
+ koji.add_file_logger("koji", options.logfile)
#note we're setting logging params for all of koji*
options = get_options()
if options.log_level:
diff --git a/util/kojira b/util/kojira
index c18f63b..86e6b80 100755
--- a/util/kojira
+++ b/util/kojira
@@ -779,7 +779,7 @@ if __name__ == "__main__":
if not os.access(options.logfile,os.W_OK):
sys.stderr.write("Cannot write to logfile: %s\n" % options.logfile)
sys.exit(1)
- koji.add_file_logger("koji", options.logfile)
+ koji.add_file_logger("koji", options.logfile)
#note we're setting logging for koji.*
logger = logging.getLogger("koji")
if options.debug:
diff --git a/vm/kojivmd b/vm/kojivmd
index cf44f0c..fa79e58 100755
--- a/vm/kojivmd
+++ b/vm/kojivmd
@@ -82,6 +82,7 @@ def get_options():
help="show xmlrpc debug output")
parser.add_option("--skip-main", action="store_true", default=False,
help="don't actually run main")
+ parser.add_option("--logfile", help="specify log file")
parser.add_option("--maxjobs", type='int', help="Specify maxjobs")
parser.add_option("--sleeptime", type='int', help="Specify the polling interval")
parser.add_option("--admin-emails", help="Address(es) to send error notices to")
@@ -108,6 +109,7 @@ def get_options():
'minmem': 4096,
'vmuser': 'qemu',
'admin_emails': None,
+ 'logfile': '/var/log/kojivmd.log',
'workdir': '/tmp/koji',
'topurl': '',
'imagedir': '/var/lib/libvirt/images',
@@ -153,6 +155,9 @@ def get_options():
if getattr(options, name, None) is None:
setattr(options, name, value)
+ if options.logfile in ('','None','none'):
+ options.logfile = None
+
#make sure workdir exists
if not os.path.exists(options.workdir):
koji.ensuredir(options.workdir)
@@ -1040,7 +1045,8 @@ class VMTaskManager(TaskManager):
####################
if __name__ == "__main__":
- koji.add_file_logger("koji", "/var/log/kojivmd.log")
+ if options.logfile:
+ koji.add_file_logger("koji", options.logfile)
#note we're setting logging params for all of koji*
options = get_options()
if options.debug:
--
1.9.3
8 years, 6 months
[PATCH 1/4] schema: add missing image channel from the default schema
by Dennis Gilmore
Signed-off-by: Dennis Gilmore <dennis(a)ausil.us>
---
docs/schema.sql | 1 +
1 file changed, 1 insertion(+)
diff --git a/docs/schema.sql b/docs/schema.sql
index 3582b87..aad24a9 100644
--- a/docs/schema.sql
+++ b/docs/schema.sql
@@ -183,6 +183,7 @@ INSERT INTO channels (name) VALUES ('maven');
INSERT INTO channels (name) VALUES ('livecd');
INSERT INTO channels (name) VALUES ('appliance');
INSERT INTO channels (name) VALUES ('vm');
+INSERT INTO channels (name) VALUES ('image');
-- Here we track the build machines
-- each host has an entry in the users table also
--
2.4.2
8 years, 6 months
Koji content generators
by Ken Dreyer
Hi Mike,
For the Ceph project, we're very interested in building Debian
packages on Red Hat's build system, so I've been kicking around the
idea of a Koji buildDSCFromSCM task. When I mentioned this to one of
the rel-eng guys in Red Hat, they mentioned that "content generators"
might be able to handle our use-case in Ceph. This was the first time
I'd heard of such a thing.
In your Koji 2.0 planning thread you mentioned "content generators" as
well. Can you provide more details about this?
- Ken
8 years, 6 months
