[BUG] Koji and Lustre
by Thomas Guthmann
Hey guys,
We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py)
when used with a Lustre filesystem. This function parses all attributes
of every file of a build and is used when you want to display
build.log/root.log through the web interface. Everything returned by
listTaskOutput is returned through XML-RPC and as a result we had this :
An error has occurred while processing your request.
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
Traceback (most recent call last):
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in
publish_object
return old_publish_object(req, object)
File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py",
line 412, in publish_object
return publish_object(req,util.apply_fs_data(object, req.form,
req=req))
File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line
439, in apply_fs_data
return object(**args)
File "/usr/share/koji-web/scripts/index.py", line 649, in getfile
output = server.listTaskOutput(taskID, stat=True)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468,
in __call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718,
in _callMethod
raise err
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
The issue comes from the st_dev value gathered by getStat (stat). In
Lustre this value can be very high and that's why it complains. To fix
that, we used the same condition than st_size, we cast the value as a
string. See attached patch.
Example (see 'Device:' the value after the '/'):
# stat build.log
File: `build.log'
Size: 106021 Blocks: 208 IO Block: 2097152 regular file
Device: e04ae70eh/3763005198d Inode: 721664 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache)
Access: 2010-07-13 10:44:40.000000000 +1000
Modify: 2010-07-12 12:54:15.000000000 +1000
Change: 2010-07-12 12:54:52.000000000 +1000
# That's what is read by listTaskOutput
relfilename=build.log ATTR=st_atime GETATTR=1278981616
relfilename=build.log ATTR=st_blksize GETATTR=2097152
relfilename=build.log ATTR=st_blocks GETATTR=208
relfilename=build.log ATTR=st_ctime GETATTR=1278903292
relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\
relfilename=build.log ATTR=st_gid GETATTR=48
relfilename=build.log ATTR=st_ino GETATTR=721664
relfilename=build.log ATTR=st_mode GETATTR=33188
relfilename=build.log ATTR=st_mtime GETATTR=1278903255
relfilename=build.log ATTR=st_nlink GETATTR=1
relfilename=build.log ATTR=st_rdev GETATTR=0
relfilename=build.log ATTR=st_size GETATTR=106021
relfilename=build.log ATTR=st_uid GETATTR=48
Hope it helps, lost a good amount of time on that one :)
Cheers,
Thomas
12 years, 2 months
Strange mock build failure due to typo
by Paul Howarth
Hi,
today I have been preparing an update to perl-Math-Pari and came across
a very strange build failure whilst doing a local mock build on a Fedora
13 x86_64 host. My package built successfully on x86_64 but when I tried
to build for i386, the build failed during %setup but without any
diagnostics. The SRPM was installed but no attempt to install its build
requirements was made. The root.log showed an exit status of 0 for all
commands that had been run.
After much experimentation bisecting the changes I had made, I
discovered that a typo in the changelog entry was the culprit: I had set
the year to 2100 instead of 2010. So it would appear that somewhere in
the mock/yum/rpm stack there may be a year 2038 problem waiting to bite
us (though I suspect there may not be too many 32-bit builds happening
by then).
Seriously though, it would be nice to have better diagnostics for this
and perhaps an rpmlint check for changelog entries in the future?
Paul.
12 years, 10 months
email notifications
by Florian La Roche
Hallo,
With koji-1.4 running on RHEL6-beta2, some email notifications
are not sent out correctly:
Traceback (most recent call last):
File "/usr/sbin/kojid", line 1437, in runTask
response = (handler.run(),)
File "/usr/sbin/kojid", line 1513, in run
return self.handler(*self.params,**self.opts)
File "/usr/sbin/kojid", line 3649, in handler
message = message.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 546: ordinal not in range(128)
Thie seems to happen e.g. on dejavu-fonts, ctags, crontabs,
cronie, amanda, akonadi.
regards,
Florian La Roche
13 years, 2 months
Trouble Upgrading Koji to v1.4.0
by Matthew B Treinish
I have been playing around with a koji 1.3.2 server on my machine for the
past couple of weeks. Today I tried updating the server to v1.4.0 and am
having some difficulty getting the koji-web interface working. Koji-hub,
kojira, kojid, and the koji cli are working fine but koji-web won't run. I
have a mod_python traceback of the error:
Traceback (most recent call last):
File "/usr/lib64/python2.6/site-packages/mod_python/importer.py", line
1537, in HandlerDispatch
default=default_handler, arg=req, silent=hlist.silent)
File "/usr/lib64/python2.6/site-packages/mod_python/importer.py", line
1229, in _process_target
result = _execute_target(config, req, object, arg)
File "/usr/lib64/python2.6/site-packages/mod_python/importer.py", line
1128, in _execute_target
result = object(arg)
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 39, in handler
return mod_python.publisher.handler(req)
File "/usr/lib64/python2.6/site-packages/mod_python/publisher.py", line
213, in handler
published = publish_object(req, object)
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 34, in
publish_object
return old_publish_object(req, _genHTML(req, 'error.chtml'))
File "/usr/share/koji-web/lib/kojiweb/util.py", line 69, in _genHTML
req._values['mavenEnabled'] = req._session.mavenEnabled()
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1468, in
__call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1698, in
_callMethod
return proxy.__getattr__(name)(*args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
File "/usr/lib64/python2.6/xmlrpclib.py", line 1235, in request
self.send_content(h, request_body)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1349, in send_content
connection.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 868, in endheaders
self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 740, in _send_output
self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 699, in send
self.connect()
File "/usr/lib64/python2.6/httplib.py", line 1073, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "/usr/lib64/python2.6/ssl.py", line 350, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "/usr/lib64/python2.6/ssl.py", line 118, in __init__
self.do_handshake()
File "/usr/lib64/python2.6/ssl.py", line 293, in do_handshake
self._sslobj.do_handshake()
SSLError: [Errno 1] _ssl.c:480: error:14094410:SSL
routines:SSL3_READ_BYTES:sslv3 alert handshake failure
Everything was working fine in v1.3.2 so I'm at a loss at what could cause
the SSL certificate to fail after the upgrade. I'd appreciate any insight
into what I'm doing wrong.
-Matthew Treinish
13 years, 4 months
koji-hub requires mod_python
by Connie Sieh
I am trying to install koji-hub 1.4 on RHEL6 Beta 2. I obtained the rpm
from EPEL. koji-hub requires mod_python but mod_python was removed from
the RHEL6 Beta even though it is still in Fedora 13. I am assuming that
mod_python was removed from RHEL6 Beta because mod_wsgi is to replace it.
For my immediate testing needs I can find mod_python on the web.
So the question is will koji be moving over to mod_wsgi or will RHEL 6
include mod_python in the future.
-Connie Sieh
Fermi National Accelerator Laboratory
13 years, 4 months
Re: Building under mock (rawhide)
by Clark Williams
On Sat, 3 Jul 2010 11:46:35 -0500
Dennis Gilmore <dennis(a)ausil.us> wrote:
> yo cant build 64 bit binaries on a 32 bit box. you will need either a 64 bit
> box or to setup a qemu guest thats 64 bit. not the qemu route will be slow as
> the cpu will be fully emulated
>
> The error you get is because the host rpm has excluded the
> unistallable/runable 64 bit binarries so you end up with only installable
> noarch packages
>
> Dennis
I'm thinking we need to catch this early in mock. Any suggestions on
how we should detect that we're on a 32-bit arch and the selected
config is 64 bit? Brute force is just to have a dictionary that rejects
certain configs.
Clark
13 years, 4 months
koji 1.4.0
by Mike McLean
The 1.4.0 release of koji is available here:
https://fedorahosted.org/releases/k/o/koji/
Updating from 1.3.2 involves a number of schema changes. A schema update
script is included (docs/schema-upgrade-1.3-1.4.sql).
The mead branch has been merged. Most the that functionality is disabled
by default. You can enable it by setting EnableMaven to yes in hub.conf.
Summary of changes:
- Merge mead branch: support for building jars with Maven *
- support for building appliance images *
- soft dependencies for LiveCD/Appliance features
- smarter prioritization of repo regenerations
- package list policy to determine if package list changes are allowed
- channel policy to determine which channel a task is placed in
- edit host data via webui
- description and comment fields for hosts *
- cleaner log entries for kojihub
- track user data in versioned tables *
- allow setting retry parameters for the cli
- track start time for tasks *
- allow packages built from the same srpm to span multiple external repos
- make the command used to fetch sources configuable per repo
- kojira: remove unexpected directories
- let kojid to decide if it can handle a noarch task
- avoid extraneous ssl handshakes
- schema changes to support starred items
13 years, 4 months
About spin ISO
by Eric Zhong
I had spin a ISO from FC13 repo , the kickstart below:
%packages --default
anaconda
kernel*
%end
it will install 211 RPMs,but the ISO/Packages/ dir have 512 RPMs.
Why it is not 211 ? Why need the extra RPMs? Can i remove them ?
13 years, 5 months
[PATCH] mock _init _umountall
by Paul B Schroeder
Hello all...
Recently, I was doing some work which needed a more extensive /dev than the mock internal dev setup. So I tried doing this in site-defaults.cfg:
config_opts['internal_dev_setup'] = False
config_opts['plugin_conf']['bind_mount_enable'] = True
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/dev', '/dev' ))
And discovered that mock chroot init was failing due _umountall getting called (in _init) before _makeBuildUser. In _makeBuildUser there is a perl command which, for whatever reason, would fail due to /dev/null not being available at that point.
At any rate, the following patch seems to take care of the issue. And it would seem to be a better way to handle this in general as I would expect _umountall to be the last method called.
It would be awesome if this or something similar could be applied..
Thanks...Paul...
diff --git a/py/mock/backend.py b/py/mock/backend.py
index ca06f59..9df438b 100644
--- a/py/mock/backend.py
+++ b/py/mock/backend.py
@@ -282,24 +282,24 @@ class Root(object):
self._mountall()
if self.chrootWasCleaned:
self._yum(self.chroot_setup_cmd, returnOutput=1)
- finally:
- self._umountall()
- # create user
- self._makeBuildUser()
+ # create user
+ self._makeBuildUser()
- # create rpmbuild dir
- self._buildDirSetup()
+ # create rpmbuild dir
+ self._buildDirSetup()
- # set up timezone to match host
- localtimedir = self.makeChrootPath('etc')
- localtimepath = self.makeChrootPath('etc', 'localtime')
- if os.path.exists(localtimepath):
- os.remove(localtimepath)
- shutil.copy2('/etc/localtime', localtimedir)
+ # set up timezone to match host
+ localtimedir = self.makeChrootPath('etc')
+ localtimepath = self.makeChrootPath('etc', 'localtime')
+ if os.path.exists(localtimepath):
+ os.remove(localtimepath)
+ shutil.copy2('/etc/localtime', localtimedir)
- # done with init
- self._callHooks('postinit')
+ # done with init
+ self._callHooks('postinit')
+ finally:
+ self._umountall()
decorate(traceLog())
def _setupDev(self):
--
---
Paul B Schroeder
<paulbsch "at" vbridges "dot" com>
13 years, 5 months
Remove a package with no builds
by Nathan Blackham
I added a package with a typo to a tag. I now have the bad package
name show up in the package list. There are no builds against it. Is
there an easy way to delete?
I would guess it probably requires a sql query? Anyone have one
handy, to pull it out of all the dependencies?
Thanks,
Nathan
13 years, 5 months
