[BUG] Koji and Lustre
by Thomas Guthmann
Hey guys,
We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py)
when used with a Lustre filesystem. This function parses all attributes
of every file of a build and is used when you want to display
build.log/root.log through the web interface. Everything returned by
listTaskOutput is returned through XML-RPC and as a result we had this :
An error has occurred while processing your request.
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
Traceback (most recent call last):
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in
publish_object
return old_publish_object(req, object)
File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py",
line 412, in publish_object
return publish_object(req,util.apply_fs_data(object, req.form,
req=req))
File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line
439, in apply_fs_data
return object(**args)
File "/usr/share/koji-web/scripts/index.py", line 649, in getfile
output = server.listTaskOutput(taskID, stat=True)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468,
in __call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718,
in _callMethod
raise err
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
The issue comes from the st_dev value gathered by getStat (stat). In
Lustre this value can be very high and that's why it complains. To fix
that, we used the same condition than st_size, we cast the value as a
string. See attached patch.
Example (see 'Device:' the value after the '/'):
# stat build.log
File: `build.log'
Size: 106021 Blocks: 208 IO Block: 2097152 regular file
Device: e04ae70eh/3763005198d Inode: 721664 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache)
Access: 2010-07-13 10:44:40.000000000 +1000
Modify: 2010-07-12 12:54:15.000000000 +1000
Change: 2010-07-12 12:54:52.000000000 +1000
# That's what is read by listTaskOutput
relfilename=build.log ATTR=st_atime GETATTR=1278981616
relfilename=build.log ATTR=st_blksize GETATTR=2097152
relfilename=build.log ATTR=st_blocks GETATTR=208
relfilename=build.log ATTR=st_ctime GETATTR=1278903292
relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\
relfilename=build.log ATTR=st_gid GETATTR=48
relfilename=build.log ATTR=st_ino GETATTR=721664
relfilename=build.log ATTR=st_mode GETATTR=33188
relfilename=build.log ATTR=st_mtime GETATTR=1278903255
relfilename=build.log ATTR=st_nlink GETATTR=1
relfilename=build.log ATTR=st_rdev GETATTR=0
relfilename=build.log ATTR=st_size GETATTR=106021
relfilename=build.log ATTR=st_uid GETATTR=48
Hope it helps, lost a good amount of time on that one :)
Cheers,
Thomas
12 years, 8 months
koji performance
by Florian La Roche
Hello,
I've setup koji and a second build server on a KVM host and played
with different VM settings. One item I noticed with koji is
that it gets really slow with small source rpms just to handel all
data and from the log files it seems like uploadFile() is very slow.
Has anyone debugged koji performance in more detail or is anyone
looking into speeding up uploadFile()?
best regards,
Florian La Roche
13 years, 1 month
Koji Setup Documentation - Fedora Wiki
by Todd Edward Westacott
Recently as part of a project at Seneca College I have been going through the process of setting up a complete Koji Build System to test the quality and the accuracy of the existing documentation on the Fedora Wiki
For the most part the information on the wiki is accurate and the instructions will work if you follow them through step by step. The issue that I found is that the there is not enough information which explains how the different components work together and what each step of the instructions is actually doing.
I would also like to invite anyone who is interested to review the edits I have made to the existing documentation on the Fedora Wiki, specifically the How To section which walks through setting up your own system:
Koji/ServerHowTo – Fedora Wiki
Over the next week or two I will be taking a look at the other sections of the Koji documentation to see what information could be clarified as well as making an additional changes which might come to my attention with the Koji setup process. Please feel free to contact me with any information you might like to share or suggestions you might like to make. Your feedback would be very valuable to me and greatly appreciated.
13 years, 1 month
arguments for koji api "createEmptyBuild"
by Eric Zhong
koji api "createEmptyBuild" arguments problems:
[zhongwenjia@build ~]$ koji call createEmptyBuild nvidia 1 1.3 ""
Fault: <Fault 1: "<class 'pg.DatabaseError'>: error 'ERROR: operator does
not exist: text = integer\nLINE 2: WHERE pkg_id=9168 AND version=1 AND
release='1.3'\n ^\nHINT: No
operator matches the given name and argument type(s). You might need to add
explicit type casts.\n' in 'SELECT id,state,task_id FROM build\n WHERE
pkg_id=9168 AND version=1 AND release='1.3'\n FOR UPDATE'">
[zhongwenjia@build ~]$ koji call createEmptyBuild nvidia 1.1 1.3 ""
Fault: <Fault 1: '<class \'pg.DatabaseError\'>: error \'ERROR: invalid
input syntax for integer: ""\nLINE 4: VALUES
(18338,9169,\'1.1\',\'1.3\',\'\',\n
^\n\' in \'\n INSERT INTO build
(id,pkg_id,version,release,epoch,state,\n
task_id,owner,completion_time)\n VALUES
(18338,9169,\'1.1\',\'1.3\',\'\',\n 1,NULL,1,\'NOW\')\n \''>
[zhongwenjia@build ~]$ koji call createEmptyBuild nvidia 1.1 1.3 2
18339
[zhongwenjia@build ~]$ koji call createEmptyBuild nvidia 1.1 1.3
Fault: <Fault 1: "<type 'exceptions.TypeError'>: createEmptyBuild() takes at
least 5 arguments (4 given)">
if version is a number , not string , how to do it ?
what is epoch argument ? Default is NULL. How to give a NULL value ?
koji-hub-1.3.2-1.fc13.noarch
Thanks
13 years, 1 month
controlling tty issues in mock
by Clark Williams
All,
Mock has a problem setting up /dev/tty and /dev/ptmx in our build
chroots. In a standard /dev setup on EL{5,6} and all our current
Fedora's, /dev/tty is a character device with major,minor == 5,0,
while /dev/ptmx is a character device with major,minor == 5,2. Both are
owned by root, group tty, mode 0666. It would follow that our chroots
should be setup the same way, but we're seeing test suite failures
inside the chroot where the build process hangs.
An early attempt to fix this was where we symlinked /dev/tty
and /dev/ptmx to /dev/pts/ptmx (making them pty instances) but that
caused other failures. I have a suspicion that we need to do something
special to setup the controlling terminal in the chroot, I just don't
know what that "special something" is.
The SRPMs that have proven troublesome are:
perl-TermReadKey-2.30-9.fc13.src.rpm
bash-4.1.2-5.el6.src.rpm
Conn-1.0.32-1.src.rpm
libssh2
I did some rough debugging for perl-TermReadKey, where I setup the
chroot with /dev/tty and /dev/ptmx as the character devices they should
be and when trying to build on F14 with the config fedora-14-x86_64,
the build hangs. Attaching gdb to the perl program and getting a
partial backtrace shows that it's hung in tcsetattr() (TERMIOS(3)). I
looked at the generated C code and saw that there are two instances of
tcsetattr() in the code:
if(DisableFlush || oldmode<=mode)
tcsetattr(handle,TCSANOW,&work);
else
tcsetattr(handle,TCSAFLUSH,&work);
The C code is generated in the build process, so I wasn't able to easily
add any debug statements to this code and couldn't get the debuginfo
into gdb to refine the backtrace, but my guess is that we're hanging on
the TCSAFLUSH version.
Anyone have any ideas on this?
Clark
13 years, 1 month
How to use "koji call host.*" api ?
by Eric Zhong
How to use host.* API ???
How to specify the "host" ???
[zhongwenjia@localhost ~]$ koji call host.getHost
AuthError: No host specified
13 years, 1 month
How to reuse a failed NVR?
by Eric Zhong
I use " koji call resetBuild" to clean a failed build , then i import the
SRPM&RPMS directly.
SRPM be imported, and when import RPMS, it gives "Error importing :
Build is CANCELED"
Cant I reuse a failed NVR ?
13 years, 1 month
