[BUG] Koji and Lustre
by Thomas Guthmann
Hey guys,
We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py)
when used with a Lustre filesystem. This function parses all attributes
of every file of a build and is used when you want to display
build.log/root.log through the web interface. Everything returned by
listTaskOutput is returned through XML-RPC and as a result we had this :
An error has occurred while processing your request.
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
Traceback (most recent call last):
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in
publish_object
return old_publish_object(req, object)
File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py",
line 412, in publish_object
return publish_object(req,util.apply_fs_data(object, req.form,
req=req))
File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line
439, in apply_fs_data
return object(**args)
File "/usr/share/koji-web/scripts/index.py", line 649, in getfile
output = server.listTaskOutput(taskID, stat=True)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468,
in __call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718,
in _callMethod
raise err
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
The issue comes from the st_dev value gathered by getStat (stat). In
Lustre this value can be very high and that's why it complains. To fix
that, we used the same condition than st_size, we cast the value as a
string. See attached patch.
Example (see 'Device:' the value after the '/'):
# stat build.log
File: `build.log'
Size: 106021 Blocks: 208 IO Block: 2097152 regular file
Device: e04ae70eh/3763005198d Inode: 721664 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache)
Access: 2010-07-13 10:44:40.000000000 +1000
Modify: 2010-07-12 12:54:15.000000000 +1000
Change: 2010-07-12 12:54:52.000000000 +1000
# That's what is read by listTaskOutput
relfilename=build.log ATTR=st_atime GETATTR=1278981616
relfilename=build.log ATTR=st_blksize GETATTR=2097152
relfilename=build.log ATTR=st_blocks GETATTR=208
relfilename=build.log ATTR=st_ctime GETATTR=1278903292
relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\
relfilename=build.log ATTR=st_gid GETATTR=48
relfilename=build.log ATTR=st_ino GETATTR=721664
relfilename=build.log ATTR=st_mode GETATTR=33188
relfilename=build.log ATTR=st_mtime GETATTR=1278903255
relfilename=build.log ATTR=st_nlink GETATTR=1
relfilename=build.log ATTR=st_rdev GETATTR=0
relfilename=build.log ATTR=st_size GETATTR=106021
relfilename=build.log ATTR=st_uid GETATTR=48
Hope it helps, lost a good amount of time on that one :)
Cheers,
Thomas
12 years, 8 months
Strange mock build failure due to typo
by Paul Howarth
Hi,
today I have been preparing an update to perl-Math-Pari and came across
a very strange build failure whilst doing a local mock build on a Fedora
13 x86_64 host. My package built successfully on x86_64 but when I tried
to build for i386, the build failed during %setup but without any
diagnostics. The SRPM was installed but no attempt to install its build
requirements was made. The root.log showed an exit status of 0 for all
commands that had been run.
After much experimentation bisecting the changes I had made, I
discovered that a typo in the changelog entry was the culprit: I had set
the year to 2100 instead of 2010. So it would appear that somewhere in
the mock/yum/rpm stack there may be a year 2038 problem waiting to bite
us (though I suspect there may not be too many 32-bit builds happening
by then).
Seriously though, it would be nice to have better diagnostics for this
and perhaps an rpmlint check for changelog entries in the future?
Paul.
13 years, 4 months
[PATCH 1/3] Install build deps with yum-builddep.
by Ville Skyttä
No longer need to screen-scrape resolvedep and feed that to yum
install, and we have a chance to get BuildConflicts handing "for free"
(when RHBZ #614191 is done in yum(-builddep)).
---
mock.spec.in | 2 +-
py/mock/backend.py | 33 ++++++++++++++++++++++-----------
2 files changed, 23 insertions(+), 12 deletions(-)
diff --git a/mock.spec.in b/mock.spec.in
index e2580ca..0949975 100644
--- a/mock.spec.in
+++ b/mock.spec.in
@@ -18,7 +18,7 @@ Source: https://fedorahosted.org/mock/attachment/wiki/MockTarballs/%{name}-%{ver
URL: http://fedoraproject.org/wiki/Projects/Mock
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildArch: noarch
-Requires: python >= 2.4, yum >= 2.4, tar, pigz, python-ctypes, python-decoratortools, usermode
+Requires: python >= 2.4, yum >= 2.4, yum-utils >= 1.1.9, tar, pigz, python-ctypes, python-decoratortools, usermode
Requires: createrepo
Requires(pre): shadow-utils
BuildRequires: python-devel
diff --git a/py/mock/backend.py b/py/mock/backend.py
index ba0ad14..81a98d0 100644
--- a/py/mock/backend.py
+++ b/py/mock/backend.py
@@ -69,6 +69,7 @@ class Root(object):
self.chroot_file_contents = config['files']
self.chroot_setup_cmd = config['chroot_setup_cmd']
self.yum_path = '/usr/bin/yum'
+ self.yum_builddep_path = '/usr/bin/yum-builddep'
self.macros = config['macros']
self.more_buildreqs = config['more_buildreqs']
self.cache_topdir = config['cache_topdir']
@@ -444,23 +445,28 @@ class Root(object):
"""figure out deps from srpm. call yum to install them"""
try:
self.uidManager.becomeUser(0, 0)
+
+ def _yum_and_check(cmd):
+ output = self._yum(cmd, returnOutput=1)
+ for line in output.split('\n'):
+ if line.lower().find('No Package found for'.lower()) != -1:
+ raise mock.exception.BuildError, "Bad build req: %s. Exiting." % line
+
+ # first, install pre-existing deps and configured additional ones
arg_string = self.preExistingDeps
for hdr in mock.util.yieldSrpmHeaders(srpms, plainRpmOk=1):
# get text buildreqs
- a = mock.util.requiresTextFromHdr(hdr)
- b = mock.util.getAddtlReqs(hdr, self.more_buildreqs)
- for item in mock.util.uniqReqs(a, b):
+ for item in mock.util.getAddtlReqs(hdr, self.more_buildreqs):
arg_string = arg_string + " '%s'" % item
-
- # everything exists, okay, install them all.
- # pass build reqs (as strings) to installer
if arg_string != "":
- output = self._yum('resolvedep %s' % arg_string, returnOutput=1)
- for line in output.split('\n'):
- if line.lower().find('No Package found for'.lower()) != -1:
- raise mock.exception.BuildError, "Bad build req: %s. Exiting." % line
+ # everything exists, okay, install them all.
+ # pass build reqs (as strings) to installer
+ _yum_and_check('resolvedep %s' % arg_string)
# nothing made us exit, so we continue
self._yum('install %s' % arg_string, returnOutput=1)
+
+ # install actual build dependencies
+ _yum_and_check("builddep '%s'" % "' '".join(srpms))
finally:
self.uidManager.restorePrivs()
@@ -676,7 +682,12 @@ class Root(object):
if not self.online:
cmdOpts = "-C"
- cmd = '%s --installroot %s %s %s' % (self.yum_path, self.makeChrootPath(), cmdOpts, cmd)
+ # invoke yum-builddep instead of yum if cmd is builddep
+ exepath = self.yum_path
+ if cmd.startswith("builddep "):
+ exepath = self.yum_builddep_path
+ cmd = cmd[len("builddep "):]
+ cmd = '%s --installroot %s %s %s' % (exepath, self.makeChrootPath(), cmdOpts, cmd)
self.root_log.debug(cmd)
output = ""
try:
--
1.7.2.3
13 years, 5 months
Koji policies
by ed_pollard@us.ibm.com
I am attempting to setup koji in a test environment and would like to
allow users to add packages to tags without having admin rights. I found
http://fedoraproject.org/wiki/Koji/Policies
but have been unable to configure a policy for my purposes. Any help would
be greatly appreciated.
------
Edward Pollard
ed_pollard(a)us.ibm.com
13 years, 7 months
Mock alternating /etc/php.ini file
by David Hrbáč
Hi,
I have the following scenario. I'm trying to build some php-pear package
with Mock for CentOS 4.x. PHP settings within php.ini has memory_limit =
8MB. This memory limit is too low to build the package. I don't want to
change php package itself. I have tried:
a) package with sed to replace memory setting - fails with access denied
on /etc/php.ini
b) package containing own /etc/php.ini - conflicts with php-common
Any ideas? Thanks,
David Hrbáč
13 years, 7 months
cannot open Providename index using db3 - Invalid argument (22)
by Vijay N. Majagaonkar
Hi,
I am running out of strange problem at the time of creating ISO, ISO created
successfully but it failed at the time of OS installation and this what i
found in root.log file
I am not able to under stand the what is the problem "cannot open
Providename index using db3 - Invalid argument (22)"
can someone please give me some point or direction to fix this,
[Log]
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building
isolinux directory
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: mkdosfs 2.11 (12
Mar 2005)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb:
/var/lib/rpm/Providename: unsupported hash version: 9
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot
open Providename index using db3 - Invalid argument (22)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb:
/var/lib/rpm/Providename: unsupported hash version: 9
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot
open Providename index using db3 - Invalid argument (22)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
http://koji.fedora.com/packages/kernel/2.6.18/164.el5/x86_64/kernel-xen-2...:
[Errno 12] Timeout: <urlopen error timed out>
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Trying other
mirror.
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Traceback (most
recent call last):
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 293, in ?
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: util =
YumDownloader()
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 42, in __init__
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: self.main()
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 80, in main
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
self.downloadPackages(opts)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 214, in downloadPackages
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: path =
repo.getPackage(download)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 849, in getPackage
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: cache=cache
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 811, in _getFile
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: raise
Errors.RepoError, errstr
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
yum.Errors.RepoError: failed to retrieve
kernel/2.6.18/164.el5/x86_64/kernel-xen-2.6.18-164.el5.x86_64.rpm from
anaconda-extrarepo-1
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error was [Errno
12] Timeout: <urlopen error timed out>
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: kernel
(kernel-xen) doesn't exist for x86_64. skipping
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building
minstg.img
[/Log]
Thanks for the help
V!jay
13 years, 7 months
Mock Error
by Narendra Sisodiya
creating 32 bit iso file on 64 bit machine using pungi and mock. but
following error I am getting.
[root@narendra mock]# mock -r fedora-13-i386 --clean
INFO: mock.py version 1.1.5 starting...
State Changed: init plugins
INFO: selinux enabled
State Changed: start
State Changed: lock buildroot
State Changed: clean
[root@narendra mock]# mock -r fedora-13-i386 --init
INFO: mock.py version 1.1.5 starting...
State Changed: init plugins
INFO: selinux enabled
State Changed: start
State Changed: lock buildroot
State Changed: clean
State Changed: init
State Changed: lock buildroot
Mock Version: 1.1.5
INFO: Mock Version: 1.1.5
INFO: enabled root cache
State Changed: unpacking root cache
INFO: enabled yum cache
State Changed: cleaning yum metadata
INFO: enabled ccache
State Changed: running yum
ERROR: Command failed. See logs for output.
# ['/usr/sbin/groupadd', '-g', '478', 'mockbuild']
[root@narendra mock]#
--
┌─────────────────────────┐
│ Narendra Sisodiya
│ http://narendrasisodiya.com
└─────────────────────────┘
13 years, 7 months
Pungi Error
by Narendra Sisodiya
I am running fedora 13 64 bit system
I am trying to run pungi.
$pungi -c test.ks
I am getting this error
/usr/lib/python2.6/site-packages/pykickstart/parser.py:673:
DeprecationWarning: %packages does not end with %end. This syntax has been
deprecated. It may be removed from future releases, which will result in a
fatal error from kickstart. Please modify your kickstart file to use this
updated syntax.
warnings.warn(_("%s does not end with %%end. This syntax has been
deprecated. It may be removed from future releases, which will result in a
fatal error from kickstart. Please modify your kickstart file to use this
updated syntax.") % "%packages", DeprecationWarning)
Warning: Reusing existing destination directory.
Traceback (most recent call last):
File "/usr/bin/pungi", line 216, in <module>
main()
File "/usr/bin/pungi", line 86, in main
mypungi._inityum() # initialize the yum object for things that need it
File "/usr/lib/python2.6/site-packages/pypungi/__init__.py", line 166, in
_inityum
del self.ayum.prerepoconf
AttributeError: prerepoconf
--
┌─────────────────────────┐
│ Narendra Sisodiya
│ http://narendrasisodiya.com
└─────────────────────────┘
13 years, 7 months
Fwd: Creating Install media of fedora.
by Narendra Sisodiya
What are the tools available to install media. I want to create custom
fedora 13 install DVD (not live). Also, which fedora version or ubuntu
version you recommend to do this task,
--
┌─────────────────────────┐
│ Narendra Sisodiya
│ http://narendrasisodiya.com
└─────────────────────────┘
13 years, 7 months