koji: add_external_rpm() unique constraint violation
Here's the post-mortem for an exception in koji-hub (EPEL6 1.8.0-2 RPM).
Earlier, I did something weird, and built a kernel RPM with some pieces
disabled, including the 'kernel-headers' subpackage. Builds following
the repo regeneration failed, since the incomplete kernel package build
overrode other kernel packages, including subpackages; 'kernel-headers'
is a dependency of build root packages. The incomplete kernel package
was removed to the trashcan and repos regenerated.
The next build of another kernel package resulted in the exception
pasted below.
*** My best guess at the problem:
Kernel packages have both 'i686' and 'noarch' builds that run
simultaneously.
The i686 build inserted the 'kernel-headers' row into the 'rpminfo'
table, along with a related row in 'buildroot_listing'.
At the same time, the noarch build attempted to do the same thing, but
the i686 build won the race. The new row broke uniqueness constraints,
(NVRAs must be unique within an external repo), and hence the exception.
If these two builds' add_external_rpm() calls hadn't overlapped, the
second would have created its 'buildroot_listing' row referencing the
'rpminfo' row that the first created.
*** Estimate of problem gravity:
This overlap is probably rarely seen because (1) most packages have
either 'noarch' builds or arch-specific builds, but not both; (2) this
happened to be the first build after the repos were regenerated after
addition of a new package; and (3) even if the first two conditions are
met, it is unlikely that the two builds happened to run
add_external_rpm() at exactly the same time.
The exception stops the build, but no other harm is done. Rebuilding
will not repeat this exception, since the row is created and the race
condition does not exist.
John
2013-12-20 13:43:56,876 [WARNING] m=host.setBuildRootList
u=infra0.zultron.com p=25442 \
r=192.168.7.29:47734 koji.xmlrpc: Traceback (most recent call last):
File "/usr/share/koji-hub/kojixmlrpc.py", line 212, in _wrap_handler
response = handler(environ)
File "/usr/share/koji-hub/kojixmlrpc.py", line 255, in handle_rpc
return self._dispatch(method, params)
File "/usr/share/koji-hub/kojixmlrpc.py", line 292, in _dispatch
ret = koji.util.call_with_argcheck(func, params, opts)
File "/usr/lib/python2.6/site-packages/koji/util.py", line 139, in
call_with_argcheck
return func(*args, **kwargs)
File "/usr/share/koji-hub/kojihub.py", line 10237, in setBuildRootList
return br.setList(rpmlist)
File "/usr/share/koji-hub/kojihub.py", line 9371, in setList
self._setList(rpmlist,update=False)
File "/usr/share/koji-hub/kojihub.py", line 9353, in _setList
data = add_external_rpm(an_rpm, location, strict=False)
File "/usr/share/koji-hub/kojihub.py", line 4590, in add_external_rpm
_dml(q, rpminfo)
File "/usr/share/koji-hub/kojihub.py", line 3985, in _dml
c.execute(operation, values)
File "/usr/lib/python2.6/site-packages/koji/db.py", line 105, in execute
ret = self.cursor.execute(operation, parameters)
File "/usr/lib64/python2.6/site-packages/pgdb.py", line 174, in execute
self.executemany(operation, (params,))
File "/usr/lib64/python2.6/site-packages/pgdb.py", line 195, in executemany
raise DatabaseError, "error '%s' in '%s'" % ( msg, sql )
DatabaseError: error 'ERROR: duplicate key value violates unique constraint
"rpminfo_unique_nvra"
' in 'INSERT INTO rpminfo (id, build_id, buildroot_id,
name, version, release, epoch, arch,
external_repo_id,
payloadhash, size, buildtime)
VALUES (10579, NULL, NULL,
'kernel-headers', '3.11.10', '200.fc19', NULL,
'i686',
21,
'1e6b65e55f166cb40dff21842941be2b', 3165695, 1386020554)
'