Revisor / yum oddity with package conflicts...
by Martin Langhoff
With the exact same commandline, roughly 50% of the time I get a
conflict betweem generic-logos and fedora-logos. This happens with
stock F9 revisor from rpm and the tip of the F-9 branch in git.
Nothing whatsoever requires fedora-logos.
Sometimes it works, sometimes it doesn't. !?
$ sudo revisor --cli --config revisor/revisor.conf \
--kickstart=revisor/anaconda-f9.ks \
--kickstart-include --kickstart-default \
--install-unified --model 'xs-f9-i386' --yes \
--rebrand-name=xs
SELinux on this host is disabled. Composed media will not have
SELinux, and as a result the system you install from the composed
media will not have SELinux either.
Loading Repositories:
######################################## 100.0%
Select kickstart packages:
######################################## 100.0%
Resolving Dependencies:
######################################## 100.0%
generic-logos-9.0.0-1.fc9.noarch from fedora has depsolving problems
--> generic-logos conflicts with fedora-logos
Resolving Dependencies:
######################################## 100.0%
Unable to resolve dependencies for some packages selected:
generic-logos conflicts with fedora-logos
*nothing* requires fedora-logos.
Calling revisor with debug=5 shows...
Adding required package kernel-0:2.6.25-14.fc9.i686
Adding required package kernel-xen-0:2.6.25-2.fc9.i686
Checking dependencies - not allowing any conflicts within the package set
Resolving Dependencies:
0.0% Removing package fedora-release-0:9-2.noarch for rebranding
Removing package fedora-release-notes-0:9.0.0-1.noarch for rebranding
Removing package fedora-logos-0:9.0.0-2.fc9.noarch for rebranding
Resolving Dependencies:
######################################## 100.0%
generic-logos-9.0.0-1.fc9.noarch from fedora has depsolving problems
--> generic-logos conflicts with fedora-logos
Resolving Dependencies:
######################################## 100.0%
Unable to resolve dependencies for some packages selected:
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
15 years, 7 months
Assembling packages for ISO's
by Bryce
Can anyone think of a more succinct means to drag out all the latest
rpms based on a tag?
I could only came up with this horror.
mkdir pool; cd pool
TAG=f9-bryced
koji list-pkgs --tag=${TAG} --quiet | cut -f 1 -d " " | xargs koji latest-pkg ${TAG} --quiet | cut -f1 -d " " | xargs -n1 koji download-build --arch=i386 --arch=i686 --arch=noarch
Phil
=--=
15 years, 7 months
Patch fixing a problem with --kickstart-include
by Martin Langhoff
By naming the kickstart file as ks.cfg, anaconda would _always_ take
it, regardless of kernel boot options. This is not what was expected -
it is safer to give it a different name, and then use the boot menu
item to select it.
The patch is on top if F-9 .
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
15 years, 7 months
[PATCH] Perforce support for koji
by Paul B Schroeder
The powers that be require us to use Perforce. Thus, the patch. A few
things to note about it:
* Our P4 server is running on a non-standard port and thus needs to be
specified. scm_tuple is now created via rsplit instead. Allows SCM
port to be specified. i.e. This now works:
allowed_scms=perforcetx.bluecoat.com:1999:/koji/sandbox/paul.schroeder/skeletor
* P4 *requires* a stinkin' password. So I added password support to
_parse_url. The "user" part of the SCM url can now be "user:password".
Still works minus the ":password" too. Let me know if there's a
better/preferred way to support passwords.
* P4 can only checkout into the static client root specified in the P4
client spec (PITA). Sooo.. After the module_checkout_cmd is executed,
if sourcedir does not exist, we do a 'os.rename' of the checked out code
into the proper task sourcedir.
* A P4 SCM URL looks like:
p4://user:pass@p4port/p4client/p4depot?path/to/module#changelist
i.e. p4port==host And p4client is your P4 client spec that you've
created.
Let me know if there are any Qs, comments, or any changes I should
make..
Cheers...Paul...
diff --git a/builder/kojid b/builder/kojid
index 1a7afbf..92f8143 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -1830,7 +1830,8 @@ class BuildSRPMFromSCMTask(BaseTaskHandler):
use_common = True
for allowed_scm in options.allowed_scms.split():
- scm_tuple = allowed_scm.split(':')
+ # Use rsplit with 1 max split to allow for port numbers w/
scm host
+ scm_tuple = allowed_scm.rsplit(':', 1)
if len(scm_tuple) in (2, 3):
if fnmatch(scm.host, scm_tuple[0]) and
fnmatch(scm.repository, scm_tuple[1]):
# SCM host:repository is in the allowed list
@@ -2325,7 +2326,8 @@ class SCM(object):
'GIT': ('git://', 'git+http://', 'git+https://', 'git
+rsync://'),
'GIT+SSH': ('git+ssh://',),
'SVN': ('svn://', 'svn+http://', 'svn+https://'),
- 'SVN+SSH': ('svn+ssh://',) }
+ 'SVN+SSH': ('svn+ssh://',),
+ 'P4': ('p4://',) }
def is_scm_url(url):
"""
@@ -2362,10 +2364,11 @@ class SCM(object):
raise koji.GenericError, 'Invalid SCM URL: %s' % url
self.url = url
- scheme, user, host, path, query, fragment = self._parse_url()
+ scheme, user, password, host, path, query, fragment =
self._parse_url()
self.scheme = scheme
self.user = user
+ self.password = password
self.host = host
self.repository = path
self.module = query
@@ -2384,9 +2387,9 @@ class SCM(object):
Parse the SCM url into usable components.
Return the following tuple:
- (scheme, user, host, path, query, fragment)
+ (scheme, user, password, host, path, query, fragment)
- user may be None, everything else will have a value
+ user and password may be None, everything else will have a
value
"""
# get the url's scheme
scheme = self.url.split('://')[0] + '://'
@@ -2396,14 +2399,21 @@ class SCM(object):
dummyscheme, netloc, path, params, query, fragment =
urlparse.urlparse(dummyurl)
user = None
+ password = None
userhost = netloc.split('@')
if len(userhost) == 2:
- user = userhost[0]
+ userpass = userhost[0].split(':')
+ user = userpass[0]
+ if len(userpass) == 2:
+ password = userpass[1]
+ elif len(userpass) > 2:
+ raise koji.GenericError, 'Invalid username:password
specified: %s' % netloc
if not user:
# Don't return an empty string
user = None
- elif ':' in user:
- raise koji.GenericError, 'username:password format not
supported: %s' % user
+ if not password:
+ # Don't return an empty string
+ password = None
netloc = userhost[1]
elif len(userhost) > 2:
raise koji.GenericError, 'Invalid username@hostname
specified: %s' % netloc
@@ -2419,7 +2429,7 @@ class SCM(object):
raise koji.GenericError, 'Unable to parse SCM URL: %s' %
self.url
# return parsed values
- return (scheme, user, netloc, path, query, fragment)
+ return (scheme, user, password, netloc, path, query, fragment)
def checkout(self, scmdir, uploadpath, logfile, use_common=False):
"""
@@ -2514,6 +2524,39 @@ class SCM(object):
module_checkout_cmd = ['svn', 'checkout', '-r',
self.revision, '%s/%s' % (svnserver, self.module), self.module]
common_checkout_cmd = ['svn', 'checkout', '%s/common' %
svnserver]
+ elif self.scmtype == 'P4':
+ if not self.user:
+ raise koji.BuildError, 'No user specified for
repository access scheme: %s' % self.scheme
+ # P4 URL:
+ #
p4://user:pass@p4port/p4client/p4depot?path/to/module#changelist
+
+ # Perforce uses a "client workspace"
+ # Before first slash is the p4 client, after is the p4
depot
+ (p4client, p4depot) = self.repository[1:].split('/', 1)
+
+ p4client_root = None
+ p4info = os.popen('p4 -p ' + self.host + ' -c ' + p4client
+ ' info')
+ for line in p4info:
+ if line.startswith('Client root:'):
+ p4client_root = line.split()[-1]
+ break
+ p4info.close()
+ if not p4client_root:
+ raise koji.BuildError, 'Could not find perforce client
root'
+
+ # Perforce cannot checkout to a random dir (in this case
scmdir).
+ # It will only checkout to the p4client_root, so we
override scmdir
+ # with the p4client_root
+ scmdir = '%s/%s' % (p4client_root, p4depot)
+
+ # When HEAD is given as the revision, we assume we want the
latest
+ p4revision = ''
+ if self.revision.upper() != 'HEAD':
+ p4revision = '@%s' % self.revision
+
+ module_checkout_cmd = ['p4', '-p', self.host, '-u',
self.user, '-P', self.password, '-c', p4client, 'sync', '-f', '//%s/%
s/...%s' % (p4depot, self.module, p4revision)]
+ common_checkout_cmd = ['p4', '-p', self.host, '-u',
self.user, '-P', self.password, '-c', p4client, 'sync', '-f', '//%
s/common/...' % p4depot]
+
else:
raise koji.BuildError, 'Unknown SCM type: %s' %
self.scmtype
@@ -2521,6 +2564,10 @@ class SCM(object):
if log_output(module_checkout_cmd[0], module_checkout_cmd,
logfile, uploadpath, cwd=scmdir, logerror=1, env=env):
raise koji.BuildError, 'Error running %s checkout command
"%s", see %s for details' % \
(self.scmtype, ' '.join(module_checkout_cmd),
os.path.basename(logfile))
+ # Perforce checks out to the client root listed for the
perforce client.
+ # Need to move checked out source into sourcedir.
+ if not os.path.exists(sourcedir):
+ os.rename('%s/%s' % (scmdir, self.module), sourcedir)
if update_checkout_cmd:
# Currently only required for GIT checkouts
15 years, 7 months
Revisor - package not found and kickstart options change dep error tolerance?
by Martin Langhoff
Hi Jeroen,
I am trying to build a revisor-based School Server installer, and I am
finding some oddities
- As I mentioned before, it complains about a missing dependency for
a package that is right there, in the same repo as other packages it
is picking up.
- If I pass --kickstart-include --kickstart-default the missing
dependency is fatal. If I don't, it shows the warning and builds the
image...
both quite strange problems... I'm jumping into the src right now to
see if I can find out more, thought I'd mention it... btw,
http://lists.fedoraunity.org/mailman/listinfo/revisor-devel is still
unreachable...
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
15 years, 7 months
Pungi kickstart packages - stage2 deps vs instaled pkgs
by Martin Langhoff
Perhaps this is by design: the "build tools" dependencies
(anaconda-runtime an friends) which end up in stage2 AFAICS need to be
listed (and their dependencies met) in the main packages listing in
the kickstart file.
Is there a way to control those package listings separately? As a
(trivial) example, we end up with an anaconda-runtime rpm in the iso,
which we are not interested in...
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
15 years, 7 months
Pungi / anaconda: seeding the pkg selection
by Martin Langhoff
The installer cd I'm trying to build (for the School Server) is driven
by 2 custom packages - a metapackage that pulls in all the
dependenciess, and a configuration package that whacks /etc .
Feeding those to Pungi via a kickstart file gets them on the installer
CD, but does not get them installed by default. My overall goals (and
stumbling blocks) are:
1 - Support interactive installs (that defaults to installing the XS
packages!) for developers & pilot sites. This should be the default on
the grub menu...
- How do get anaconda to pick those 2 custom packages by default?
Using a kickstart file is not an option - it switches off the
interactive install mode. Is anaconda auto-installing the 'base' group
and should I add them there? Should I create a new group in comps.xml
and get anaconda to have it selected by default?
- How can I feed Pungi a custom comps.xml?
1.1 - The interactive install should have a custom 'default' layout
for the autopartitioning. Can I seed that somehow?
2 - Support a fully automated kickstart install (non-default option
in grub menu). This is also an example for local teams to edit to
their needs...
- Does Pungi support a %post that can "copy in" a new grub menu to
the isolinux/syslinux directory?
- ... and add a ks.cfg file to the initrd?
- Will anaconda probe for ks.cfg in any directories that are easier
to edit than the initrd?
lots of questions :-/
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
15 years, 7 months
koji-1.2.6
by Mike McLean
It's been a long time since we posted a fresh tarball, so I just tagged
and posted 1.2.6. Following is a summary of changes since 1.2.5. A
couple of them /may/ cause some slight upgrade issues.
- kojid no longer sleeps after taking a task. This shouldn't be too much
of a problem, but it is a behaviour change and may affect your load
distribution.
- pathinfo.taskrelpath. The upload path for task workfiles has changed
to avoid creating over-large directories. This may cause older clients
to report broken urls. Updated clients, of course, will report the
correct ones.
- unified boolean option parsing in kojihub. This is a bugfix, but may
expose configuration errors.
- ServerOffline fault. This is a new fault code, and older clients will
not be able to interpret it. Howevever, in the case where the exception
is intended to be used this should be no worse than previous behavior.
(mostly) full change summary:
• don't sleep after taking a task
• pathinfo.taskrelpath
• unified waitrepo logic
• scm user_common flag
• shorter waitrepo timeout
• support building from git subdirs
• robust sessions for kojid
∘ new config options: retry_interval, max_retries, offline_retry,
offline_retry_interval
• authtype option for koji cli
• top level quiet option for koji cli
• select config file section based on invocation name
• larger blocksize for uploads
• prune-signed-copies improvements
• saner duplicate filtering in list-pkgs
• expand group deps in generated comps
• several new options for download-build
• wait-repo cli command
• advertise help --admin
• support db password auth in kojihub
• listTagged/listTaggedRPMS by owner
• handle importing an srpm for an existing build
• unified boolean option parsing in kojihub
• alias system.multiCall as system.multicall
• ServerOffline exception
∘ raised when db connect fails
∘ also ServerOffline hub config option
• fix Source: line in spec
• package koji-gc (in -utils)
• changed default config location for koji-gc
• gc: fix age calculation for untagged builds
• option to provide a ts for get_rpm_header
• set rpmbuild_timeout of 24hours in mock configs
• more robust ClientSession connections
∘ better retries
∘ retries for anonymous sessions
∘ special treatement for ServerOffline case
∘ don't mask real exceptions
• use _callMethod in ClientSession.multiCall
• larger default blocksize in uploadWrapper
• automatically refresh stale db connections
• kojira: don't throw an exception if a tag is renamed or deleted
• add new session options for kojira:
∘ retry_interval, max_retries, offline_retry, offline_retry_interval
• kojiweb:
∘ handle exceptions gracefully
∘ allow rss feed to filter by package name (specified with package=)
∘ allow listTagged* and readTagged* to filter by package owner name
(specified with owner=)
∘ allow the builds page in the web UI to filter by package ID
(specified with packageID=)
∘ only send the session cookie over a secure connection (patch from
Toshio Kuratomi, with modifications)
∘ remove the KojiWebURL config option (no longer necessary)
∘ sort rpm listing on the buildinfo page by arch and -debuginfo
∘ add links to the headers on the info pages
∘ put identification information into the title on the *info pages
∘ change the title format to put the important bits first (format
suggested by Will Woods)
∘ put the task label into the taskinfo header
∘ move the task ID into the main table
15 years, 7 months
