I've just tagged and posted the 1.7.0 release of Koji. The most significant change is the move from mod_python to mod_wsgi, but there are a number of other changes also (it's been a while since 1.6.0).
I've written a document for migrating from 1.6.0 to 1.7.0. Look under:
docs/Migrating_to_1.7.txt
Here is the changelog entry: * Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1 - mod_wsgi support - mod_python support deprecated - kojiweb configuration file (web.conf) - split storage support (build volumes) - configurable resource limits (hub, web, and kojid) - drop pkgurl in favor of topurl - better approach to web themes - more helpful policy errors - clearer errors when rpc args do not match function signature - avoid retry errors on some common builder calls - don't rely on pgdb._quoteparams - avoid hosts taking special arch tasks they cannot handle - kojid: configure yum proxy - kojid: configure failed buildroot lifetime - kojid: literal_task_arches option - support for arm hardware floating point arches - maven build options: goals, envs, extra packages - store Maven build output under the standard build directory - make the list of files ignored in the local Maven repo configurable - add Maven information to taginfo - make kojira more efficient using multicalls and caching - speed up kojira startup - kojira: configurable sleep time - kojira: count untracked newRepo tasks towards limits - kojira: limit non-waiting newRepo tasks - gssapi support in the messagebus plugin - grant-permission --new - improved argument display for list-api command - moshimoshi - download task output directly from KojiFilesURL, rather than going through getfile - option to show buildroot data in rpminfo command - show search help on blank search command - wait-repo: wait for the build(s) to be the latest rather than just present
Hi Mike,
Mike McLean píše v Čt 31. 05. 2012 v 16:07 -0400:
I've just tagged and posted the 1.7.0 release of Koji. The most significant change is the move from mod_python to mod_wsgi, but there are a number of other changes also (it's been a while since 1.6.0).
I've written a document for migrating from 1.6.0 to 1.7.0. Look under:
docs/Migrating_to_1.7.txt
Here is the changelog entry:
- Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1
- mod_wsgi support
- mod_python support deprecated
- kojiweb configuration file (web.conf)
- split storage support (build volumes)
could you be more specific what does the split storage means? Will I be able to easily combine 2 or more storages (NFS mounts, local disks, etc) and they will create one big storage for koji usage?
Dan
On 05/31/2012 04:21 PM, Dan Horák wrote:
could you be more specific what does the split storage means? Will I be able to easily combine 2 or more storages (NFS mounts, local disks, etc) and they will create one big storage for koji usage?
It does not quite work that way, and 'easy' is not a word I would use here. I DO NOT recommend that folks use this feature unless they absolutely have to. It is much better to simply buy a bigger disc if at all possible.
WARNING. Using split storage is potentially dangerous. If you do not manage your mount points carefully you could cause difficult to debug problems or even CORRUPT your file store.
That being said. Here is how it works.
Each build now has a volume attribute that indicates which volume it is stored on. The default volume is named 'DEFAULT' and corresponds to the same storage paths under /mnt/koji that we have always used.
Additional volumes can be set up by creating the directory /mnt/koji/vol/NAME (where NAME is the name of the volume). The expectation is that this will be a mount point, but it doesn't have to be.
The new volume directory should initially contain a packages/ subdirectory, and the permissions should be the same as the default packages directory.
Assuming you do use a mount for a vol/NAME directory, you will want to ensure that the same mounts are created on any builders in the createrepo group, any hosts running kojira or similar maintenance, and any hosts that rely on the topdir option rather than the topurl option.
Once you have the directory set up, you can tell Koji about it by running 'koji add-volume NAME' (which will fail if the hub can't find the directory). You can get a list of known volumes with the 'koji list-volumes' command, and you can move a build to a different volume with the 'koji set-build-volume' command. Like all koji subcommands, these have a --help option.
Moving a build across volumes will cause kojira to trigger repo regenerations, if appropriate. When the volume is not DEFAULT, koji will create a relative symlink to the new build directory on the default volume. Moving builds across volumes may immediately break repos (until the regen occurs), so use caution.
Policies involving builds (e.g. gc policy, tag policy), can test a build's volume (by name) with the 'volume' test.
That's pretty much it. It is a very simplistic system. There is no automation. It is up to the administrator to manage the mount points and to manage which builds are stored on which volumes. All new builds are stored on the default volume until they are moved elsewhere.
As an additional datum, I'm happy to report that the shop's koji instance seems to be running perfectly for several days after an update to the new Fedora 16 RPMs (from shop-built RPMs of HEAD from a few months ago) and a reasonably easy switch to the mod_wsgi configuration.
We'll be migrating our build system over to EL6 soon. Crossing my fingers that everything goes smoothly there too!
John
On 05/31/2012 03:07 PM, Mike McLean wrote:
I've just tagged and posted the 1.7.0 release of Koji. The most significant change is the move from mod_python to mod_wsgi, but there are a number of other changes also (it's been a while since 1.6.0).
I've written a document for migrating from 1.6.0 to 1.7.0. Look under:
docs/Migrating_to_1.7.txt
Here is the changelog entry:
- Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1
- mod_wsgi support
- mod_python support deprecated
- kojiweb configuration file (web.conf)
- split storage support (build volumes)
- configurable resource limits (hub, web, and kojid)
- drop pkgurl in favor of topurl
- better approach to web themes
- more helpful policy errors
- clearer errors when rpc args do not match function signature
- avoid retry errors on some common builder calls
- don't rely on pgdb._quoteparams
- avoid hosts taking special arch tasks they cannot handle
- kojid: configure yum proxy
- kojid: configure failed buildroot lifetime
- kojid: literal_task_arches option
- support for arm hardware floating point arches
- maven build options: goals, envs, extra packages
- store Maven build output under the standard build directory
- make the list of files ignored in the local Maven repo configurable
- add Maven information to taginfo
- make kojira more efficient using multicalls and caching
- speed up kojira startup
- kojira: configurable sleep time
- kojira: count untracked newRepo tasks towards limits
- kojira: limit non-waiting newRepo tasks
- gssapi support in the messagebus plugin
- grant-permission --new
- improved argument display for list-api command
- moshimoshi
- download task output directly from KojiFilesURL, rather than going
through getfile
- option to show buildroot data in rpminfo command
- show search help on blank search command
- wait-repo: wait for the build(s) to be the latest rather than just
present
buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
buildsys@lists.fedoraproject.org
-
Dan Horák -
John Morris -
Mike McLean