Koji 1.7.0 release

[Mike McLean]

I've just tagged and posted the 1.7.0 release of Koji. The most 
significant change is the move from mod_python to mod_wsgi, but there 
are a number of other changes also (it's been a while since 1.6.0).

I've written a document for migrating from 1.6.0 to 1.7.0. Look under:

   docs/Migrating_to_1.7.txt

Here is the changelog entry:
* Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1
- mod_wsgi support
- mod_python support deprecated
- kojiweb configuration file (web.conf)
- split storage support (build volumes)
- configurable resource limits (hub, web, and kojid)
- drop pkgurl in favor of topurl
- better approach to web themes
- more helpful policy errors
- clearer errors when rpc args do not match function signature
- avoid retry errors on some common builder calls
- don't rely on pgdb._quoteparams
- avoid hosts taking special arch tasks they cannot handle
- kojid: configure yum proxy
- kojid: configure failed buildroot lifetime
- kojid: literal_task_arches option
- support for arm hardware floating point arches
- maven build options: goals, envs, extra packages
- store Maven build output under the standard build directory
- make the list of files ignored in the local Maven repo configurable
- add Maven information to taginfo
- make kojira more efficient using multicalls and caching
- speed up kojira startup
- kojira: configurable sleep time
- kojira: count untracked newRepo tasks towards limits
- kojira: limit non-waiting newRepo tasks
- gssapi support in the messagebus plugin
- grant-permission --new
- improved argument display for list-api command
- moshimoshi
- download task output directly from KojiFilesURL, rather than going 
through getfile
- option to show buildroot data in rpminfo command
- show search help on blank search command
- wait-repo: wait for the build(s) to be the latest rather than just present

[Dan Horák]

Hi Mike,

Mike McLean píše v Čt 31. 05. 2012 v 16:07 -0400: 
>  I've just tagged and posted the 1.7.0 release of Koji. The most 
>  significant change is the move from mod_python to mod_wsgi, but there 
>  are a number of other changes also (it's been a while since 1.6.0).
>  
>  I've written a document for migrating from 1.6.0 to 1.7.0. Look under:
>  
>     docs/Migrating_to_1.7.txt
>  
>  Here is the changelog entry:
>  * Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1
>  - mod_wsgi support
>  - mod_python support deprecated
>  - kojiweb configuration file (web.conf)
>  - split storage support (build volumes) 
could you be more specific what does the split storage means? Will I be
able to easily combine 2 or more storages (NFS mounts, local disks, etc)
and they will create one big storage for koji usage?


Dan

[Mike McLean]

On 05/31/2012 04:21 PM, Dan Horák wrote:
>  could you be more specific what does the split storage means? Will I
> be
>  able to easily combine 2 or more storages (NFS mounts, local disks, etc)
>  and they will create one big storage for koji usage? 
It does not quite work that way, and 'easy' is not a word I would use 
here. I DO NOT recommend that folks use this feature unless they 
absolutely have to. It is much better to simply buy a bigger disc if at 
all possible.

WARNING. Using split storage is potentially dangerous. If you do not 
manage your mount points carefully you could cause difficult to debug 
problems or even CORRUPT your file store.

That being said. Here is how it works.

Each build now has a volume attribute that indicates which volume it is 
stored on. The default volume is named 'DEFAULT' and corresponds to the 
same storage paths under /mnt/koji that we have always used.

Additional volumes can be set up by creating the directory 
/mnt/koji/vol/NAME (where NAME is the name of the volume). The 
expectation is that this will be a mount point, but it doesn't have to be.

The new volume directory should initially contain a packages/ 
subdirectory, and the permissions should be the same as the default 
packages directory.

Assuming you do use a mount for a vol/NAME directory, you will want to 
ensure that the same mounts are created on any builders in the 
createrepo group, any hosts running kojira or similar maintenance, and 
any hosts that rely on the topdir option rather than the topurl option.

Once you have the directory set up, you can tell Koji about it by 
running 'koji add-volume NAME' (which will fail if the hub can't find 
the directory). You can get a list of known volumes with the 'koji 
list-volumes' command, and you can move a build to a different volume 
with the 'koji set-build-volume' command. Like all koji subcommands, 
these have a --help option.

Moving a build across volumes will cause kojira to trigger repo 
regenerations, if appropriate. When the volume is not DEFAULT, koji will 
create a relative symlink to the new build directory on the default 
volume. Moving builds across volumes may immediately break repos (until 
the regen occurs), so use caution.

Policies involving builds (e.g. gc policy, tag policy), can test a 
build's volume (by name) with the 'volume' test.

That's pretty much it. It is a very simplistic system. There is no 
automation. It is up to the administrator to manage the mount points and 
to manage which builds are stored on which volumes. All new builds are 
stored on the default volume until they are moved elsewhere.

[John Morris]

As an additional datum, I'm happy to report that the shop's koji 
instance seems to be running perfectly for several days after an update 
to the new Fedora 16 RPMs (from shop-built RPMs of HEAD from a few 
months ago) and a reasonably easy switch to the mod_wsgi configuration.

We'll be migrating our build system over to EL6 soon.  Crossing my 
fingers that everything goes smoothly there too!

	John


On 05/31/2012 03:07 PM, Mike McLean wrote:
>  I've just tagged and posted the 1.7.0 release of Koji. The most
>  significant change is the move from mod_python to mod_wsgi, but there
>  are a number of other changes also (it's been a while since 1.6.0).
>
>  I've written a document for migrating from 1.6.0 to 1.7.0. Look under:
>
>  docs/Migrating_to_1.7.txt
>
>  Here is the changelog entry:
>  * Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1
>  - mod_wsgi support
>  - mod_python support deprecated
>  - kojiweb configuration file (web.conf)
>  - split storage support (build volumes)
>  - configurable resource limits (hub, web, and kojid)
>  - drop pkgurl in favor of topurl
>  - better approach to web themes
>  - more helpful policy errors
>  - clearer errors when rpc args do not match function signature
>  - avoid retry errors on some common builder calls
>  - don't rely on pgdb._quoteparams
>  - avoid hosts taking special arch tasks they cannot handle
>  - kojid: configure yum proxy
>  - kojid: configure failed buildroot lifetime
>  - kojid: literal_task_arches option
>  - support for arm hardware floating point arches
>  - maven build options: goals, envs, extra packages
>  - store Maven build output under the standard build directory
>  - make the list of files ignored in the local Maven repo configurable
>  - add Maven information to taginfo
>  - make kojira more efficient using multicalls and caching
>  - speed up kojira startup
>  - kojira: configurable sleep time
>  - kojira: count untracked newRepo tasks towards limits
>  - kojira: limit non-waiting newRepo tasks
>  - gssapi support in the messagebus plugin
>  - grant-permission --new
>  - improved argument display for list-api command
>  - moshimoshi
>  - download task output directly from KojiFilesURL, rather than going
>  through getfile
>  - option to show buildroot data in rpminfo command
>  - show search help on blank search command
>  - wait-repo: wait for the build(s) to be the latest rather than just
>  present
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys